Controlling the pencil mode programmatically

Another interesting question in the newsgroups from a David Kelley recently set me athinking!  So, I thought I’d share my response here.

First, the question:

“Programmatically,  after I drop a new shape on the page, how can I then move into the "editing mode" tool to use the pencil (to allow moving the control points on a polygon).  And once this is done (and the user has moved all the points to be where they should) how can I detect that every point has been edited/moved so that I can then automatically lock the polygon shape into place and prevent further movement (I understand the lock, just not how to detect the event)?   My general need is to allow the user to refine the corner points to match an underlaying image before (in the next step) I do some math on the final position.”

I thought the first part, getting the Pencil Edit mode to become active, was going to be easy, but it turns out not so…  Every shape has an EventDrop cell, in which you can add ShapeSheet formula.  So, I initially entered =DOCMD(1221) because 1221 is the value of Visio.VisUICmds.visCmdDRLineTool.  Therefore, my theory was, that this would change the mode to Pencil Edit whenever a copy of the shape was dropped.  Not so.  It never changes out of Pointer mode.  However, if I wrapped the command in a method that can be called from CALLTHIS() then it does work!

I drew a simple rectangle, and added it as a new Master in my local document stencil, then renamed it as Mask.  I then edited the Master shape, opened the ShapeSheet of the rectnagle, and entered the formula =CALLTHIS(“PencilEdit”) into the EventDrop cell, and added the PencilEdit method to a module in the documents VBA project:

   1: Public Sub PencilEdit(ByVal shp As Visio.Shape)
   2:     Visio.Application.DoCmd Visio.visCmdDRLineTool
   3: End Sub

Okay, so now Visio automatically enters Pencil Edit mode when the Master is dropped on the page or copied.

Now, the second bit … how to detect if each vertex has been edited.  In the following screenshot, you can see that the PinX and PinY values, of the shape instance of my Mask Master, are blue, but all the rest are black.  This is because they are the only values that are not inherited from the Master shape.


If I change the size of the shape by dragging a corner, then you can see that the Width and Height values also become blue because they are no longer inherited:


If I were to move a vertex within the rectangle, then you can see that two of the LineTo rows in the Geometry 1 section become local, ie not inherited, and turn blue.


If I were to move a vertex outside of the original rectangle, then all of the rows become local.

imageAll of the points in a Visio ShapeSheet are measured relative to the bottom left corner of the enclosing rectangle.  In this case, the width and height of the shape has also changed, thus the formulae in the X and Y cells have to change, even if they need to maintain their absolute position, because they are relative to the Width or Height values.  For example, the second row had the formula Width*1 originally, when the Width was equal to 30 mm.  Now that the width is 35 mm, the formula has changed to Width*0.8571, so that the value can stay at 30 mm.  In other words, the vertex in the bottom right corner of the shape has not moved, but its formula has had to be updated just to stay where it is.

If I were to add (or delete) a vertex into the geometry, then again, all of the Geometry 1 section values become local.

imageNo vertex has been moved, but one has been added on the top edge, on the left side.

Therefore, it is very difficult to detect if every vertex has been adjusted since it was originally dropped on to the page.  In practice, I would probably position the bottom left corner over a required vertex position, and then alter all of the other vertices to suit.  Even adding or deleting vertices as required.  Consequently, row 1 may never be adjusted at all.

David wants to detect vertex changes, but the nearest event in the ShapeSheet is the EventXFMod cell, so I added a CALLTHIS(“CheckEdit”) formula in there.  This will fire whenever the Width, Height, PinX or PinY changes.  It will not fire if a vertex movement does not cause a resizing of the shape.


The CheckEdit method will loop through all of the vertices in the first geometry section, and check if the vertex is in a different position to the same vertex in the master shape.  If all of the vertices have been moved relative to the previous vertex, then Visio is put back into Pointer Tool mode, and the vertices on the shape are locked from further editing.

   1: Public Sub CheckEdit(ByVal shp As Visio.Shape)
   2: 'Abort if no Geometry section
   3:     If shp.SectionExists(Visio.visSectionFirstComponent, Visio.VisExistsFlags.visExistsAnywhere) = 0 Then
   4:         Exit Sub
   5:     End If
   6: 'Abort if no master
   7:     If shp.Master Is Nothing Then
   8:         Exit Sub
   9:     End If
  11: Dim iSect As Integer
  12: Dim iRow As Integer
  13: Dim iCell As Integer
  14: Dim rowIsChanged As Boolean
  15: Dim iCountUnChangedRows As Integer
  16: Dim segmentAngleMaster As Double
  17: Dim segmentLengthMaster As Double
  18: Dim segmentAngle As Double
  19: Dim segmentLength As Double
  20: Dim deltaX As Double
  21: Dim deltaY As Double
  23:     iSect = Visio.VisSectionIndices.visSectionFirstComponent
  24:     iRow = Visio.VisRowIndices.visRowComponent
  25:     'Ignore the first unchanged row because it may be left intact
  26:     iCountUnChangedRows = 0
  27:     For iRow = 1 To shp.RowCount(Visio.visSectionFirstComponent) - 1
  28:         rowIsChanged = False
  29:         If shp.CellsSRC(iSect, iRow, iCell).IsInherited Then
  30:             iCountUnChangedRows = iCountUnChangedRows + 1
  31:             If iCountUnChangedRows > 1 Then
  32:                 Exit For
  33:             End If
  34:         End If
  35:         If shp.MasterShape.CellsSRCExists(iSect, iRow, 0, Visio.VisExistsFlags.visExistsAnywhere) Then
  36:             'Check that the vertex has moved relative to the previous vertex
  37:             iCell = Visio.VisCellIndices.visX
  38:             deltaX = shp.MasterShape.CellsSRC(iSect, iRow, iCell).ResultIU - shp.MasterShape.CellsSRC(iSect, iRow - 1, iCell).ResultIU
  39:             iCell = Visio.VisCellIndices.visY
  40:             deltaY = shp.MasterShape.CellsSRC(iSect, iRow, iCell).ResultIU - shp.MasterShape.CellsSRC(iSect, iRow - 1, iCell).ResultIU
  41:             segmentLengthMaster = Sqr((deltaX ^ 2) + (deltaY ^ 2))
  42:             If deltaX = 0 Then
  43:                 segmentAngleMaster = 0
  44:             Else
  45:                 segmentAngleMaster = Math.Atn(deltaY / deltaX)
  46:             End If
  48:             iCell = Visio.VisCellIndices.visX
  49:             deltaX = shp.CellsSRC(iSect, iRow, iCell).ResultIU - shp.CellsSRC(iSect, iRow - 1, iCell).ResultIU
  50:             iCell = Visio.VisCellIndices.visY
  51:             deltaY = shp.CellsSRC(iSect, iRow, iCell).ResultIU - shp.CellsSRC(iSect, iRow - 1, iCell).ResultIU
  52:             segmentLength = Sqr((deltaX ^ 2) + (deltaY ^ 2))
  53:             If deltaX = 0 Then
  54:                 segmentAngle = 0
  55:             Else
  56:                 segmentAngle = Math.Atn(deltaY / deltaX)
  57:             End If
  58:             '14 decimal places is too accurate
  59:             If CDbl(Format(segmentLengthMaster, "0.000000")) <> CDbl(Format(segmentLength, "0.000000")) Then
  60:                 rowIsChanged = True
  61:             ElseIf CDbl(Format(segmentAngleMaster, "0.000000")) <> CDbl(Format(segmentAngle, "0.000000")) Then
  62:                 rowIsChanged = True
  63:             End If
  64:         Else
  65:             rowIsChanged = True
  66:         End If
  67:         If rowIsChanged = False Then
  68:             iCountUnChangedRows = iCountUnChangedRows + 1
  69:             If iCountUnChangedRows > 1 Then
  70:                 Exit For
  71:             End If
  72:         End If
  73:     Next iRow
  75:     If rowIsChanged And iCountUnChangedRows = 0 Then
  76:         'All rows have changed
  77:         shp.CellsSRC(Visio.VisSectionIndices.visSectionObject, Visio.VisRowIndices.visRowLock, Visio.VisCellIndices.visLockVtxEdit).Formula = 1
  78:         Visio.Application.DoCmd Visio.VisUICmds.visCmdDRPointerTool
  79:     End If
  80: End Sub

Hopefully, this will answer the original question!


Looking forward to the launch of Visio 2010!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.


O365 and SharePoint

Simplify Tasks

Want to learn the simple way?

Paul Turley's SQL Server BI Blog

sharing my experiences with the Microsoft data platform, SQL Server BI, Data Modeling, SSAS Design, Power Pivot, Power BI, SSRS Advanced Design, Power BI, Dashboards & Visualization since 2009

John Goldsmith's visLog

be smart, be clear, be visual ...

Mo's blog

Personal views on Dynamics 365 for Operations and Technical Architecture.

Chris Webb's BI Blog

Microsoft Power BI, Analysis Services, MDX, DAX, M, Power Pivot and Power Query

Azure Solutions for Office 365, and more...

Rob Fahrni



Life with Visio and other Microsoft Toys!

Nilsandrey's Weblog

Just another weblog

Things that Should be Easy

Every so often (too often in the IT industry) I encounter things that should have been very easy to do but turned out to be far too complicated. My favorite topics include SharePoint, .Net development, and software architecture, especially distributed systems.

Visio Guy

Smart graphics for visual people

%d bloggers like this: