Brandon Paulson
Long, Summary, Visual Similarity of Pen Gestures

The main purpose of this paper was to perform some experiments in order to determine how users determine 
gesture similarity, and to derive a computational model based on their findings.  This computation will 
later be used in the gesture design tool from “Those Look Similar!”  The paper has a very good prior 
work section which discusses some common pen-based devices like the Apple Newton MessagePad and the 
PalmPilot.  This section also discusses other applications like spreadsheets, word processors, music 
editors, equation editors, and other drawing programs.  The authors also discuss prior work in the 
areas of gesture similarity and multi-dimensional scaling.  They note a study by Attneave which 
determined that the logarithm of certain metrics could be used to correlate with similarity.  Like the 
Rubine recognizer, the recognizer developed by the authors is only a single-stroke recognizer.  In their 
first experiment, the authors developed a gesture set based on their own intuition.  Their goal was to 
have a set that spanned a wide range of gesture types and also varied in orientation.  They decided not 
to have participants draw the gestures, but instead used a program that animated the gestures.  They 
then randomly displayed a triad of gestures to the user and asked them to pick the gesture that was 
most different from the others.  The program then calculated a dissimilarity matrix based on the 
user’s selection.  This data was then analyzed in order to determine relevant geometric properties and 
to develop a model of gesture similarity.  MDS plots were used to determine relevant geometric properties.  
In addition, they used regression analysis in order to determine weights for a feature set they developed. 
The feature set they used consisted of most of the Rubine features (except bounding box length and stroke 
length) and their own features – aspect, curviness, 2 density metrics, “openness” and the log of length 
and aspect.  The model produced correlated with their findings in the experiment with 74% accuracy.  They 
believe this is a good value given the differences in opinion by the participants.  The authors then 
conducted a second test in order to investigate the relationships between 1) absolute angle and aspect, 2) 
length and area, and 3) different rotation-related features like the sine and cosine of the initial angle. 
The second experiment was run very similarly to the first.  This time their model had an accuracy of 71%.  
Some important findings were that bounding box angle was an important feature and alignment with the 
normal coordinate axes was also important.  It was also shown that length and area were not significant 
contributors to similarity judgment. Additionally, gestures composed of horizontal and vertical lines 
were perceived to be more similar than gestures with diagonal lines.  The total correlation factor of 
both experiments combined was 56%.