Computer interfaces transform mathematical studies

Typical geometric problems of interest to mathematical visualization applications involve both static structures, such as real or complex manifolds, and changing structures requiring interactions, such as the moves that transform one knot into an equivalent knot. Our goal is to show the nature of the interrelationship between mathematics and computer science, especially computer graphics, computer haptics, and multitouch technologies. We adopt for the most part a computer scientist's perspective on the progress, techniques, and prospects of mathematical visualization, emphasizing those areas of 3D and 4D geometry where interactive paradigms are of growing importance. Here, we present a selection of three computer interfaces by which we may push our visual understanding of relatively simple geometric objects as far as our imagination can take us.

Our first example concerns multimodal exploration of the fourth dimension. People learn about the everyday world by combining different sensory modalities. In particular, knowledge of shape comes from a combination of sight, touch, and exploration. By combining computer graphics with computer haptics, which imitate the 3D sense of touch, we can provide multimodal exploration tools that can in principle improve on real life. To see how this improvement is possible, imagine that you have a real, physical rope that is tied in a knot. Your view of the rope is interrupted where one part crosses another. If you try to trace the rope with a fingertip, your finger will eventually collide with another part of the rope, interrupting its smooth passage. With a touch-based computer interface, however, there are no physical obstructions to the motion of a pictured ‘computer hand,’ and the entire rope (or any other object) can be traced without the interruptions imposed by sight or a real physical obstruction.

We can use the same methods of this touch-based multimodal paradigm to help us understand and manipulate the much more complicated case of shapes with self-intersecting surfaces,^1–3 which arise naturally when projecting objects from 4D down to 3D. If sound cues are added to describe the passage of the computer hand across a visual obstruction, one can explore a knot or a surface without necessarily having to use vision at all. When used in combination with a visual representation, the auditory cues provide additional redundant feedback, which can improve the intuitive perception of spatial structure.

One basic achievement of this research is to exploit the free motion of a computer-based haptic probe to support a continuous motion that follows the local continuity of a 4D surface, allowing collision-free exploration in the 3D projection of the surface. In 3D, this interactive probe follows the full local continuity of the surface as though we were physically touching the actual static 4D object (see Figure 1).

Figure 1. Multimodal perceptual exploration of a shape known as the spun trefoil (a knotted sphere embedded in 4D) projected down to 3D. See video⁴ for the exploration of a 4D torus.

Our second example of visualization applications concerns knot theory, which lies at the crossroads of mathematics and physics.⁵In knot theory, two knots are considered to be the same if one knot can be distorted into the other without breaking it, much as we might physically play with a real knotted rope where one part cannot physically intersect or pass through another. (In general, knot theory considers the ‘ropes’ to be closed loops, unlike a real piece of rope with free ends.) Mathematicians represent knots with 2D diagrams (a projection down from three-space, where the knots exist) in which the lines cross over one another. The equivalence relation among knots is mathematically generated on knot diagrams by a sequence of moves of the lines and the crossing points called Reidemeister moves.⁶A Reidemeister move (see Figure 2) is one of three kinds of moves that is performed on a local part of a knot diagram without changing the rest of the diagram. These three Reidemeister moves are representative (they generate all possible ways to change a knot diagram while still representing the same topological knot) and are useful for doing combinatorial topology of both knots and related mathematical objects called links. By applying the Reidemeister moves, we can gain insights into the underlying topological structure of a knot by obtaining a continuous family of embeddings of circles in three-space from one knot to its equivalent.

Figure 2. The three Reidemeister moves, which relate equivalent knot diagrams. Each move operates on a small region of a knot diagram. (I) Twist or untwist in either direction. (II) Move one loop completely over another. (III) Move a string completely over or under a crossing.

We developed an application called KnotPad,⁷ an interactive paper-like system for visualizing and exploring mathematical knots (see Figure 3). We exploited topological drawing and math-aware deformation methods in particular to enable and enrich our interactions with knot diagrams. Whereas most previous efforts typically employ physical-based modeling to simulate the 3D dynamics of knots and ropes, our tool offers a Reidemeister-move-based interactive environment that is much closer to the topological problems being solved in knot theory, yet without interfering with the traditional advantages of paper-based analysis and manipulation of knot diagrams. Drawing knot diagrams with many crossings and producing their equivalents is quite challenging and error-prone. KnotPad can restrict the user's manipulations to the three types of Reidemeister moves, resulting in a more fluid and mathematically correct experience with knots.

Figure 3. The supported manipulations in KnotPad are based on the three types of Reidemeister moves. This tool allows users to produce and trace equivalent knot diagrams by verifying the mathematically valid moves and recording their sequence.

KnotPad permits us to draw mathematical knots and edit their diagrams. Furthermore, we exploit supplementary interface elements to enrich the user experience. For example, this application allows one to pull and drag on knot diagrams to produce mathematically valid moves. Navigation enhancements provide further improvement: by remembering and displaying the sequence of valid moves applied during the entire interaction, KnotPad allows a much cleaner exploratory interface for the user to analyze and study knot equivalence.

The final example concerns our work to redesign mathematical visualization interfaces to exploit multitouch input. For instance, proposing a Reidemeister move requires operating on a small region of the diagram while keeping the rest of the knot representation unchanged. Such manipulations can often be more facile and satisfying by using multifinger manipulations (see Figure 4). Figure 5 shows screen images of a trefoil-type knot diagram being manipulated with multitouch gestures. This interface leverages the traditional advantages of paper-based analysis and multifinger manipulation. Interactive visualization, a 2D sketch-like interface, and multitouch technologies combine to reveal the complex spatial relationships of mathematical knots with a mathematically true user experience.

Figure 4. Mastering mathematical knots with multitouch on a multitouch tabletop.

Figure 5. Using multitouch gestures to pull a local part of a knot diagram to create Reidemeister moves. Applying a type-II Reidemeister move with (a) two-finger hold plus (b) one-finger drag. Finalizing crossing signs with two-finger hold plus one-finger double-tap (c, d). Applying another type-II Reidemeister move with (e) two-finger hold plus (f) one-finger drag.

Starting from this basic framework, we also plan to extend the range of objects that we can support to include more complex knots, links, and Riemann surfaces. The interaction paradigm presented here for knots has a 4D analogy: effective manipulation of 4D surfaces (e.g., 4D knotted sphere) might be greatly assisted by a friendly interface supporting 4D Reidemeister moves. We anticipate that a graphical user interface that supports step-by-step 4D Reidemeister moves could be used to simulate more complex and abstract mathematical phenomena, such as untying a 4D twist-spun trefoil knot.

Mathematical visualization is the art of creating a tangible experience with abstract mathematical objects and concepts. While this process has been a cornerstone of the mathematical reasoning process since the times of the ancient geometers, the advent of high-performance interactive computer graphics systems has opened a new era whose ultimate significance can only be imagined.