Ted Selker [1] is the amazing guy who invented and refined the Trackpoint [2] or "Joy Button" as he called it (but IBM refused to call it). He put years of research and development into the product, and I'm happy he's finally written up and published the story. [3]
[1] https://en.wikipedia.org/wiki/Ted_Selker
[2] https://en.wikipedia.org/wiki/Pointing_stick
[3] http://research.microsoft.com/en-us/um/people/bibuxton/buxto...
I'm glad for the chance to share some of the stories he's told me verbally over the years, some of which I wrote up about a year ago, and I will update with links to more fascinating information about his work. [4]
[4] http://www.reddit.com/r/gifs/comments/1o26fb/old_school_thin...
Ted is one of my favorite successful mad scientist inventor heroes, who's created many amazing ideas, and followed through to make them practical products! He's in the same league as Will Wright [5], Trurl and Klapaucius [6], Dr. Emmett Lathrop "Doc" Brown, Ph.D. [7], and Rick and Morty [8].
[5] http://en.wikipedia.org/wiki/Will_Wright_%28game_designer%29
[6] http://en.wikipedia.org/wiki/The_Cyberiad
[7] https://en.wikipedia.org/wiki/Emmett_Brown
[8] http://en.wikipedia.org/wiki/Rick_and_Morty
In 1984 he observed that it took 0.75 - 1.75 seconds to reposition the hand from the keyboard to the mouse, which is a long time for something that you do quite often. `
He tried many different ideas and built several prototypes, then later when he was working at IBM Alameda Research Lab, he had a chance to refine the idea into a product.
He had his father, a material scientist, help by designing the special non-skid rubber that the clitoris was made from.
IBM wouldn't let him ship it until it was measurably as efficient as a mouse for common tasks.
The thing going for it was that it eliminated the 0.75 - 1.75 second hand repositioning penalty, but of course the fundamental problem with it that you can't get around is that it's a relative positioning device, not an absolute positioning device like a mouse. So he had to come up with ways of overcoming that problem.
The trackpoint performs very well for mixed typing and pointing tasks, since you switch between typing and pointing so often, and that adds up to a lot of time, and is a very common way of using computers. The mouse is still better for tasks that are mostly pointing and clicking, but it takes up some prime real-estate on your desk, and there are many situations where a mouse is impossible to use with a laptop.
He also made the observation that when the cursor moved above eye tracking speed, you tended to lose track of it. And also the observation that some of the time you needed to position it finely around a small area, and other times you needed to move it quickly across a large area.
So he came up with a pressure-to-speed "transfer function" that had a non-linear mapping from how hard you were pressing it to how fast the cursor moved.
The mapping had a plateau at "predictable fine positioning speed" (i.e. there was a wide range of light pressure that would map to moving the cursor at one exact slow predictable speed, so you could smoothly cruise the cursor around with a light touch at a speed that was good for exact positioning. Then after the plateau of light pressure, it sloped up smoothly until just below eye tracking speed, where there was another plateau, mapping a wide range of harder pressure to a fast-but-not-so-fast-that-you-lose-track-of-it speed, for coarse positioning without losing the cursor. And then above that there was a fast speed for quickly flicking the cursor to the other side of the screen.
They did lots of user studies and took lots of measurements and performed lots of experiments to determine the best parameters for the pressure-to-speed transfer function, and finally came up with one that was measurably good enough to make IBM happy and ship in products.
So after pooh-pooh-ing the name "Joy Button", IBM finally settled on and trademarked the name "Trackpoint." But one concession they made, was when they published a two page ad spread in Time Magazine with a close-up of the trackpoint, above the slogan "So hot, we had to make it red!"
Another crazy but brilliant innovation he developed was the Thinkpad 755CV [9] that you could remove the back of the LCD screen and lay it down on an overhead projector to project the video! Nobody probably remembers overhead projectors any more, but they were very popular at the time, and that feature could save you a lot of money, and you never had to reboot it three times just to get the video on the projector, like with modern laptops!
[9] http://www.thinkwiki.org/wiki/Category:755CV
He also made a prototype Thinkpad with TWO hot red trackpoints on the keyboard, which invitingly resembled a pair of nipples. It was very popular with everyone he tested it on, but unfortunately OS/2 had no idea how to cope with two pointing devices, so there wasn't much use for it, besides being a wonderful ice breaker at parties.
I don't know if his lab is the one that invented the butterfly keyboard, but it was another in a long line of wonderful innovative ideas that were coming out of IBM's research labs and showing up in the Thinkpad at the time.
I learned about this stuff from the talks and demos he gave at his NPUC - New Paradigms for Using Computers [10] workshop that he produced at IBM Alameda Research Labs. [11] -- it was a really great free workshop, including free lunch, with people like Marvin Minsky and John McCarthy!
[10] http://www.youtube.com/watch?v=sC3SPFfKQMM
[11] http://www.youtube.com/watch?v=10AB0yqxGEw
Ted Selker, not Edwin Selker, explains the theory and story behind the Trackpoint. [12]
[12] http://www.youtube.com/watch?v=H6guBllqPPY?t=0s
Computer Human Interface Technology IBM ARC - 8_95. [13]
[13] http://www.youtube.com/watch?v=eSJbjmZaAI4
1995 IBM NPUC Augment and Bootstrap Institute (July 27, 1995), with Ashok Chandra (IBM), Ted Selker (IBM), Marvin Minsky (MIT), Terry Winnograd (Stanford), Henry Lieberman (MIT), David Kelly (IDEO) , Douglass Engelbart (Bootstrap Institute), Nolan Bushnell (Atari), James Gosling (Sun), Marc Davis (Interval Research), Ramana Rao (Xerox PARC), Steve Mann, Thad Starner (MIT). [14]
[14] http://archive.org/details/XD1932_95IBM_NPUC_AugBootstrapIns...
NPUC '94 Tape 1A Window Dub. [15]
[15] http://www.youtube.com/watch?v=ZtUGnhVJsh8
NPUC '97 Highlights #1. [16]
[16] http://www.youtube.com/watch?v=NSOmRK_PTqM
NPUC '97 Highlights #2. [17]
[17] http://www.youtube.com/watch?v=sC3SPFfKQMM
NPUC 1997 Part of Ted's Talk From Compilation 2. [18]
[18] http://www.youtube.com/watch?v=10AB0yqxGEw
NPCU talks by Don Hopkins and Don Norman, discussing how well intentioned user interface designs can unintentionally enable users to easily destroy cities and cause nuclear meltdowns. [19]
[19] http://www.donhopkins.com/home/movies/NPUCHopkinsNorman.mov
How We Create Power Supply _ From Compilation 1 [20], in which Ted Selker explains why and how his group designed new kinds of power supplies, and focuses on their design approach of doing lots of competing designs at once and comparing them.
[20] http://www.youtube.com/watch?v=dApu7r1WxNk
Once I was sitting in a coffee shop in Mountain View hacking on my Thinkpad, and Ted and his wife Ellen rolled in, sat down, and started chatting. Ted noticed that my Thinkpad's Joy Button was all worn down, and he was mortified and quickly excused himself to go out to the car. Ellen rolled her eyes and shrugged, explaining that he was always like that. Then he came back with a big bag of red Joy Buttons, and replaced my worn-out one right there in the coffee shop, and gave me a few extras as spares!
He's a brilliant inventor, and a really nice guy, who apparently always carries around a big bag of spare Joy Buttons in case anybody needs one.
Thanks for the remarkable post - one question, "the fundamental problem with it that you can't get around is that it's a relative positioning device, not an absolute positioning device like a mouse." - it seems to me that touchpads, mice, and trackpoint nubs are all relative positioning devices. The only input devices that I can think of that are absolute would be a touch-display or digitizer.
What I meant was that some input devices are "more relative" than others, i.e. second derivative, pressure controlling velocity, instead of movement controlling position. With pressure controlling velocity, you're one more level removed from what you actually want to control, which is position. But that level of abstraction does have its benefits, like avoiding the "nulling problem" that makes context switching more difficult.
And in particular, the huge advantages the trackpad benefits from, which overcome the costs of being "more relative", is that it's in a small fixed location, and that location is right where your hands already are, minimizing the (enormous and frequently repeated) cost of switching between pointing and typing tasks (0.75 - 1.75 seconds).
Since the trackpad controls velocity, you can press it for as long as you want to move the cursor as far as you want in one direction. But with a mouse or trackpad that controls position, you eventually have to lift your mouse or finger and recenter (nulling it) when you hit the edge of the desk or trackpad.
Simply taking your finger off of the trackpoint stops moving (nulling it), while a joystick requires a spring to return it to the center (nulling it) to make it stop moving when you take your hand off of it.
http://en.wikipedia.org/wiki/Pointing_device#Joystick
>Isotonic joysticks are handle sticks where the user can freely change the position of the stick, with more or less constant force. Joystick. Analog stick.
>Isometric joysticks – where the user controls the stick by varying the amount of force they push with, and the position of the stick remains more or less constant. Isometric joysticks are often cited as more difficult to use due to the lack of tactile feedback provided by an actual moving joystick.
>Pointing stick: A pointing stick is a pressure-sensitive small nub used like a joystick. It is usually found on laptops embedded between the 'G', 'H', and 'B' keys. It operates by sensing the force applied by the user. The corresponding "mouse" buttons are commonly placed just below the spacebar. It is also found on mice and some desktop keyboards.
Bill Buxton wrote an insightful and classic paper (one of my favorite of his many great papers) called "Lexical and Pragmatic Considerations of Input Structures" that explains what I'm getting at.
http://www.billbuxton.com/lexical.html
It's well worth reading the whole paper (it's short and accessible), but here's the relevant stuff:
Check out "Figure 1: Taxonomy of Input Devices", with the vertical axis labeled "Property Sensed: Position, Motion, Pressure (Sensing Mechanism, Touch Sensitive)", and the horizontal axis labeled "Number of Dimensions (1, 2, 3)".
http://www.billbuxton.com/lexical1.gif
The column labeled "Number of Dimensions / 2 / Small Fixed Location" has the following cells:
"Isotonic Joystick" @ "Position / Sensing Mechanism"
"Spring Joystick, Trackball" @ "Motion / Sensing Mechanism"
"X/Y Pad" @ "Motion / Touch Sensitive"
"Isometric Joystick" (i.e. trackpoint) @ "Pressure / Touch Sensitive".
>Caption: Continuous manual input devices are categorized. The first order categorization is property sensed (rows) and number of dimensions (columns). Subrows distinguish between devices that have a mechanical intermediary (such as a stylus) between the hand and the sensing mechanism (indicated by "M"), and those which are touch sensitive (indicated by "T"). Subcolumns distinguish devices that use comparable motor control for their operation.
>[Discussion...] Before leaving the topic of the tableau, it is worth commenting on why a primary criterion for grouping devices was whether they were sensitive to position, motion or pressure. The reason is that what is sensed has a very strong effect on the nature of the dialogues that the system can support with any degree of fluency. As an example, let us compare how the user interface of an instrumentation console can be affected by the choice of whether motion or position sensitive transducers are used. For such consoles, one design philosophy follows the traditional model that for every function there should be a device. One of the rationales behind this approach is to avoid the use of "modes" which result when a single device must serve for more than one function. Another philosophy takes the point of view that the number of devices required in a console need only be in the order of the control bandwidth of the human operator. Here, the rationale is that careful design can minimize the "mode" problem, and that the resulting simple consoles are more cost-effective and less prone to breakdown (since they have fewer devices).
>One consequence of the second philosophy is that the same transducer must be made to control different functions, or parameters, at different times. This context switching introduces something known as the nulling problem. The point which we are going to make is that this problem can be completely avoided if the transducer in question is motion rather than position sensitive. Let us see why.
>Imagine that you have a sliding potentiometer which controls parameter A. Both the potentiometer and the parameter are at their minimum values. You then raise A to its maximum value by pushing up the position of the potentiometer's handle. You now want to change the value of parameter B. Before you can do so using the same potentiometer, the handle of the potentiometer must be repositioned to a position corresponding to the current value of parameter B. The necessity of having to perform this normalizing function is the nulling problem.
>Contrast the difficulty of performing the above interaction using a position-sensitive device with the ease of doing so using one which senses motion. If a thumb-wheel or a treadmill-like device was used, the moment that the transducer is connected to the parameter it can be used to "push" the value up or "pull" it down. Furthermore, the same transducer can be used to simultaneously change the value of a group of parameters, all of whose instantaneous values are different.
Thank you for the effort you put into this post, this is the stuff I come to HN to see and it's increasingly rare.
I took one of his classes. Didn't know he had done so much, that is awesome.