I think there is something to be said for "muscle memory". Tog was testing on relatively inexperienced users. Expert Vim and Emacs users do not take two seconds to decide which key to press, and we have real corpuses of computer interaction data which couldn't exist if the two-second figure was a fast rule -- specifically, StarCraft game logs. A lot of mousing happens in StarCraft, but a lot of keyboarding happens too, and lots of productive APM just wouldn't be possible if Tog's observations were a fast rule for all users.
> Expert Vim and Emacs users do not take two seconds to decide which key to press
Many times when editing in emacs I just "do the thing" without consciously knowing what keys I pressed.
If given a keyboard with no keys labeled, I would not easily know what each key was. Just the same, I could look away from it and type the alphabet. Magic.
> Many times when editing in emacs I just "do the thing" without consciously knowing what keys I pressed.
More than once I've tried to describe a feature to some other Emacs user, and had to physically put my hands on a keyboard and watch them in order to say which keys were used to invoke it.
I'd add that the opposite can be true, too. My 'command menu' is like VSCode's: you can narrow down the list of commands, and you can see the keyboard shortcut for running the command beside that.
I use MessageEase keyboard on my phone and you can disable the letters and just have blank buttons. I love people's reactions when they see me thumb pecking at this inscrutable black grid, it does feel like magic.
MessagEase is pretty much the only viable option for typing on my phone for me - Although I found it while I was on the search for a chording keyboard, which then didn't really meet my expectations...
I was given a mechanical keyboard with a set of blank keycaps (it was in my wish list). People thought it was black magic when I typed, but I never even noticed.
It's exactly like how you totally forget PINs, phone numbers, or passwords once you've typed them enough. You need a keypad in front of you (or at least to pretend there's one) and to just let your fingers do what they've done so many times before in order to figure it out.
I don't do well with laptops and mac keyboards due to their nonstandard key arrangements though.
> It's exactly like how you totally forget PINs, phone numbers, or passwords once you've typed them enough.
That happened to me once, the electrician had to change the electricity meter, he called me to ask for the code to enter the flat and I couldn't recall it; I had to go down and open the door.
> I would also like somebody—anybody—to show up with one properly-conducted test report that shows that command keys are equivalent or superior to pull-down menus somewhere, somehow.
It's been 30 years since he wrote that. Has anyone in that time been able to produce data that refutes Apple's findings?
> It's been 30 years since he wrote that. Has anyone in that time been able to produce data that refutes Apple's findings?
Yes, all of the specific experiments Tog describes are bogus. In versions of the experiments where the keyboard user isn't crippled by weird limitations that don't apply to actual computer users, the keyboard is often superior to the mouse (not always, just often): https://danluu.com/keyboard-v-mouse/
Everyone ever using a POC application in a store - the DOS/Windows kind, from before the current generation browser-based nonsense?
If you saw them, you probably noted they were fully keyboard-operated, had a lot of functionality accessible under function keys, and enabled the clerks to accomplish a lot of tasks without actually looking at the screen(!). You can't beat something like a muscle-remembered "F2, enter, enter, <type name>, tab, enter, enter" with a dropdown menu. I've seen cashiers do the former faster than it'd take to select a single option from a menu.
The only way I can imagine Tog's assertion to make sense is if we're comparing a mouse menu vs. looking up a shortcut on a physical, printed cheat sheet (which is essentially a menu-equivalent), for an operation that you're using for the first time in your life. None of that resembles actual use of real software.
Most likely, but I don't know how to do a search for it.
The more items in your menu the slower it is to find things in it. I can type NY faster than I can find New York in your states menu (I picked a state near the middle to ensure you have to scroll, but can't scroll to the end which might be faster if you know).
Tog was forced to back off his claim, but this wasn't publicized as much. A well trained expert with key shortcuts used all the time is faster than someone with a mouse - but if you don't use the shortcut often, or there are too many keystrokes the mouse wins.
Maybe not the same, but it is a fundamental user experience principle.
"Showing users things they can recognize improves usability over needing to recall items from scratch because the extra context helps users retrieve information from memory."
Doesn't this depend on the number of actions available at a given point in time? For example, if I'm "using my Macbook", I could be wanting to do any of no less than a million (not an exaggeration) actions at any given moment. It's infinitely easier for me to open Alfred (or the OSX spotlight for a weaker example) with one keystroke and type the thing I need to do. It simply is not possible to give me a menu (presciently) for this kind of thing.
Perhaps if you're restricting context to one app that doesn't have too much going on it's a different story, but compared to 30 years ago there is a lot more people (especially devs, or Starcraft players) need to be doing on the computer at a given time.
I suspect that graphical/menu input is preferable only for a Goldilocks range of decision tree complexity. Too many options and displaying them all to the user at once is impossible or unhelpful, forcing you to provide a search function (see the Windows Start Menu). Too few options, and the user won't need any prompting to know what choices they have and what keys they can be selected with (eg. Escape vs. Enter).
People use/used these systems faster than the computer could even run. The input would be buffered so all of it happens correctly but the screen would struggle to keep up.
Starcraft supports the theory that mouse is much better for positioning and selection than keyboard. It's so much better, it's worth it keeping one hand at the mouse.
I mean, that was what the vi fans I met in my youth tried to deny. They claimed positioning by incremental search was better.
In a late uncle's stuff, I came across a copy of "The psychology of human computer interaction" by Stuart K. Card et al. A book from the early eighties detailing Xerox PARC UI research. They did a study comparing cursor keys (or vi keys), word and line keys (home, end, next word etc.), incremental search and mousing. They found the same as Apple later would.
I think "mouse is better" was a conclusion in search of evidence. It was Apple in the 80s -- Steve Jobs was a huge believer in mouse-based UIs, but most users and developers outside Cupertino were not. Thry had to be convinced that mousing is universally better than keyboarding.
> I came across an interesting example of perception vs. reality while designing a small text editor: When scrolling the text horizontally in a window we would refresh the text by redisplaying each line starting at the top. This resulted in a wave of text rippling down the screen, and many complaints that the screen refresh was too slow. The remedy was to scroll the bits already on screen and then redisplay each line from the top. The second implementation was actually slower than the first because we incurred the overhead of scrolling the bits before we even started to display the new text on the screen. However, the perception was that there was an immense increase in speed. We stuck with the second implementation because it increased the overall satisfaction of the user even though it actually decreased the throughput of the product.
The commenter is actually subtly wrong about what's going on here though (but fortunately does the right thing by focusing on user satisfaction).
Without scrolling the bits first there's a "wave" artifact. That movement will prevent most users from starting to get their bearings on the new position of the content until after the wave is done. Scrolling the bits first hides the wave and allows the user to start getting their bearings as soon as the bits are scrolled. Newly revealed content does take longer to appear, but if scrolling the bits is significantly faster than the redraw the difference is small and the user is able to process the new content more quickly overall by already having their visual bearings.
"It takes two seconds to decide upon which special-function key to press. Deciding among abstract symbols is a high-level cognitive function. Not only is this decision not boring, the user actually experiences amnesia! Real amnesia! The time-slice spent making the decision simply ceases to exist.
While the keyboard users in this case feels as though they have gained two seconds over the mouse users, the opposite is really the case. Because while the keyboard users have been engaged in a process so fascinating that they have experienced amnesia, the mouse users have been so disengaged that they have been able to continue thinking about the task they are trying to accomplish. They have not had to set their task aside to think about or remember abstract symbols.
Hence, users achieve a significant productivity increase with the mouse in spite of their subjective experience."
this is not research, this is jingoism. i've heard this study cited so many times but never read it ... i'm astonished it is so bad.
2 seconds to decide what command key sequence to press? Sure, if you contort your test so much (only novel command sequences) that the results apply only to your contrived situation.
I'm doubtful that mouse usage is really faster than keyboard shortcuts for experienced users, but supposing it is, there are at least two reason that I prefer to have keyboard shortcuts available.
The first is that keyboard input is much easier to reliably repeat. Keyboard macros are much less simple with a mouse and even when keyboard macros aren't an option, I often find it easier when making repetitive inputs to convert the inputs to key presses and repeat them.
Furthermore, I find mouse usage interrupts my flow. Using the mouse is a very different mode of input than keyboarding. I have the disadvantage that my dexterity is worse than many people's and so when I have to switch to the mouse, I have to be very careful to click the right thing. I don't usually make the same mistakes with keyboard shortcuts that I use regularly. Even if the mouse is technically faster, that interruption is very unpleasant.
It's true that the mouse is bad at macros. That's not somehow inherent to mousing, though! Operating system designers simply don't seem to care about this. Why can't I type control-U 10 and then select a menu item with the mouse, and have it execute 10 times? That works in Emacs. Or record a macro which consists of a combination of keyboard and mouse actions. This shouldn't be hard.
(Automator has some "record" functionality but it's so naïve as to be useless. It records the mouse by where on the screen you move it. If you take several seconds to decide which menu item to select, it pauses during playback, too. If you bump the mouse during playback, it will select the wrong menu item.)
There seems to be at least one person at Apple who understands the importance, because macOS does support Emacs-style control-U repeats for text editing.
The mouse forces one to stare at the screen and precisely move the cursor from position to position. While this is useful in some scenarios it becomes taxing for a repetitive workflow. To me it is like noise added to my train of thought, I do use the mouse some, it's hard not to these days but the keyboard helps me stay more productive and more relaxed too
I could use some education on how GUIs actually rolled out to real customers. I was born in 1982, so I am familiar with computers of the era, but I was consistently about 5 years behind the times, being from a very small town and not very well off family. FYI, my first computer was a Commodore 64/128.
This article has more than a faint whiff of "new hotness" vs "old crap". They have rushed to the new thing and hate the old thing. Like rushing from Traditional Relational DBs to NoSQL. Whenever you get a new tool, that doesn't always make the old tool useless.
I think this article also came well before the proliferation of functionality. There are hundreds of functions in a simple text editor nowadays. I can't imagine accessing them all via keyboard alone, but I also can't imagine losing my keyboard shortcuts while seriously editing text.
Programs also added context menus, and more recently search capability for commands and settings.
I think we've finally learned some lessons about not throwing old tools away, but I think we need to keep learning that lesson as well.
"As a result of my observations of users, there are NO command key equivalents in my product, Smart Labels, except for Undo, Cut, Copy and Paste. The Clear and Escape keys will clear the label text edit box. Users may demand key equivalents, but I will resist them"
- While I largely agree with this, I think one thing this era of computing missed (and Windows still does) is the power of querying for items - I'd love to see a comparison between mousing over, keyboard shortcuts, and querying (e.g. macos's cmd+shift+/, plugins like https://shortcatapp.com) for items between users that are deeply familiar with an application versus occasional users of an application and new users.
This. On the condition that the search has good fuzzy matching otherwise it's tedious to remember the name of each operation. I think Sublime Text cmd+p made the mouse vs ctrl-shortcuts moot.
Yikes. I always thought that when I was forced to use the mouse to do something, it was simply laziness and incompetence on the part of the developers. It never occurred to me that it could be a deliberate design decision.
TLDR: Keyboards are best for professional developers. Mice are still best for manipulators--photo editing, etc. Trackpads are best for consuming content like browsing and reading email.
From the article:
> Keyboarders only think they are faster—I think its because of the sound of the keys ...
This might be true, as a generalization, but it should be obvious that it doesn't apply to everyone. That's a bit like me pontificating on how a basketball free-throw should executed. Listening to me or studying my free-throw will reveal nothing interesting or important about performing a free-throw. Talk to a professional player, study the players that hit over 90% of their free-throws, but don't try to draw conclusions from non-players just making thing up.
I can't play basketball worth a crap, but I have used computers and their interfaces for over 50 years. All this time I have been programming, and I've programmed on more different systems than I can remember (really).
I've used every mainstream word-processor and every mainstream text-editor, often for years at a time. I have owned at least 100 keyboards and 100 mice and trackpads over the years. These are my personal observations.
* People are surprisingly good at learning repetitive fine motor skills. The average typing speed of regular people is surprising and would be, IMHO, counter-intuitive if not for our ubiquitous exposure to good typists and piano players.
* One doesn't need to be a great typist. My big fingers seem to be made for picking up rocks, not for typing, so I "only" type around 50-60 wpm despite half a century at the keyboard. However, even at half this speed my thinking speed is what limits most of my coding speed and writing speed. I don't do data-entry or transcription where more speed would be really valuable.
* Fast typists are fast because they make less mistakes than the rest of us[1].
* Qwerty is superior to Dvorak, Workman, Coleman, and other layouts for me because I
have to use other people's keyboards often. I don't work entirely by myself. If I did I might be drawn to these improved layouts, but even then a small percentage increase in typing speed wouldn't be a big benefit for me (see above). Furthermore, my hands don't seem to be vulnerable to Emacs pinky syndrome or repetitive use issues where these layouts do seem to be beneficial.
* The strange historical accident that gave us staggered keyboard keys[2] is unfortunate. I have experimented with ortholinear keyboards, I have two TEK keyboards[3] and two Kinesis Advantage[4] keyboards. These keyboards have the keys arranged in columns. I really like the Advantage layout, but I make errors when jumping back and forth between my laptop keyboard and my more extreme ergonomic keyboards. However, the main advantage of these alternative layouts for me is not their ergonomics (see above) but the better utilization of the thumbs for accessing additional "shift" keys layers (control, alt, meta, hyper, command) and special vim style leader keys. I have tried the other ortholinear keyboards as well[5].
* Major keyboard manufacturers have ergo-style keyboards, but they suffer from poor physical keys (like my Microsoft Sculpt Ergonomic Keyboard[6]) or poor utilization of the thumbs (the Microsoft keyboard has split shift keys, which is good, but they can't be programmed to transmit different keystrokes, which is bad).
* The mouse is really good at controlling analog style controls on the screen.
* The mouse is really bad for drawing. For this a tablet is slightly better--I own six. I also have also owned a couple of expensive Wacom tablet/display combinations[6] these fail to be useful to me because they are (1) poor displays, (2) expensive, (3) awkwardly big, (4) require software drivers and hardware interfaces that don't work with my newer equipment--usb-c thunderbolt for example.
* The iPad Pro with Apple Pencil if the best really practical, useful stylus for drawing note taking that I have found. It also is quite easy to use as a tablet with a Mac.
* The one-button mouse was great for my Mom, but I am more productive with more buttons, and for gaming I regularly use a gaming mouse that has 12 thumb buttons. This works great, but it does take a few hundred hours of use to become proficient. (My daughter, an avid gamer, turned me on to the Razor mouse[7]).
* The Apple trackpads are the best trackpads. Trackpads seem particularly well suited for content consumption, like browsing the internet and reading email. They are much less suited for operating other analog style screen based interfaces such as draging icons (folders, selected files, etc.) around the screen. For these later tasks, a mouse is superior.
* The Macbook Touch Bar is pretty, but for professional users, it is not worth giving up function keys to have a touch bar. The lack of an escape key makes it difficult to use vim. I can remap the escape to caps-lock or use a control-key combination instead, but then I fumble around when I'm trying to use a different system. The butterfly keys suck--I haven't decided when/if I'm going to buy a new Apple MacBook Pro with the newer keys because they still don't have function keys. Furthermore, my desktop Apple systems don't have the same keyboards and they don't have the Touch Bar.
* In the early days of GUI interfaces, Apple was right to put the menu at the top of the screen. It served two purposes, it prevented confusion as to which app menu on a cluttered screen was the correct one to interact with and it was easy to get the mouse to the menu--just shove the mouse up and to the left. It eventually hit the upper corner of the screen where you could visually pick up the cursor and guide it to the appropriate menu item. With separate windows on MS one had to guide the cursor visually all the way to the menu. However, now with the large/wide/multiple screens on my desktop systems it means dragging the mouse a long way from where I was using it to interact with a menu.
* I'm not very happy with the wide variety of keybindings that exist for Emacs. However, a properly configured Emacs system has the absolute best system there is for managing software that has one-thousand different functions that a user may want to execute. Nothing I've seen compares to a well configured Emacs for making hundreds of commands available at one's fingertips. Using a mouse instead would be laughable. Emacs requires an investment in time that will not be worth the effort for most people. Like practicing free-throws, there is no point for a regular person to master Emacs. However, if you strive to be a professional developer (or basketball player) practice makes perfect and it is worth your effort.
* Configuring Emacs is important, however it is completely programmable so there is no limit to how much time this will take. Stand on the shoulders of the giants that have come before us and use someone else's configuration. Spacemacs and Doom are great configurations there are many others[9,10,11]
* Vim is great too and is better than Emacs at some things (but I'm currently being faithful to Emacs)
* Keypunch machines suck as IDEs. Just ahead of keypunch machines on the hierarchy are line editors on paper teletype machines. The next step up are editors like TECO. All of these are archaic ways to program that I've been saddled with in the past.
* Real developers should adopt tools as powerful as they can learn to control: Emacs, neovim, and JetBrains suite of tools, all take study, pick one because they are totally worth it. There are other editors, but don't wimp out and pick one less powerful than you can handle. Almost every developer should also learn git, another tool that has tremendous advantages and is worth the effort. Every academic that works with mathematics, data, or code should learn LaTeX--this will take time but again it's worth it for serious work.
* Xah Lee has thought about keyboard, mice, and menus a lot. I like his thinking about keyboards; you should check out his website if this subject interests you. [8]
On screen keyboards aren't the best to enter text with a pointer. I wrote this with a swiping predictive keyboard for instance, and it went pretty damn fast.
With the right interface, I wonder if we couldn't learn to enter text really fast with a mouse, even without prediction. Of course there's little point in exploring that design space since we're so fast with (and have invested so much in) keyboard typing.
Either way, I think designing a keyboard only interface to play Doom efficiently would be much harder!
I found the most insightful idea in a comment at the very bottom: "A user’s perception of a product is what causes him to purchase it and influences his satisfaction with the product..."
If users prefer to use the keyboard, they're not as likely to enjoy (and use) your "mouse first" software.
A take-home message from there: "In the experience of many Plan 9 users, using the mouse in Plan 9 for an extended period of time and then going back to using vi in Unix highlights the amount of time you spend watching the screen as you cursor around with hjkl. Having broken out of the hypnosis that Tog describes, I just get frustrated beyond belief. Yes, I am watching the cursor move so I should be occupied, but all I can think is ``damnit, if I could just click where I want to go I'd be there by now.'' "
See also high-brow three button mousing in Acme, Plan9's text editor:
I'm a long time Plan9 fanboy, but I find myself more leaning towards line editors (Pla9's sam in terminal mode), partly to avoid watching the cursor all the time. In a line editor, you have to carry the structure in your head, and it is liberating and calming in a strange way.
Ken Thompson, prominent sam user, has been attributed saying "I don't want to see the state of the file when I'm editing." I really like this; then again, I am not a real coder, just a hobbyist and writer of words. So I have no idea how realistic it would be to use console sam or ed for daily professional coding. The way line editors resemble a typewriter can be an excellent cure to the so-called writer's blocks, though.
> vi in Unix highlights the amount of time you spend watching the screen as you cursor around with hjkl
vi is four decades old. It's not the pinnacle of keyboard-driven interface design, vast improvements have been made since then.
For example, I use vim-sneak, which lets you jump to anywhere on your screen in 3 keystrokes flat 99% of the time (and the 1% of the time where you miss, well, 99% of those times, just one more keystroke is all it takes to get to the right spot): https://github.com/justinmk/vim-sneak
I also use relative line numbers, so I can select or delete most blocks of text in 3 or 4 keystrokes, for example d4j deletes the current line and the next 4 lines (5 lines total), or d23k deletes the current line and the previous 23 lines (24 lines total): https://jeffkreeftmeijer.com/vim-number/
Obviously number of keystrokes isn't the only factor that determines operation speed. This is actually exemplified by my second example, commands like "d23k" actually work out-of-the-box even in original vi, the problem is that counting 23 lines by eye is way slower than selecting 24 lines with a mouse by eye, which is solved by relative line numbers.
> The stopwatch consistently proves mousing is faster than keyboarding.
Interesting. I wonder did anyone conducted a recent research.
Keyboards stayed the same over all these years while mice progressed a lot, modern ones have many thousands of DPI and deliver up to 1KHz of samples. This makes mice work much better in comparison.
Modern displays have much higher resolutions and sizes, a 4k 27” display, even scaled to 150% DPI, contains much more stuff than 9” CRT of 40 years old Apple computers. This makes mice work much worse in comparison.
My experience is mostly with Macs, but I guess other OSes have all the similar things.
First, physical size matters. A button on a 4k screen will probably be a similar physical size as a button on a 9" CRT (or even bigger), so targeting is usually not an issue (unless there are glitches in scaling or you don't go for Retina's 2x scaling).
Second, mouse movement is calculated based on speed. If you drag the mouse quickly, the pointer will travel a greater distance for the same mouse movement than a slow drag. This helps move the mouse quickly and then adjust position.
Modern mice are much more precise than old mice. The high resoltion screen makes things more readable. All in all My guess is that the situation is actually better for mouse users than, say, 15-20 years ago.
> If you drag the mouse quickly, the pointer will travel a greater distance for the same mouse movement than a slow drag. This helps move the mouse quickly and then adjust position.
IME, mouse acceleration is one of the bigger problems for people who rarely interact with computers. Not sure why it's turned on by default so often.
Have you ever used a computer with no or bad mouse acceleration? (If you’re a longtime Mac user, you probably have not.) It’s really unpleasant. It’s one of the things the Mac got right really early on.
I have a Mac Plus at home, and the smoothness of the mouse basically feels the same as my magic mouse. I do notice a bit of difference due to the tracking ball vs the magic mouse's laser, but it doesn't seem like the mouse sample rate matters much.
The screen, on the other hand, is much smaller and therefore faster to navigate by mouse. The top menu bar, in particular, doesn't feel like a chore to reach for like it does in modern macOS.
> It takes two seconds to decide upon which special-function key to press.
I'm definitely in the keyboard camp and I know for a fact this is not the case for me.
When I am keying in code what I tend to do is just read the text as it is produced.
I'm not consciously thinking about which key needs pressing, and I know this only because on rare occasion I actually find that the text I reading is not the text I was expecting.
On those occasion I stop typing and at that point I then do need to spend a few seconds consciously trying to figure out what went wrong.
> Hence, users achieve a significant productivity increase with the mouse in spite of their subjective experience.
Sorry but between doing a high level, interesting activity that takes a few more microseconds and doing a mind numbing activity that makes me want to die inside, and also takes a few less microseconds, I choose the former.
We should not base everything in efficiency. Eating only soylent green is efficient but also awfully boring.
