1) Extremely fast GPU-accelerated terminals have a latency of slightly more than one frame due to only rendering once per frame (for performance and latency reasons).
2) Gnome-terminal sucks.
3) Old school terminals, like xterm, are extremely slow in practice, but look very fast on benchmarks like this, as they render on TTY input and do not wait (thus wasting time and latency rendering unseen changes).
4) This article is old, and misses out a lot of optimization work that has occurred on Alacritty.
5) The author of this article has a 60hz monitor. These numbers would be unreproducable on higher refresh monitors.
Yeah, unfortunately he tested on OSX; notice all his numbers are bad, an unfortunate side effect of Apple unwilling (or unable) to focus on performance as a way to improve user experience and finally fix the compositor lag that's been there since the dawn of OSX.
Given the vast majority of the market has been and continues to be 60hz, their testing on a 60hz monitor isn't just understandable, it's highly relevant.
If a terminal program running on current low-end PC hardware can't handle updating/drawing within 1/60th of a second, something is very wrong.
Can you expand on (3)? It seems at first glance that the end-to-end testing (USB input through to monitor photon output) performed in this test captures the whole of the user's perception and experience.
Latency and throughput are not inverses, although they may exhibit anti-correlation over small ranges. A low latency does not imply high throughput. In particular xterm does not buffer input / rendering, so while individual keystrokes produce a response quickly, when there is a high volume of text the throughput will be lower than other approaches.
This will still feel laggy, just a different kind of lag to the one that you perceive using a high latency terminal.
For 1), aren’t there two gpu-based terminal emulators on that list with under 16ms latency on a 60Hz display. I think that should mean less than one frame but I don’t really understand the lack of variance in the measurements. I think if reported latencies are very low, the software methodology doesn’t work so well.
The only other GPU accelerated one I see is Kitty. Current Alacritty numbers are closer to Kitty than rxvt.
Kitty statically renders at 100hz, thus skews numbers a bit; however, this also proves all this guy measured was "time to next frame", not actual performance numbers.
in the ideal case where every keystroke shows up in the next frame you should get an average of 8.3ms on a 60hz display, with a range from 0 (i did say this was ideal) to 16.7ms
Alacritty is adding scheduled frames, a technique used by modern game engines to reduce frame variance and latency, hopefully in time for 0.13.
To get any better, you'd really want something like windowed VRR to consistently work on your target OS, and then any issues in Alacritty solved (if any) to take advantage of that.
For me, the amenities of a Vte based terminal outweigh the slight negative of it being somewhat slow.
That said, not all Vte based terminals are equal, IIRC last time I had to hunt quite far to find one that handled some of the more esoteric key modifier combinations properly, and since then I've been settled on it (Sakura)
I use st daily. Having it open side by side with xterm (and also, by the way, typing out this comment in Firefox), I don't see how the differences mentioned in the article are relevant to the user experience. The character appears before my finger lifts from the key. Perhaps improvement /could/ be made, but I don't think it'd be worth it. There are other things to prioritize.
It is interesting to see the numbers laid out like this, though.
Latency becomes more relevant the less you are able to muscle-memory your way through an input. Audio apps often need relatively short latencies(<5ms is considered professionally useful) because musicians are hearing, responding and adjusting to match time, so when they do something and the delay on output is perceptible, their timing starts to oscillate.
On the other hand, if I'm drawing on a graphics tablet, and I've trained up some muscle memory skills with blind contour exercises, I don't care that much about application latency because I'm using the tactile feedback to know what the pen is doing. I only have to look to see where the cursor starts, everything else is minor adjustment.
Keyboarding is also definitely more on the muscle-memory end of the spectrum: you look to see errors, but you can generally type a whole password without seeing it, once you've practiced.
Sure, but the article isn't about audio apps or drawing with a graphics tablet; it's about typing delay in terminal emulators.
I have to agree with the OP here. I've used urxvt almost every day for years (I remember using it on a laptop with 32 MB of memory), and have never once had a problem with typing latency. I didn't even realize it was a thing people bothered to measure until I saw this article.
there's always someone in these threads who says something like this, and to be honest https://danluu.com/keyboard-latency/#does-keyboard-latency-m... could benefit from a stronger argument that latency impedes performance or pleasure specifically for keyboard tasks rather than touchscreen or stylus tasks
I can only offer anecdote and no concrete measurements. But I've used bad terminal emulators and good ones, and I can really notice the difference. When the letters appear even a few frames after they should, there's a definite (if mild) feeling of discomfort. Like some kind of sensory-motor "friction". I can still get work done, but I'm just a little less happy.
There was another HN thread recently where this topic came up, I posted about the human nervous system as a prediction-feedback system where we should expect even small latencies to have a bad effect. And someone replied with the interesting insight that 10-100ms latency feels like "you are using the tool", while <10ms latency feels like "the tool is an extension of your body". (I'd quote the whole thing but I think there's an HN rule against that.)
I'd certainly like to see an experiment where typing latency could be controlled artificially, on a high refresh rate monitor. It would test what level of latency/jitter has a perceptible effect on objective typing error rates, and subjective ratings of "satisfaction". Ask them to compare two text input fields where one has more latency than the other, see what degrees of difference are noticeable, to determine the "resolution" of time perception for fine motor tasks. I would wager that for good typists the "threshold of perceptibility" is as low as 10ms, and perhaps even lower. But that's based on my own intuition and experience; I'd love to know for sure.
> 10-100ms latency feels like "you are using the tool", while <10ms latency feels like "the tool is an extension of your body".
Seems like that's similar to how Vim can feel so smooth to use, with experienced users not even thinking about what keys to press but just what operation to do, as the inputs can be combined in intuitive ways and the entire program has very low latency to do anything. Very different from "ugh, now I again need to move my hand to the mouse to select that bit of text".
my intuition matches yours (and luu's) but that isn't the same kind of conclusive refutation as luu's generic input latency arguments https://danluu.com/input-lag/
i mean it isn't going to convince anyone who says they 'have never once had a problem with typing latency'
In older games like CS 1.6 I believe I could feel the difference between say 10 and 40 ms ping. Maybe PS/2 keyboards were faster. I got the feeling older computers were way faster to respond ... but I might remember wrong.
> I got the feeling older computers were way faster to respond ... but I might remember wrong
I don't know the specific devices you've been using over time, but in general the measurements in the following article back up your memory: https://danluu.com/input-lag/
PS/2 Keyboards are faster than a USB keyboard with slow-speed negotiation with the host computer but for USB keyboards with high speed negotiation, they are at par[1]
>if you actually measure time from keypress to event
Nobody measures keyboard latency this way because when you do the keyboards with less key-travel (Apple Magic) end up with lower "latency"; clearly an absurd result and not at all what you are actually trying to measure. It does not line up with anyone's definition of keyboard latency.
>Nobody measures keyboard latency this way (keypress travel)
You'll find most keyboards in your link, despite these fullrate-enabled high poll rates, actually have higher latency from keypress to event than what PS2 trivially achieved.
Which, by the way, is consistent with the findings in Danluu's article ("a joke"), showing that pressing 2 keys at once might not be as accurate, but does not completely compromise the results, relative to using a mechanical arm.
Note that Danluu documents his methodology, and even expresses that he would prefer a mechanical arm setup.
I have Freesync, it doesn't it eliminates tearing without vsync, but it does nothing about consistency of framerate. What I'm saying is that I'd rather have a narrow P99, even if it's just above 60, rather than experience 50-144fps variations.
>but it does nothing about consistency of framerate.
It does quite a bit. e.g.:
If your refresh rate is 60Hz, that's some 16.6ms per frame.
If the frame takes 16.7ms and thus isn't ready by the deadline, you get either tearing (very visible, annoying, experience breaking) or your frame takes 33.3ms, which is a lot of Jitter.
Jitter between 16.6ms and 33.3ms or tearing, are both much worse than 16.7ms you could get with freesync, which is just some 100µs of jitter.
Even at 120Hz, the barely miss deadline case would be half the jitter, but still 8ms vs 50µs. And the tearing alternative is still every bit as horrible.
Yes, it helps there, but I'm talking about frame rate constantly jumping around. I'm not talking about 16.6 vs 16.7 or even 16.9, I'm talking about 16.6 vs 40/60/90 and so on. VRR isn't going to fix it at all:
I want the same consistency for terminal input delay. Suckless terminal, for example, has higher latency, but P99 is much closer to avg than Alacritty and urxvt. While on the subject of terminals: st will stay mostly the same when there is additional load present, while all GPU-based terminals will start slipping if I'm doing something that involves GPU (discord voice call, screen sharing/recording, spotify).
Pointless to compare those toy computers against cherry-picked modern desktop which is the equivalent of an old supercomputer. Indeed if you look at the numbers for the older workstations: SGI Indy, NeXTcube, Symbolics 3620 they are not that different and even worse than the newer machines. 30ms is not remarkable these days, with proper gaming keyboard, gaming monitor, config etc. you can get 5ms or less for the whole chain.
old supercomputers also sucked at latency too, and the modern desktop isn't cherry-picked
it's common for computers with less latency to also have less throughput; an arduino, the avr kind, can reliably respond to inputs in less than 200 nanoseconds
but these 'toy' computers are capable of running a word processor or spreadsheet, writing email, browsing the web (without js), compiling c, running an ide, etc. the zx spectrum even had a first-person 3-d shoot-em-up called elite. so i don't think it's pointless
So why would you expect it to be better if you shrunk them down to micro size? That is the whole point of my comment, that the observed latency is better explained as a function of hardware/software complexity and not "year of production". If you understand this, it should not surprise you at all that most contemporary desktops have the latency characteristics they do since we have seen it before in the old supers/workstations, their closest equivalents in terms of hw/sw complexity and capability. Of course it could be better and some are but you won't find them in that article.
>but these 'toy' computers are capable of running a word processor or spreadsheet, writing email, browsing the web (without js), compiling c, running an ide, etc. the zx spectrum even had a first-person 3-d shoot-em-up called elite. so i don't think it's pointless
Can it perform those tasks all at the same time like we expect now or like a mainframe/workstation user would have expected then? When you look at the entire landscape of computing hardware and software it is difficult not to see the early micros in that light. An extreme example would be the Altair 8800. I don't see how you could describe that computer as anything other than a plaything for enthusiasts. I made the analogy between modern desktops and old supers which you seem to agree with; could you even say that an early micro is the rough equivalent of a 50's mainframe with a straight face? It should be but the analogy is more difficult to justify: 36-bit vs 8-bit words, FORTRAN vs BASIC, FPU (some) vs no FPU, at every level of the analysis there are compromises. It should be clear that the micro represents an extremely compromised version of general computing that would be alien not only to us but to the serious computer user in the 70s/80s and yet their characteristics should be held as a benchmark for computing devices in general? Outside the context of "the home computing experience" they make for a poor comparison. Unfortunately there is great ignorance in this area due to the disproportionate focus on micros when discussing historical computing. Videos and articles on the Apple II series, awash with insufferable amounts of sentimentality, are a dime a dozen but you will be lucky to find anything on the IBM TYPE 704, a much more interesting and equally significant machine.
Unless you can produce actual measurements following the same methodology, for these mythical "non-cherrypicked" modern machines, I am going to call bullshit here.
> This program measures end-to-end latency: it emits keypress events and captures the screen to note when it gets updated (as actually visible to the user)
> [...]
> The screen resolution was 1920x1080 and the LCD refresh rate 60 Hz
How exactly can xterm have an average end-to-end latency of 2ms, if on average you need to wait 8ms for the next frame?
But the next frame takes a while to actually show up on the screen, so the "as actually visible to the user" part simply can't be right. This may be isolated latency of just the program, but definitely not something reflecting real-world user experience, it's 1-2 orders of magnitude off in that regard.
In case of st, it actually delays the rendering. Its default config says min-latency is 8ms, and max-latency is 33ms. This probably is why OP got 20ms as an average value.
The thing is that suckless people simply want to render less, and they have always kept a way to delay rendering in st. Also, the current latency is good enough that no one really complains. TBH, it's about 1~2 frame latency on 60 fps displays, and you hardly notice the difference.
> xterm does direct rendering via Xlib, which is the primary reason it can be so fast in responding to user input. This is also the reason why its throughput is, on the other hand, rather poor.
My guess is the rate at which stdout from a process shows up in the terminal?
I think Terminal.app on macOS is likely the inverse of this… it likely has a fair amount of input lag, but holy moly can it handle a lot of output being dumped at once without breaking a sweat.
I thought terminal.app has very short latency, looking back at the benchmarks written about here https://danluu.com/term-latency/. Terminal.app’s latency measures around 5ms beating most other terminals. (It’s a shame it still doesn’t support true color.)
Software latency measurement consistently understates latency which magnifies 1-2 frame differences. Unfortunately, camera-based measurement is a lot more work.
There’s an iPhone app called Is It Snappy that records in slow motion and makes it pretty easy. I think the camera maxes out at 240 fps so it’s not perfect but it lets you measure true end-to-end latency.
Yeah this is the easiest way to get started. You probably want a low travel keyboard and ideally a low-latency-variance one to make it easier to find the start. It can be tricky to find where the update ends too because you usually see several frames of lcd switching.
The more advanced version is putting together a microcontroller which acts as a usb keyboard pressing a key, then backspace, then the key again, and uses a light sensor aimed at the right part of the screen to detect rendering in a repeatable way (slightly different fonts/ positions might change when it triggers but that’s likely on the order of a few ms tops, much less than the variance from frame boundaries).
Yeah, I don’t really understand what the difference in latencies reported here would mean for the observed latency distribution of an actual end-to-end test. The latencies reported here are less than the difference in latency one should expect between the top and bottom of a screen (depending on vsync…). The repeatable way to measure these things is with a little embedded device which pretends to be a keyboard and uses a light sensor, but that’s a lot of work to set up even if you don’t need to take and label a bunch of videos yourself.
Zutty is pretty cool, although lack of bidi text rendering is very unfortunate. I also wonder how Konsole would fare in these tests, it seems fairly snappy.
Alacritty being so slow is surprising to me here. I only use it on macOS, but it feels faster than kitty when I'm looking at application logs scrolling quickly across the screen. Perhaps responding to typing events has different latency than tailing a log file or listening to stdout?
The latencies are different, and kitty is slower at this because the author doesn't care about huge output[0]:
> Some people have asked why kitty does not perform better than terminal XXX in the test of sinking large amounts of data, such as catting a large text file. The answer is because this is not a goal for kitty. kitty deliberately throttles input parsing and output rendering to minimize resource usage while still being able to sink output faster than any real world program can produce it. Reducing CPU usage, and hence battery drain while achieving instant response times and smooth scrolling to a human eye is a far more important goal.
I use 'yes | head -n 1000000' for a throughput test, and kitty is about 5x faster than xterm and 1/3 the speed of alacritty (zutty does not appear to be packaged in my OS). It's rather unusual to see something that is held up by terminal rendering (sbcl's build process used to be, and they recommended using xterm for faster builds, so 5x faster than xterm is probably "good enough" there).
This sounds backwards from both an energy perspective and human factors perspective. I don't understand the throughput vs. smooth scrolling tradeoff though.
Xlib is actually somewhat easier to use than xcb. Xcb however has better support for asynchronous operation. In case of xterm this wouldn't help latency much but would greatly improve throughput.
For terminal aficionados here, which terminal works well with chatGPT ? I want chatGPT to predict my next keystroke, based on my previous commands I typed.
GPT doesn't work that way. Any such shell would have to send a GPT request for every single keystroke, and you'd hit the API ratelimit very quickly. It would also be very expensive.
Maybe a solid use case for local models are they're optimized for consumer hardware-- someone clever could preserve as much context as possible between requests
Couldn't you just use Fish or zsh-autosuggestions? I don't see the benefit of using a web api or AI for every keystroke when the current solutions work well enough.
1) Extremely fast GPU-accelerated terminals have a latency of slightly more than one frame due to only rendering once per frame (for performance and latency reasons).
2) Gnome-terminal sucks.
3) Old school terminals, like xterm, are extremely slow in practice, but look very fast on benchmarks like this, as they render on TTY input and do not wait (thus wasting time and latency rendering unseen changes).
4) This article is old, and misses out a lot of optimization work that has occurred on Alacritty.
5) The author of this article has a 60hz monitor. These numbers would be unreproducable on higher refresh monitors.