This is straight up awesome and must watch video just for the lolz.
But there is a thread to pull on here in terms of tangible user interfaces and the replacement of keyboards as a traditional input system. Meta's CTRL-Labs (ex Myo Armband) are working on EMG where just signaling to your muscles can trigger keyboard strokes and the bandwidth is faster than fingers or so they say. There are several other R&D efforts in that direction from BCIs to free space gesture detection through different flavors of computer vision and sensor fusion. Mr Everything Is Hacked is doing it here just for the sheer joy of the journey but there are serious reasons to play around with qwerty alternatives.
Or augmentation of keyboards: wand/stylus; 3D finger tracking; multitouch tablet. Extending rather than dumpstering existing refined comfortable power. My laptop used to have three extra fold-out cameras for TFA-like mediapipe games. Though tech resets are an opportunity to escape trap deadends, to "do better this time" - that thinkpad keyboard was a crippled and crippling 2-key rollover. And sensor fusion with diverse latencies (keypress vs pose-tracked video) is nontrivial - complex event processing with backtrackable app state. LLMs with UI/HID "languages" will be such fun!
But there is a thread to pull on here in terms of tangible user interfaces and the replacement of keyboards as a traditional input system. Meta's CTRL-Labs (ex Myo Armband) are working on EMG where just signaling to your muscles can trigger keyboard strokes and the bandwidth is faster than fingers or so they say. There are several other R&D efforts in that direction from BCIs to free space gesture detection through different flavors of computer vision and sensor fusion. Mr Everything Is Hacked is doing it here just for the sheer joy of the journey but there are serious reasons to play around with qwerty alternatives.