I think the truth is that we don't feel like we need a release. We've never had a security hole, our track record speaks for itself, and I wouldn't really want us to rush anything and blow that. There are some features that it would be really nice to have in a release (like 24-bit color support, and working around a recent MacOS clock bug), but I handed over the maintainership to somebody else about six years ago and that person is currently, I think, dealing with a bunch of stuff (look around the world the last year...). I'm not eager to step back in and bigfoot them -- especially when I'm no longer particularly familiar with the codebase (and recent commits) as I used to be.
Mosh works as advertised and has never had a security hole -- we're pretty proud of that! We'll probably cut a release at some point to add those features (24-bit colors, the MacOS clock workaround) but I'm not feeling like it's urgent enough to upset what I had hoped was a transition plan.
It would feel arrogant to compare Mosh to TeX, but it doesn't seem that crazy to imagine that some software might reach a point where it has accomplished 95% of its goals, and the benefit from adding further features has to be weighed against the risk of introducing a security hole or other regression through further churn. If the TCP specification, or OpenSSH, or TeX, or GNU bash had canonical GitHub repositories, they would probably be full of a bunch of user support issues and inactive PRs too. :-)
Thanks much and keep the same heading please! I really enjoy using tools that have matured and become good enough that there's no longer any major need for new features — and where nothing is added just for the sake of "improvement". I personally haven't had a single thing in Mosh I would've wanted to add to it, and I'm glad nothing has been added either.
A lot of Unix tools are like that. They do what they were written to do and that's the scope they're sticking with. It's fine if, maybe once in five or ten years, some support is added for a thing that grew outside the tool itself, such as interfacing a new system component. This conservative development might be a trait of the less-flashy command-line world.
In contrast, the desktop is full of programs that were in that good phase once but then development continued further, adding satellite features that just make the program worse until it's unusable even for the original purpose. Why not write another program to do the new things then, instead of packing everything into a single package? I don't know.
I concur, but at the same time have a counterpoint. It always trips me how I can user Perl regex for powerful searching in grep, but then these are somehow not available in sed. The Extended syntax it has is not the same, still too verbose in comparison
Except that it's a lie. Not accepting new feature requests is one thing, but what actually seems to be happening is that new code has been accepted into the master which then never sees the light of day.
Hiding behind "well we don't want to compromise the core software" after stringing other contributors along for years doesn't pass the sniff test.
Is there anything preventing you from running the latest version from git? Is there something special about a "release" that you're looking for? (I can think of several reasons why someone might want a release and not the latest master branch, but I'm curious about your specific reasons)
I am not OP, but think of it from a contributor's point of view: you work X hours on a new feature that even gets accepted into the master, but then you know that 99.9% of mosh users will not see it because it's not part of a release. I don't know how I would feel about that.
> I think the truth is that we don't feel like we need a release.
Maybe issue a point release where the only change is updating the documentation (man page, output of --version, ...) to state that it is 2021 and you are still here, stable, free of security issues, but not adding/updating features ATM. Then the project doesn't look dead (which can be a security concern) when it is in fact just quietly carrying on with achieving its goals without the need for changes.
> I think the truth is that we don't feel like we need a release. We've never had a security hole, our track record speaks for itself, and I wouldn't really want us to rush anything and blow that. There are some features that it would be really nice to have in a release (like 24-bit color support, and working around a recent MacOS clock bug), but I handed over the maintainership to somebody else about six years ago and that person is currently, I think, dealing with a bunch of stuff (look around the world the last year...). I'm not eager to step back in and bigfoot them -- especially when I'm no longer particularly familiar with the codebase (and recent commits) as I used to be.
I can appreciate that, but what do you say to e.g. the contributor that added true color support nearly 4 years ago?
cgull also hasn't authored or committed anything in mosh in more than two years, so his inactivity predates the pandemic by quite a bit.
Everyone loves your software, and that's the reason they want to see another release with the many improvements already in git, most for multiple years. I really don't think you're going to step on any toes by making a new release.
I don’t use mosh myself as I thought the project had been abandoned, but I know several folks that love it. Just want to thank you on their behalf.
Also, I appreciate knowing you all are watching PRs and bug reports even if you don’t see the need to take action. Makes me feel the project is just dormant and not abandoned. If you care about continued use and adoption, you might consider posting an update to the website similar to this post you made here. If you aren’t worried about adoption, then no problem and thanks again for your effort.
Midnight Commander is also being drawn in a jumpy way.
Note that I'm not speaking about new emojis or some novel Unicode stuff, it's the same basic multilingual plane and line drawing characters we've had for decades now.
Thanks for the tool! I use it all the time, even just now I’m on a crappy connection via cellular modem that oscillates from few bars LTE, to 3G at <10KB/s, to occasionally fully dropping out for a few seconds, and mosh keeps the session alive.
I like the philosophy here, if the software is “done” and there is no immediate need for security fixes, don’t touch it.
I think a lot of users look at the release history and cadence as a sort of heartbeat; in order to tell if a project is still being maintained. That can be a problem when a program reaches a mature state.
> It would feel arrogant to compare Mosh to TeX, but it doesn't seem that crazy to imagine that some software might reach a point where it has accomplished 95% of its goals
The difference that I think makes this comparison invalid is that Mosh is a communication tool, whereas Tex is running locally on files. We (as a community) have learned that any software with a networked attack surface slowly gets less secure over time - you (almost always) need to provide ongoing security fixes to maintain the desired level of security.
To be clear - I'm not saying you're wrong. I'm an intermittent user of Mosh and I can believe that no holes have been found that need patching (and that the team would in short order if necessary). It is, however, a signal that users of software look for. I like the idea in a sibling comment of the "documentation" commit just saying "we're still here" to reassure people.
If a security issue was discovered in mosh, I feel pretty confident that we would find a way to make a release with that fix (even if it's just a point release that applies the fix on top of the current `1.3.2` tag)
mosh has been a pleasure to use. thanks for all the hard work :). i think it’s worth experimenting with blink, too, if someone (like me) has the odd desire to try and be productive from an iphone or ipad
As someone over a year into a burnout... good for them for guarding themselves from overextending. Maintaining a project is hard even if you have the whole of your energy and attention to devote. It's a lot harder when you're trying to power through.
I've held off finishing and releasing half a dozen projects over the last year because while I think they'd be useful I know I don't have bandwidth to support them.
I assume this means you’re working remote and it benefits you? If so, that’s awesome and it should be accommodated!
Edit: I take that (assume) back and wonder if there’s other social benefits you’ve gotten from isolating regardless of work conditions. I also support that if so!
Personally, I believe a good software should end up not in "we stopped releasing as we feature complete" but "from now we do only maintenance releases as we are feature complete" because the rest of the world is still evolving: apps, environments, OSes, some bugs could be sleeping for decades waiting to be discovered, etc. It's unfortunately an endless process to adapt even a finished project to a rapidly evolving world.
Unicode (really, knowing the width and printability of any given character) is mostly up to the C library on the client and server -- Mosh has historically stayed out of that. Unfortunately Apple is not always super-great about keeping their C library up to date with new Unicode releases, which leads to some frustration, and even on Linux it takes too long for a new Unicode release to percolate down to the libc on the client AND server. (I think the model of "libc implements wcwidth" probably made more sense in the era before Unicode decided to start pumping out new releases every year with lots of new characters everybody wants to use. But we try to keep Mosh's own attack/implementation surface small.)
With fish we've made the experience that relying on libc isn't good enough.
Specifically in the case of connecting to a server that typically has an old libc with old unicode information, from a desktop that has a much newer system, or in case of ambiguous characters, where libc will just give you one width that might not match what the terminal actually renders (and they frequently have configuration options to change it!).
So we've made something we call widecharwidth (https://github.com/ridiculousfish/widecharwidth), which is a python script that parses the unicode datafiles (UnicodeData.txt, emoji-data.txt and friends) and generates a header you can #include.
It's tempting, and thank you for the PR, but, do we then have to push a new Mosh release every time there's a Unicode update? And the users have to get this release installed on both the server and their client? It seems... like it fixes part of the problem, sure, by us taking on more of the load and locking us into an annual release cadence.
I think if we give up on libc, my "perfect" solution would probably be something like:
(a) user runs a script that prints every single Unicode character in their local terminal and learns its width (probably we can do this in some smart way)
(b) client somehow communicates this info to the server at runtime, over the protocol.
But that's a lot of protocol work and kind of annoying. The good-enough solution might be:
(a) user runs a script that prints every single Unicode character in their local terminal and learns its width, or perhaps user just runs your script on the Unicode tables (+ user-supplied info about how their terminal handles ambiguous-width East Asian characters)
(b) user is responsible for distributing this data file to every server they feel like connecting to and putting it in some well-known location in their homedir.
I think the current maintainership has their own idea of what they want to do that's not quite this either.
Just to be clear: It's not my PR. The person who made that must've just seen widecharwidth and thought it was a potential solution.
>do we then have to push a new Mosh release every time there's a Unicode update?
In theory, yes. In practice the new codepoints take long enough to be available anywhere and there are few enough of them that being a bit out of date isn't a problem.
(case in point widecharwidth is still on Unicode 12 apparently - I should update that)
>user runs a script that prints every single Unicode character in their local terminal and learns its width
And they would have to re-run that regularly, whenever the terminal updates or they switch.
>user is responsible for distributing this data file to every server they feel like connecting to and putting it in some well-known location in their homedir.
And they would have to do all that setup.
That's a lot of annoyance to put on your users when you can solve 99% of the problem by just incorporating a semi-up-to-date width table yourself.
The perfect is very much the enemy of the good here.
There needs to be a trust with a deed indicating the beneficial purpose of Mosh. Then the devs would be in a trust management corporation and could get paid for managing the project in the interests of the computing public.
Because 99% of open source software has about 1-3 real devs and after they give up there is no one who can be bothered to take over. Several older projects like X11 and CUPS have no real developers anymore.
Yep. It's trivial to fork, not trivial to maintain. Especially when it's something where security is important - mosh will allow anyone to connect to the server directly if there is a bug there.
How many of these 600 forks have unique code i.e. aren't just snapshots of the canonical repository? A lot of people use the fork button as a way of bookmarking or mirroring in case the canonical repo disappears.
That's because Github, in their early days, got very confused about what forking was and made banners saying "Fork me on Github" when what they actually meant to say was "Star me on Github".
All I can say is THANK YOU!!! I use mosh 10+ hours a day. Maybe it needs a bit of help, but it is in my top 10 of "how did I accomplish anything with out mosh.
It looks like there's no sponsorship for Mosh. Had this been considered?
Not saying money would solve the challenges the maintainer faces but there might be things it eases. With a tool this popular I expect it wouldn't struggle to raise a fair bit.
I don't think it's been considered. To be honest, sometimes adding money to a development projects makes things worse. It can sometimes introduce new unwanted obligations that weren't there previously.
It is a VERY useful tool that I have used every day for at least 3 years.
There are two really annoying niggles that I have with it, but both I have solved by using a Screen session on the server:
- Scrolling turns out to be very important to most workflows.
The choice to start mosh in Append mode (with chance of output corruption) rather than replace mode would be a cool thing.
- There is no easy way to pick up a session again when the client dies. I understand that this is an intended security "feature" though.
But beyond this it is a truly fabulous tool that has saved me many many hours of frustration.
Thank you, this is better, but if I cat a 1000 line file I can only scroll up about 150 lines, I think this has something to do with the speed of the output(?); how does this work under the hood?
Such an indispensable tool! I’m running mosh+tmux 24/7 on my remote desktop and every day just open my laptop, connect to VPN and continue where I left off the previous day
Thank you for creating a fantastically useful tool. I use it every day, it works very well, and I'd be happy to contribute to your Github sponsors account.
I disagree. SSH is fine for short connections (like git push or running some Ansible playbook), but it's far too fragile for longer uses. For example, if you're running something like dd with no output, you end up in a situation where you're not sure if dd is still running or did your connection just break.
With mosh, I could start dd, close my laptop, move to a cafe, open up my laptop, and know for certain I'll re-establish the connection when I'm back online.
When you change wifi networks you either have to remember to manually disconnect in advance, or wait for ssh to time-out the connection, then you have to create a new ssh connection, then you have to run an extra command to re-attach to the {screen/tmux} session you started earlier.
It’s not insurmountably difficult work - but for the specific use-case of roaming across wifi networks, “doing a small disconnect and reconnect dance” with tmux is way more complicated and annoying than “open your laptop and all your connections are still working” with mosh.
Will `screen ./command` automatically and seamlessly reconnect my ssh session when I roam between different wifi networks? Or do I need to reconnect to ssh for myself, and then run a second command to reattach to the screen session?
In general, I prefer SSH when using reliable, low-latency links, mainly because of the various forwarding capabilities (agent, X11, ports), and sometimes because I need my output to be sequential. But for any kind of situation where the link might be dodgy, mosh is king.
Indeed, a common paradigm is to use mosh to connect over the internet to some bastion host, and then use ssh for all subsequent connections over a highly reliable internal network.
As someone that used to use SSH port forwarding, I have a recommendation that may be a suitable alternative to the lack of port forwarding in Mosh, as well as being an alternative to port forwarding over SSH. Wireguard! This is what I do instead of port forwarding over SSH since quite a while back now.
I run a Wireguard VPN on a VPS, and have machines connect to that VPN. This allows me to reach the machines on the VPN from almost anywhere in the world. Recently I changed the port that Wireguard is listening on to port 443 UDP, which also allows me to connect to my VPN from a few public WLANs that are very restrictive on which ports they allow outbound traffic to.
Wireguard is super easy to configure and run, and very secure.
Definitely give Wireguard a go. It's open source and awesome.
I think you could setup something like this on the fly too without root access. I’m not entirely sure, but a while back fly.io published [1] an article talking about how they use wireguard-go [2] to do something similar in user space. I might even try this too…
you can compile them yourself or if you want to skip the step I recently set up GitHub actions to compile linux binaries of this [1][2], tested by a sample of 1 so no guarantees it works, was planning on doing a tap PR/tap of it at some point
also the official developers have been involved a project to solve this while improving the whole-agent approval things also https://github.com/StanfordSNR/guardian-agent , but I couldn't get it to work which is why I tried the fork and got that working
I'm confused. I read the whole thing but couldn't find the specific reason for why it's not been merged. But I assume it's because of the things that were pointed out in the code review comments?
Also, the issue you linked is about SSH Agent forwarding, not port forwarding.
Yes you are 100 correct, I mixed up port and agent forwarding, I’ve needed both at different times and last time it was agent forwarding so got confused.
My understanding is that the maintainers prefer doing one thing well (and securely). Which to be honest is something I really appreciate even if it means I might have to figure out some agent and port forwarding workaround :-/ at least I don’t have to worry about if my version of mosh will work with whatever the server runs
I love mosh. It's great on slow connections of any sort, but particularly great on phones.
On short/medium trips I no longer bring a laptop. Just my phone and my bluetooth keyboard. I have successfully updated my rspamd DKIM signing config from a hotel lobby, among other things, with just those devices and mosh.
I'm curious, do you use any kind of terminal windowing, or what do you do about dividing up screen space / multitasking on your phone? This is one thing I really like on my laptop--multitasking with the side-by-side advantage, by keeping various windows visible at once, and using virtual workspaces with hotkeys assigned. Anyway, cool to read your endorsement.
Trying to use SSH via LTE over a mobile phone hotspot was downright horrible. It took ridiculous keepalive settings and a separate window pinging the destination to make it even halfway usable.
Then someone on another HN thread mentioned mosh. That was just what the doctor ordered. It's something I've wished I knew about years ago, given that it's also useful when Wi-Fi gets dodgy.
Love mosh, back when the connection was bad I had it aliased to ssh. Somewhat relevant: if you actually a need to take your shell mobile, check out my tmux plugin that allows you to quickly transfer your tmux session to your phone without any app, using websockets and a QR code: https://github.com/bjesus/muxile
I typically use ssh over wireguard and tmux if I need persistent sessions. Wireguard takes care of the unstable remote link and changing IPs. I can suspend my laptop on one connection, wake it up later on another and still have an active session. I use openssh via Termux for the client on Android or whatever terminal emulator on the laptop. Can mosh offer me anything?
mosh has very good latency hiding and your connection can actually be interrupted for longer than a TCP session would tolerate.
The latency hiding alone makes it worth it; SSH is miserable over links with any lag, mosh feels as responsive as typing locally until you get into hundreds of milliseconds of latency.
My setup is iPad + MBP + Linux server. I use mosh to connect, also with a Wireguard VPN, and have tmux running on the server. It's very convenient to pop open Blink on my iPad and have everything right where I left it (modulo changes made while on the MBP) and the reverse when I pull up the terminal on my MBP.
The main benefit for me is mobility, though, not the restore from sleep itself. When I started using this setup I was traveling quite a bit, including just locally (living in a downtown area, I'd start on something in the apartment, then decide to go to the coffee shop, or the growler place, and continue working). Not having to reestablish the connection (even using SSH certs that's still some friction) was very pleasant.
The second is when dealing with an unreliable network. Though this is less common these days, when I started with it I had a lot of issues in my apartment thanks to a neighbor with a noisy microwave oven.
Can you speak to the experience of coding on the iPad? I have been considering picking one up for personal reasons but the price seemed prohibitive when I thought it simply could not be used for portable work purposes at all.
It works well for me, but I have a personal preference for command line tools. I use emacs as my editor and am able to do everything I need on my Linux server (this is strictly for personal/hobby programming). There are a couple good source code aware (but not project aware) text editors for the iPad that work well with Working Copy, a really nice git client for iPad. On occasions when the Linux server was not enough, I set up Github Actions to respond to commits made through the use of Working Copy + an editor (Textastic in my case, though I'm not ecstatic about it, it does get the job done).
I'd prefer a real IDE on a few occasions (work is C# and Java). Trying to do some side projects to relearn or expand my knowledge of them and their libraries was infeasible on the iPad alone.
In the end, my conclusion is that if you don't need an IDE and can use a CLI or git-based workflow, then the iPad is a fine tool for programming and writing in general. I'm not even stymied by the relatively small screen, it's still better than the monitors I grew up with. The fullscreen and split screen modes also work well with my particular manner of maintaining focus on tasks (I use fullscreen/split screen on my MBP almost exclusively as well).
> Wireguard takes care of the unstable remote link and changing IPs.
Isn't that the point of mosh?
"Remote terminal application that allows roaming, supports intermittent connectivity, and provides intelligent local echo and line editing of user keystrokes."
Mosh is the only reason I can bear working over a satellite internet connection. The local echo is fantastic. It's also fantastic for working over VPN connections where the tunnel collapses occasionally and you don't want to lose 30 active sessions.
DomTerm has nice remote-terminal solution: https://domterm.org/Remoting-over-ssh.html . Compared to mosh is supports a much more complete xterm-like terminal, full scrollback, runs on Windows (using WSL), does not require superuser/admin access (runs on top of plain ssh), has an integrated tmux-style multiplier, and is actively developed. Like mosh it has predictive echo, and detach/attach. It also has an optional local-input-edit mode (used by default if the remote terminal is in icanon mode). An easily-installable AppImage is available (https://github.com/PerBothner/DomTerm/releases). Feedback welcome.
I know the pace of development has been pretty slow for the past few years — but Mosh is a good enough idea that I really hope it picks up steam again (better true color support, for one thing, would be a big feature)
Big fan of Mosh for what it does. But the fact that it can't render Unicode properly makes it really hard to work with some files remotely. IIRC there's a PR that fixes and has never been merged
I've been watching mosh for years, but I kept running into some sort of environment/locale issue that was getting in the way. Finally, a couple months ago I got "Linux (Beta)" available for my Chromebook and switched to using it rather than a Chrome SSH plugin to login to my work computers.
I started up mosh and it's been glorious! I used to have to reconnect every time my chromebook went to sleep for any length of time, but now I've had a single session open for ~6 weeks.
It has a nice banner when it loses connectivity, so I immediately know there are network/VPN issues. I don't love the "Control-^" escape key, because it interferes with "Last buffer" in vi, but otherwise it's worked flawlessly.
A decade or so ago I pointed a client at mosh, he had a rural property with satellite Internet and he managed a bunch of other peoples computers. He was overjoyed with mosh!
It's a neat solution, indeed. But is that actually viable and useful?
Not sure.
First, you need to open firewalls to UDP port ranges.
Second, you still need SSH to be in place and reachable.
Third, as soon as you kick off mosh you loose all bells and whistles by SSH (like TCP forwarders, SOCKS proxies etc.).
So my idea is: nice for amateur small infrastructures, pretty useless for real world (cloud) infras. Maybe you can make mosh-reachable a bastion or two, but no more than that.
Not useless, you just need to think about how to use the tool in a different way. I use mosh to login to a stable jump box and then from there use tmux/ssh to branch out all my sessions to cloud infrastructure or other devices that only accept ssh.
Local echo always felt like a handy feature for any kind of laggy terminal app, not just ssh / mosh.
I imagine such a tool would emulate a terminal a la screen / tmux, and just print what you type until the underlying program outputs to stdout, at which point it undoes what you typed and replays the stdout? I realise I may be betraying a complete lack of understanding of how Unix terminals actually work :)
Is there a Unix utility that implements a local-echo wrapper?
Hmm, rlwrap[0] is a really nice tool when working with applications that are poor at handling terminal input. It basically handles your keystrokes in rlwrap and whenever you type enter it will send the input line to the application to handle. It's very useful also for application that don't have their own input handling (arrow keys, etc.) or have extreme keystroke latency. It will even give you ctrl+r feature to search commands you typed earlier.
I use it often for pentesting when getting a raw reverse shell. I'll get access to a remote bash session through netcat, and using rlwrap I can do the line editing locally and find the complex commands I've written before without copypaste.
To me, conceptually, mosh is more like "VNC for character terminals." Overall, I think it's a brilliant solution for terminal sessions over links not intended for terminal sessions.
I always want to try alternative shells but I always come back to bash. I think Fish had the longest residency. I have yet to find the killer reason to never come back.
It’s a bit like programming languages. So much cool stuff out there but it’s hard for me to leave Python. Like Bash, everything works with it.
I’m curious what people’s “killer apps” are for permanently changing shells.
I have this same problem. I’ve tried fish and zsh but all my scripts are in bash and over the years I’ve honed an environment I can just git clone to any Linux machine and it works
The way I see it, mosh has an advantage over et: while mosh is started on connection as the remote and unprivileged user, et relies on etserver running as root on the remote box (after connection is established, etterminal runs as the unprivileged user, though), so any vulnerabilities in etserver may have serious consequences.
I don't believe it's identical at all. Eternal terminal is a TCP-like abstraction that adds another layer of tracking and acknowledgement so that the reliable ordered byte stream can be reconnected and resumed. Mosh is more like a video codec mixed with a terminal that knows how to do limited, speculative local echo that gets subsequently corrected once the display round trip is complete. They're fairly different in approaches.
For example eternal terminal has to send all of the bytes in order, so more output requires more bandwidth. Mosh only has to redraw bits of the frame that have changed, at a presumably adaptive rate -- you could scroll a hundred megabytes of text by really quickly, but only need to send a few screen-fulls of updates, which is of course why it doesn't allow for regular terminal scrollback.
No, it doesn’t. The best way I understand it is that mosh syncs a “view” of your current session, whereas et just sends everything. In practice, what this means is if I accidentally cat a 5GB file, mosh zips past as if I were working locally (and ^C works!) whereas et completely locks up until the file reaches its end. That said I still prefer and use et daily due to the scrollback and tmux -CC support.
Mosh is perfect for ephemeral client connections to a long-standing session. My ipad likes to kill ssh sessions every time the screen locks. But using mosh insulates me from that and I just go back to a hot shell with no interruption.
Re: Unicode, please see https://news.ycombinator.com/item?id=28151652. (Mosh really just uses your C library for knowing the width of characters -- if the server's and client's libc support a character, Mosh will too. Unfortunately Apple in particular has historically been not super-great about keeping theirs up-to-date once Unicode went to an annual release cycle. screen and tmux have similar issues and basically ship their own Unicode tables, which has its own problems.)
It's the tragedy of open source. Lots of times, the software used by hundreds of FAANG developers is maintained by a single overworked developer in their spare time.
How much do you want? We'd very much welcome your involvement -- we'd love to have you start helping out in the IRC channel, help us put together a credible plan for the next release (probably helping the current maintainers), and let's talk.
https://github.com/mobile-shell/mosh/issues/1115