Hey HN! We started Jamsocket a few years ago as a way to run ephemeral servers that last for as long as a WebSocket connection. We sandboxed those servers, so with the rise of LLMs we started to see people use them for arbitrary code execution.
While this works, it was clunkier than what we would have wanted in a first-principles code execution product. We built ForeverVM from scratch to be that product.
In particular, it felt clunky for app developers to have to think about sandboxes starting and stopping, so the core tenet of ForeverVM is using memory snapshotting to create the abstraction of a Python REPL that lives forever.
When you go on our site, you are given a live Python repl, try it out!
---
Edit: here's a bit more about why/when/how this can be used:
LLMs are often given extra abilities through "tools", which are generally wrappers around API calls. For a lot of tasks (sending an email, fetching data from well-known sources), the LLM knows how to write Python code to accomplish the same.
Any time the LLM needs to do a specific calculation or process data in a loop, we find it is better to generate code than try to do this in the LLM itself.
We have an integration with Anthropic's Model Context Protocol, which is also supported by a lot of IDEs like Cursor and Windsurf. One surprising thing we've found is that once installed, when we ask a question about Python, the LLM will see that ForeverVM is available as a tool and verify it automatically! So we cut down on hallucinations that way.
What are some of the edge cases where ForeverVM works and doesn't work? I don't see anything in the documentation about installing new packages, do you pre-bake what is available, and how can you see what libraries are available?
I do like that it seems the ForeverVM REPL also captures the state of the local drive (eg. can open a file, write to it, and then read from it).
For context on what I've tried: I used CRIU[1] to make the dumps of the process state and then would reload them. It worked for basic things, but ran into the issues stated above and abandoned the project. (I was trying to create a stack / undo context for REPLs that LLMs could use, since they often put themselves into bad states, and reverting to previous states seemed useful). If I remember correctly, I also ran into issues because capturing the various outputs (ipython capture_output concepts) proved to be difficult outside of a jupyter environment, and jupyter environments themselves were even harder to snapshot. In the end I settled for ephemeral but still real-server jupyter kernels where I via wrapper managed locals() and globals() as a cache, and would re-execute commands in order to rebuild state after the server restarts / crashes. This allowed me to also pip install new packages as well, so it proved more useful than simply static building my image/environment. But, I did lose the "serialization" property of the machine state, which was something I wanted.
That said, even though I personally abanonded the project, I still hold onto the dream of a full Tree/Graph of VMs (where each edge is code that is executed), and each VM state can be analyzed (files, memory, etc.). Love what ForeverVM is doing and the early promise here.
[1] https://criu.org/Main_Page