Thanks for the concise and extremely helpful answers. I have some followups:
* What is the Jit itself written in?
* You mention that for numeric code PyPy hovers around gcc -O1. What are the other classes of code that you see PyPy as having to deal with? And how would it perform compared to normal C on those?
* Very interesting comments re the benchmark game. Did anyone ever optimise some of these tasks specifically for PyPy?
* Just a clarification re Cython. Yes, I believe it was forked from Pyrex which was mainly for nailing up C code I think. Though since that time my impression is it has become its own language, similar in parts to C/C++ and similar in parts to Python. They appear to have put a fair bit of work into making it fast. It is compiled though (to C), not interpreted. There was some talk about them trying to get it into CPython. But I'm not sure what was meant by that.
One final followup:
* Do you know if there are any plans to develop a new language built on top of the PyPy VM similar to Python but extended in some way? For example would it be feasible to implement a statically typed language with or without type inference on top of the same technology?
By the way, if you google virtual machine you get pages and pages of links to LLVM, JVM, Parrot and Dalvik (go figure) and the occasional reference to a few others. I'm really surprised that the PyPy VM is not better known!
* Comparing code with C is pretty difficult for all but the easiest 1-1 ports (i.e. numeric stuff), so I don't have good comparison numbers for it.
* Yes, I tried to improve the submissions to the benchmark game, this led down a long path which resulted in the benchmark game only having a single implementation for each language (e.g. tracemonkey is no longer on there, only v8)
* We don't plan on writing a new language, we kind of like Python, but someone absolutely could.
I think I understand now. The PyPy implementation of Python is written using the RPython -> Jit interpreter framework that the PyPy project has. Clever.
Am I right in thinking that the drawback of this method is very long compile times for producing the interpreter?
Yes, compile times are rather long, about 40 minutes for the full interpreter on a recent, fairly good, laptop. On the other hand you don't need to do a compile all that frequently, because all the code is testable by executing directly atop another Python.
Cython just has a syntax for annotating your code with types so it can work its magic and make it go fast, although you can add the annotations to a separate file and keep your code in pure Python.
I'm not sure if they're going to use Python 3's annotations to make this cleaner, that might be worth exploring...
* What is the Jit itself written in?
* You mention that for numeric code PyPy hovers around gcc -O1. What are the other classes of code that you see PyPy as having to deal with? And how would it perform compared to normal C on those?
* Very interesting comments re the benchmark game. Did anyone ever optimise some of these tasks specifically for PyPy?
* Just a clarification re Cython. Yes, I believe it was forked from Pyrex which was mainly for nailing up C code I think. Though since that time my impression is it has become its own language, similar in parts to C/C++ and similar in parts to Python. They appear to have put a fair bit of work into making it fast. It is compiled though (to C), not interpreted. There was some talk about them trying to get it into CPython. But I'm not sure what was meant by that.
One final followup:
* Do you know if there are any plans to develop a new language built on top of the PyPy VM similar to Python but extended in some way? For example would it be feasible to implement a statically typed language with or without type inference on top of the same technology?
By the way, if you google virtual machine you get pages and pages of links to LLVM, JVM, Parrot and Dalvik (go figure) and the occasional reference to a few others. I'm really surprised that the PyPy VM is not better known!