Welp, I'm not entirely surprised. GPT-3 already understands programming "language". It does know what should come after an expression, and also can process a certain amount of logic and context, which has been intensively demonstrated using human language. That's all we need.

On top of that, ChatGPT comes with a big implicit memory, so you can simulate a REPL session without manually appending stuffs to your prompt.