Currently, all code is executed server-side (on a Node VM Module created on Worker Threads, using Hono+Websocket to provide the ability to run it in Minecraft), so it is not possible to run Turtle inside the screen. However, if there is a lot of desire for such functionality, it may be possible to use a VM (such as scratch-vm) within the browser to run the code.
Correct, I am using OpenAI through the software, includes, gpt-4o, 4o-audio-preview(for multi-modal response, includes audio and text), and whisper(speech to text for generating transcript for user).
When I teach people to learn to code in Minecraft, I basically recommend using Minecraft Education Edition. This combines the Bedrock edition with the original editor called Makecode. They run as two separate apps within the application and are connected via Websockets. Websockets are supported in both the Bedrock and Education Editions, but the editor is only included in the Education.Java Edition does not support Websockets and requires the use of a mod or server.
By the way, TutoriaLLM hacks this mechanism of Minecrafft to create code in the browser and run it in Minecraft Bedrock without installation on a small VM thread created on the server. Currently only some of the functionality is available as it is still under development, but in the future it will offer the same functionality as Minecraft Education for free (at least until Microsoft finds it and block them...).
Hmm but I wonder whether people who already have it can continue using it?
I don't mind the extra $12 per year for Minecraft Education Edition (although it seems unfair when we already paid for Bedrock when this functionality was included), but the sign up process is really convoluted, so much so that the support team made an 8 minute YouTube video walking people through it!
As this service is under development, so there are no active users. But user testing has already started in programming schools in Japan with 10~20 kids from primary school. In the future, the aim is to make the software available to programming schools and primary schools worldwide.
