This is exactly what I've been trying to figure out. At some point the LLM needs to produce text, even if it is structured outputs, and to do that it needs careful prompting. I'd love to see how that works.