I wish the LLMs would tell you exactly what the input was (system prompt, memory, etc, at least, the ones we have control over, not necessarily their system prompts) that resulted in the output.
Also, out of curiosity, do you use LLMs for coding? Claude Code, Cursor, etc? I think it's a good idea to limit llm conversations to one input message but it makes me wonder how that could work with code generation given that the first step is often NOT to generate code but to plan? Pipe the plan to a new conversation?
The basic process is that you use a "plan mode" with whatever model is good at planning. Sometimes it's the same model, but not always.
You refine your plan and go into details as much as you feel necessary.
Then you switch to act mode (letting the model access the local filesystem) and tell it to write the plan to docs/ACDC1234_feature_plan.md or whatever is your system. I personally ask them to make github issues from tasks using the `gh` command line tool.
Then you clear context, maybe switch to a coding model, tell it to read the plan and start working.
If you want to be fancy, you can ask the plan system to write down the plan "as a markdown checklist" and tell the code model to check each task from the file after it's complete.
This way you can easily reset context if you're running out and ask a fresh one to start where the previous one left off.
I use plan mode, but then I let it go using its own todo tool and trust its auto-compaction to deal with context size. It seems to almost always work out okay.
The rule of thumb is that when you've compacted, you've already lost. But YMMV.
The internal todo list works well if the task is something that can be completed within one context pass, otherwise it should be an external task list - whatever works for your flow, markdown, github issues, memory MCP etc.
Also, out of curiosity, do you use LLMs for coding? Claude Code, Cursor, etc? I think it's a good idea to limit llm conversations to one input message but it makes me wonder how that could work with code generation given that the first step is often NOT to generate code but to plan? Pipe the plan to a new conversation?