Sure, but when we worry about jailbreaks we worry about people doing bad things with the knowledge.
The worry is about someone seriously asking the question and getting serious answers.
You could do that by jailbreaking an LLM. My contention is you can’t readily jailbreak most humans this way - not seriously. People would get uncomfortable quickly.
I think that's what we're getting at here. I don't want to actually murder people