One approach might be to redact sensitive parts of the input, replacing private data with tokens. Then substitute the tokens back again in the output.

But this only works if the sensitive data isn't needed for inference and you have a reliable way of detecting it.