No reason to do this - you have the shares and the CEO wants to buy you out. Remember - the CEO wants to buy you out for a reason, and I guarantee you the investors want you bought out. Don't underestimate how much leverage "them wanting you bought out" is. It may even be a stipulation of the funding round.

Do the investors even know about the conversation?

People paying good money for shares will jump at the chance to buy them cheaper.

A CEO closing an investment round may have very different incentives, like not frightening the horses.

[edit, clarity]

It would be a bad sign if the investors weren't asking who the ex-employee with 5% of the cap table was!

Sure, but the investors would probably want to buy you out ALSO.

Equity is a market. The CEO is offering a price. Maybe the investors are willing to offer a higher price.