This all boils down to how much you trust your management with the long-term health of the business. Not refactoring may actually be the right call. Maybe it will become a necessity later, maybe not.

If management just wants to flex their authority, you can inflate your initial estimates. It's only unfair when one side treats an estimate as a negotiation and the other side doesn't.