I was thinking about legal monopolies or companies barely profitable by virtue of tax credits/breaks or companies that don't have to follow certain costly regulations for some (legal) reason.

Would those factors actually make them "not really profitable" or "not actually private businesses"? Or is this where the "In a sense" part of your reply breaks down? (These are legitimate questions, but I'm afraid it might sound snarky.)

I'm pretty sure he meant "not actually private", in a sense of something like "any organization dependent on the government for survival can be regarded, to some extent, as being effectively a branch of the government".

Of course, by that standard, I'm not sure how many "truly private" companies exist, since it seems to rule out (among others) any company that relies on intellectual property law, including trademark enforcement to prevent bootlegs and cheap imitations.

Sure there are. Take General Dynamics, for example. A private company that sells billions of dollars of military hardware to the government.