In France, you can only pay certain government filing fees (driver's license for instance) by using "fiscal stamps" which are postage-type stamps only available at certain retail outlets. They won't accept cash, check, or credit card, only stamps.
Same in the UK, I was exchanging my EU licence for a UK one, and the only way to pay for it was to either include a cheque(I have literally never seen a cheque, I asked my bank how to get a cheque book and I would have to wait 3 weeks for one so I said no) or go to the post office and buy a "postal order" for the same amount and send it to DVLA. Most other things you can pay for using a debit card though.