But you do: you have to give PayPal your payment information.

Sure, you don't have to give other people your information, but honestly there are few people that I’d want to business with that I feel less comfortable giving payment information to than PayPal.