If you don't brush, do you get more plaque build up? From what I've understood, people generally can have mouths that are more basic/acidic depending. If basic, plaque tends to be a bigger issue but few if any issues with cavities is common. The converse for mouths that trend acidic.
I had a thing about avoiding fluoride toothpaste, since I was fed fluoride pills daily as a child. So I just stopped using toothpaste for 20 years or so; I'd brush with water and then use mouthwash. I definitely built up quite a bit of protective plaque. I guess it's not real pretty, but I think you're right if what you mean is that it guards against cavities.
[edit] to be clear, it's not like orcishly heinous or coated with nasty algae or something; I even have a somewhat disarming smile, it's just not gleaming bleached white.