Wow, this couldn't be further from the truth. It might be true for DJs playing "main stage" style EDM (poppy mainstream music) but for most electronic subgenres – especially techno – the crowd absolutely expects the DJ to be a superb crate digger and pull out new and deep tracks they've never heard before.
No one goes to a techno club to hear rinsed tracks; they want the DJ to show them music they've never heard before. Before the digital age, people would go out to see touring DJs specifically for their collection of rare records that no one else had and you couldn't hear anywhere else. This is still true today in the more underground scenes. It's the opposite of cookie cutter.
Most DJs do not make their livings at techno clubs. The majority are hired to play bars and events that do not cater to particularly discerning audiences.
Exactly. The DJs that are innovative, crate digging, slightly pretentious music nerds are almost exclusively hobbyists, with a vanishingly tiny percentage of them being able to eek out a meagre living from it. The majority of full time DJs cater to mainstream audiences who absolutely do want to hear the same 50 - 100 tracks on rotation every time they go out. They want to dance and sing along to music that they're familiar with, and if the DJ doesn't play what they know then they won't dance, they won't stay, and the DJ won't remain employed.
It's actually very similar to web design - innovation has its place, but 99% of the time people want familiarity.
A typical DJ is allowed one weird song nobody has heard before. If it is a long show maybe one per hour. The rest better be songs the majority of people know and sign along with.
You might have a bit of confirmation bias based on the particular environments you've been in.
Having taken part in various types of electronic music/art scenes since the early 2000's, I've met all kinds of people. Local hobbyist bedroom producers playing for free. Semi-professional artists juggling gigs&touring with one or more side jobs. Full-time DJ's playing everything from small underground parties to some of the biggest parties/festivals at the time. They all cater to their audience to varying degrees, mainstream or not.
Granted, the scenes I've bumped into tend to be on the non-mainstream side. That's where you can actually go professional being that "innovative, crate digging music nerd" you refer to (removed "slightly pretentious" because that hasn't been my experience). It's tough, but it can be done, and it's a larger group of people than you seem to think.
I've also met some professional DJ's that fully cater to the audience in the way you describe. Many of them make statements like yours like e.g. "99% of the time", "almost exclusively hobbyists", "slightly pretentious", etc. I really don't get why, because it's just not true, and it comes across as a bit defensive or passive-aggressive to be honest.
I mean, of course there is the mainstream audience of the type you describe. But even that audience changes its opinion about which 50-100 tracks they expect you to play on a regular basis. That change has to come from somewhere, otherwise they'd still demand disco tracks from the 70's. That somewhere is the stuff that hasn't gone mainstream yet, and while the percentage of people that can make a living of it is probably not very high, it's a lot higher than what you claim it to be.
That vast overlap between underground/alternative scenes and the mainstream is super interesting, and I'm pretty sure that if you included that part into your statistics, you'd see a different picture.
NB: I might have a bit of confirmation bias based on the particular environments I've been in ;)
It's like the perfect conterexample, a good DJ needs to have really good taste and constantly listen to new tracks and think about where they can be used.
Designers everyone thinks are more creative than they are. DJs most people think are less creative than they need to be.
Sounds to me like they work out to be the same in this respect, only differing in perception. Both are solving problems with approximately similar levels of caching ideas and techniques and experience, while developing proficiency with similar quantities of tools. They're both generally making their money solving problems with all of that, only occasionally getting a chance to express themselves in some unusual artistic way that most likely won't be a viable income stream until they've really made a name for themselves; save for the rare circumstance they are Aphex Twin
No one goes to a techno club to hear rinsed tracks; they want the DJ to show them music they've never heard before. Before the digital age, people would go out to see touring DJs specifically for their collection of rare records that no one else had and you couldn't hear anywhere else. This is still true today in the more underground scenes. It's the opposite of cookie cutter.