And it's much more deeply rooted in human nature than merely the design of peer review or the quantitative metrics used for evaluating scientists.
Indeed, it is not just about money either. Intangible prestige and status among the scientific expert community is just as much, or even more, coveted. Do people read your papers and talk about it at dinner parties? Do you get invited to give talks at prestigious institutions? Do a lot of interesting and similarly active people turn out to your talks? Do people with good connections and resources want to collaborate with you on exciting ideas?
And it turns out that what people including scientists actually care about is novel, bold, visionary ideas, not drone-like repetitive meticulous detail-oriented work following the footsteps of some other group. People want something new, something cool, something flashy, something sexy, something surprising. Not just the media! Scientists themselves, too!
Most scientists only want something flashy within the confines of the "cookbook" they've already learned on how to do studies. NIH grant applications are formulaic as hell.
Moreover, many labs organize around a methodology like fMRI rather than around any particular type of question. Imagine planning a multi-decade future career around a single tool that exists today and then pretending you truly care about novelty.
You're spot on that people care deeply about academic status. But what is valued in gaining that status has become deeply broken. The fastest way to status is to put out multiple overhyped individual publications that meet the minimum viable novelty threshold for inclusion in a good journal. Half of the battle is a marketing game.
Increased rigor is a time and money commitment that many aren't willing to make, but true novelty is a much bigger risk and a good way to kill a career for anyone not yet tenured.
My larger point is that this is a general human problem, not a science-specific one. Voters listen to the flashy corrupt demagogue politician who speaks to emotions, not the boring one who speaks in nuance and works transparently. In dating people complain about the other gender being shallow and overlooking deeper values. On TV, people watch garbage reality shows so those make the most money. On YouTube, people click thumbnails with obnoxious facial expressions so those win out. In the movies the safe bet is to churn out films from the same franchises as before (not unlike the scientist who barely changes things between papers).
It's not gonna change, one has to learn to accept it and to adapt to it.
Indeed, it is not just about money either. Intangible prestige and status among the scientific expert community is just as much, or even more, coveted. Do people read your papers and talk about it at dinner parties? Do you get invited to give talks at prestigious institutions? Do a lot of interesting and similarly active people turn out to your talks? Do people with good connections and resources want to collaborate with you on exciting ideas?
And it turns out that what people including scientists actually care about is novel, bold, visionary ideas, not drone-like repetitive meticulous detail-oriented work following the footsteps of some other group. People want something new, something cool, something flashy, something sexy, something surprising. Not just the media! Scientists themselves, too!