> This sounds like you're describing a system that is a failure but that the solution is not to fix the system but to give up and do something else. Can we not fix academia?

The problem with academia is basic mathematics. In a system where there are a fixed number of tenured professorships, where each tenured professor has the job for life, each professor should produce only a single tenure-track PhD student in their entire career.

OK, so to deal with attrition and other unforeseen circumstances, perhaps 10 tenured professors should produce 11 tenure-track PhD students. But definitely not the situation we have now, where each professor produces dozens of PhD students in the course of their career who go on to attempt to get tenure.

The people trying to get tenure don't want to admit the system is broken, because that would be to admit they just weren't good enough to get tenure.

The people who have tenure have every incentive to keep it going, because it means they have an army of highly-skilled and highly-motivated postdocs willing to work long hours for peanuts for a decade or two.

It's a bit worse than that. Prestige matters a lot too. General rule of thumb is that you can only apply to a position at a university of equal or lesser ranking than the one you were previously at. (obviously can still climb the ladder, in many ways, but this is a strong pattern all throughout the process from High School to Tenure) You're exactly right about the number of positions problem too. Because clearly the result of this is that the quality and methods from higher ranking institutions diffuse into lower ranking ones. Hell, when I was at a community college I had a professor from Harvard and several from Berkeley. Prestigue doesn't make sense in a system like that because you can't maintain an edge when you are literally telling the people you trained to go work somewhere else.

I think you're exactly right about the reasons people keep quiet. But this is not helpful to anyone, especially the universities. They are certainly losing a lot of money and even prestige from all of this. You don't make Nobel laureates with publish or perish. But no one wants to shake things up, which is weird because academia is __explicitly__ supposed to be the place where you can focus on things that aren't profit driven. Or at least short ROI. It is a loss for the country too, as it means a lot of academics move away from low risky TRL research and follow a model much closer to industry research (which is profit driven) Historically industry has (generally) relied on academic research doing low TRL and then they bring it to mid and high TRL.

We've lost sight of what we're trying to accomplish.

I just wanted to comment regarding the rule you mentioned and wanted to say that I was always told that, unofficially, you can always move up but only a tier at a time and you don’t want to move down if you can avoid it because then you likely have to move back up. The calculus does get harder though the longer you’re in the game — you might elect to move down because you’re significant other gets a promotion or you don’t want to uproot your family, etc.

For sure, they aren't hard rules. There is also the aspect that locations matter to some people. It's worth noting though that universities have a unique negotiating token, in that you can also negotiate for a job for your partner as well. I think universities are often forced into this because of the natural location constraints.

I think they may have been commenting a bit more about the incentive structure and that any "prestigious researcher" or institution typically got there via questionable ethics, etc.

First, a postdoc on the way out of academia is not going to fix academia.

Second, how is the system broken? There are mismanaged resources, some nepotism (in the U.S., a lot of nepotism if we look at countries other than the U.S. or the like), some research questions that are (wrongly) favored, but it doesn't change the fact that many more PhDs with academic career ambitions are being produced than there are (and will be) positions available. And this is not just an academia problem: the same mismatch between supply and demand is found in many other creative and aspirational careers, think actors, singers, sports. Only a small percentage of those who want that life can get it, and those who are left out are often frustrated by the perceived unfairness of life. But that does not change the fact that many aspiring creatives, because of that massive mismatch between supply and demand, will not be able to pursue those careers.

Sure, more permanent research positions could be thought of instead of forcing careers through the very narrow bottleneck of tenure track positions, but the vast majority of PhDs with academic ambitions (at least 8 out 10) will not have the opportunity to make that career, and they are (or are themselves) being bread-crumbed for years and years hoping that their dreams will, one day, come true. But they won't come true. And, as soon as they find another fulfilling occupation, they will find out it was not a real dream anyway, just a dream they thought they had.

> And this is not just an academia problem: the same mismatch between supply and demand is found in many other creative and aspirational careers, think actors, singers, sports

I don't often hear people complaining that sports, music, theatre are broken because of this supply/demand mismatch. (People certainly complain about other problems, e.g. streaming music royalties). It's generally accepted that it's a rare and special thing to make a good living in those industries. Why do we view academia differently? Is it because it seems more like "work"?

I think it's because people who do research are perceived as someone who wants, out of good heart, to provide a benefit to humanity.

"How can you not want me to work on this enzyme pathway that can explain how this very rare and aggressive form of cancer develops?" sounds different from "How can you not want me to star in this silly sitcom?". I used extreme examples, but still.

Interesting. That perception underestimates how much researchers like doing research I think. The demand shortage and price paid is just function of how much society values research versus entertainment.

That may be so.

But it is the same when we talk about surgeons and teachers, who, in the collective subconscious, are seen as saintly figures, people who have dedicated themselves to the betterment of society. But the vast majority of surgeons and teachers love to be surgeons and teachers, love to cut and repair and explain things to young people. In fact, people who teach, for example, sales to employees of a company basically do the same job as high school teachers, but we do not consider them as worthy of our admiration.

And as for the price paid by society for research, the public knows approximately nothing about research. And when they think about scientists they think about Albert Einstein, people of genius frequently lost in their thoughts solving equations. But if they knew the reality of most universities, departments, and research groups, they would be willing to pay way less than the state/government is paying scientists and other researchers for their, I dare to say, often meaningless investigations.

> Second, how is the system broken?

I want to d my best to respond, but can you help me determine my audience? That way I can be clear? Are you coming from inside or outside academia? If inside, what field? If outside, have you gone through a graduate program? Which decade? No need to give highly specific answers, I'm just trying to get some additional context to best respond. There are problems more visible to those inside and often nuanced and problems that are invisible to outside.

I was referring to "system broken" with respect to many PhDs with ambitions of an academic career having to leave academia because there are not enough tenure-track positions available. With respect to that problem, it would be equivalent to say that "Hollywood is broken" because many talented aspiring actors cannot find stable work in films and tv shows. It is a supply and demand mismatch problem.

I was not referring to the paper review and publication process, funding, grants, teaching etc.

Your audience is someone with a PhD in the biological sciences, with more than 10 years of postdoc experience (before leaving for greener pastures a few years ago, after having realized that I was not interested in tenure-track positions anymore and in any case no position had been offered to me) in Europe and in the US, who won grants, awards, competitive scholarships, and have dozens of first-author papers published in some of the most prestigious disciplinary and interdisciplinary journals.

> I was referring to ... many PhDs with ambitions of an academic career having to leave academia...

I see. I don't think this makes the system broken and I don't think a large portion of people going into industry is a bad thing. At the end of the day, schooling, even at the PhD level, is training. I am not concerned that we do not have a high retention rate, though I am concerned that the retention is not targeting the best of the best (but that's a different conversation). I do not think this is what most academics are referring to when they mention a broken system.

My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting. This is the more frequent usage of this phrasing that I see, so I was quite surprised at the mention of your background. The problem I am more concerned about is the entire existential question of academia. It is being treated like a business but academia is explicitly about not being subject to such things like a short term return on investment. I think an important aspect of academia is being able to do risky and low level research where the returns are going to be years or decades. So I think we have a broken system when we are trying to perform high pace output with high impact. I simply do not think you can be fast and revolutionary at the same time (at least consistently). It seems that h-index is of greater concern and this is a compounding metric that does not requisite the highest quality work but rather is more dependent upon frequency and highly influenced by publicity.

In my mind, the system is broken when the it is not encouraging academics to be the best scientists they can be. When the system encourages bad science and when a noisy process is held up as if it is the arbiter of truth (journals and conferences are even given the misnomer of "peer review" when this, as you know, takes place before and after venue publication and peers still review "preprint" works that never end up getting "published." The whole language is misleading when you break it down). I'm sure a lot of my feelings come from working in ML where the signal to noise ratio of venue publications is exceptionally low (nearly all CS domains target conferences rather than journals and so there is little to no rebuttal period. The number of submissions in each conference is well over 10k now and you cannot assume a reviewer has domain knowledge in your subfield), but I do believe that it still parallels other domains and that the problem is growing not shrinking. I think the system is broken when we discourage foundational research and the pursuit of knowledge for scientific value. Certainly something is broken when we hire a ton of administrative staff and the administrative tasks given to researchers only increases.

> My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting.

We have to keep in mind that those incentives, targets, and metrics were set up by the academics themselves. It is the same in funding agencies and departments.

I'd be a bit hard pressed to agree with this. While I think it is technically correct I do not think it is meaningful. Because anyone working in the academic system is specifically considered an academic. But a researcher is not the same thing as an admin. The truth is that a dean is not the same thing as a researcher even if they previously were.

I'd specify that the metrics were setup by the bureaucrats, which does include overlap between administrators and researchers. I'd also add that just because the metrics were made up by these groups doesn't mean they can't decide to move past them. In fact, it even makes the case of internally solving the problem stronger because we did it before so we can do it again.

> First, a postdoc on the way out of academia is not going to fix academia.

I disagree: just found an edtech startup for tertiary education or (post-)graduate level research.

> just found an edtech startup for tertiary education or (post-)graduate level research.

How is an edtech startup going to solve post-graduate research, an enterprise that involves millions of people around the world working on all fields under the sun, and is, by its very nature, unprofitable?

> How is an edtech startup going to solve post-graduate research, an enterprise that involves millions of people around the world working on all fields under the sun, and is, by its very nature, unprofitable?

Well, I have two ideas:

1. Concentrate on academic disciplines where research is "less unprofitable" than others, and by using well-known methods of business administration make it profitable.

2. I know quite a lot of people who were previously in academic research, and now (because of job perspectives in the academic job market) work somewhere else in some industry. Because they still love research, they spend quite some money on post-graduate level textbooks and similar things. People who might not be rich, but love spending money on learning does not sound like the worst foundation for a business modell ... ;-)

Honestly if I knew how to do this I would do it. How do I convince a bunch of billionaires to hand me money to do very fundamental research? Dream would be to have it be very open too and hire people from multiple domains but keep a high focus on being nuance and detail oriented. Honestly, I don't think companies like OpenAI are going to end up being the first to AGI because they get caught in the profit stages and so just pivot to say that more data and more compute is all you need. Maybe it is true, but I doubt it. And most certainly there are other ways considering a 3 pound piece of meat does it using less than a kW per day.

> Honestly if I knew how to do this I would do it.

Do an MVP in your spare time.

I guess if you have a normal programming job in most countries (even in countries where programmers are not paid that well), you will likely be able to pay the bills for, say, server renting, domains, ...

For things that you will do likely badly by yourself like design drafts (if you are bad at visual design), ask some friends whether they would be willing to do it for you in exchange for something that you could do for them (so that you don't have to spend money).

Start with kinds of research that can be done with little money, but have a possible insanely high impact.

Iterate until your platform cannot be ignored anymore.

> Do an MVP in your spare time.

I think you gravely misread my comment if this is your first suggestion. I am essentially saying that what I would want to do is closer to starting something akin to DeepMind or what OpenAI looked like in the beginning. There is no clear minimal viable product because the goal is to create general intelligence. There are many demonstrations of narrow cases of generalizability and I even have work that demonstrates this, but the stronger sense and actually trying to build sentient machines.

I would make the argument that there is an advantage in what I'm proposing. Since the existing labs that are trying to perform a similar goal are all following similar paths. The chance of beating them using similar methods is quite low. But if you believe (as many researchers do) that the conventional approach is not correct or not efficient, then it reasons that you should take an unconventional approach. I'd also say that my preferred method can be great publicity for them if we do allow for a fully open research platform (open training/models/communication) and given the goals the computational costs are not quite what they are for OpenAI nor DeepMind. As I believe I could do a lot of work with a small team with only a handful of A100 or H100 nodes. That's pocket change to the Musk types and especially if split between multiple entities. It is a high risk high reward situation but I think also low cost.

> There is no clear minimal viable product because the goal is to create general intelligence. [...] I would make the argument that there is an advantage in what I'm proposing.

Then set the MVP to some project that partially solves some small aspect of what you general intelligence is and proves that your approach is so much better than what other labs are doing.

If you succeed on this, you can iterate.

> if we do allow for a fully open research platform (open training/models/communication) and given the goals the computational costs are not quite what they are for OpenAI nor DeepMind.

Then work on a way to decrease these computational costs by magnitudes. Or find a way how even volunteers with "skinny" GPUs in their computers can still support your computations via crowdcomputing.

Perhaps this is even the more important problem to solve ...

When I was still a PhD student, my advisor lamented that he couldn’t find postdocs to hire because big tech was paying them more for the same role with minimal constraints. If that kept going for a while, I imagine it’d put pressure on at least some parts of academia (CS and engineering don’t expect you to do a series of postdocs like biology does, and pure math has their own system of seniority I only know a little about). The money dried up though.

I literally went through this recently with someone in my department. They wanted to know what professors did similar research to me at R1 universities of comparable rank. I told them no one and they didn't believe me. I sent them an annotated version of a bibliography to a survey paper I had recently written, writing down where each person worked. I just don't think they understand that right now, as an intern for a big tech company I am making more money than some of our junior professors. When I can get a full time position my cash in hand compensation goes up, plus equity and more benefits (like 401k). Who the hell would go into academia? You have lower pay, fewer resources, and more bureaucratic BS. There's lots of bureaucracy in industry too, don't get me wrong, but it's easier to deal with when you're getting paid more.

A post doc position wants me to move across the country, into a major city, and pay me $50k/yr for a position with low growth opportunities and where I will have to move again in another few years? No thanks.

> I'm being asked to do things that have never been done before -- and I love this, it is the ultimate puzzle -- but how the fuck do you expect me to give accurate ETAs and to do this 3-5 times a year and launch ground breaking work with some 2080Ti nodes and maybe one A100 node?

If it is work that some industry is highly interested in, and this industry has deep pockets, it is very hard to compete with it in academia. Look for more fundamental questions to research (there exist insanely many, just don't follow the hype) that can be investigated with a lot less ressources.

Fwiw, this is the route most academics take, but it is still not easy to publish in this domain. I've seen workshop papers get rejected from very clearly academic work because they don't use real world or large datasets. I've seen theory papers get rejected because they don't have experiments. Yes, read that last one again. And mind you, these big companies also do a lot of theory and foundational work too. The process is just noisy.

I think having trouble finding a postdoc is a good outcome of the process, the incentives in this case are working properly. Postdoc positions were designed as a short bridge between a PhD and a tenure-track position, not as a long-term crutch for people who are not ready to leave academic research.

When postdoc positions first were created, they were illusive and of high prestige. Then the idea got abused to add labor for professors to varying degrees depending on the field.

We are in the big science phase of biology using AI. Individual labs and researchers will be hard pressed to compete but the worse part is the lack of original thinking.

In the academic system, positions are globally accessible and therefore hyper-competitive. Moreover, headcount grows slowly as it depends on funding and attrition through retirement/death. As such, demand outstrips supply. Strip everything extraneous away and that's the heart of it.

If enough post-docs leave, it will reduce the demand for tenure-track positions. Eventually fewer PhDs will become post-docs, further decreasing demand.

> This sounds like you're describing a system that is a failure but that the solution is not to fix the system but to give up and do something else. Can we not fix academia? Is the signal of so many people, especially in CS, abandoning academia for industry not as clear of a sign as you can get?

Not enough people have left yet. Until enough people leave that academia has a hard time finding talent that it needs, as far as they're concerned there's no problem.

> Until enough people leave that academia has a hard time finding talent

That's going to be a long time. If by talent you mean people who can do the job. But that doesn't fit the story that universities sell, which is about learning from the cream of the crop.