Honestly, I find it kind of wild that someone would spend a big chunk of their time at work trying to drag down someone else's work. Fine, if you really think that someone has done something egregiously fraudulent, then you probably should pursue it. But to spend so long working on a rebuttal to a paper which basically boils down to "Well your benchmarks aren't good enough" is pretty weak. I've spent a lot of time looking at research papers - the benchmarks are always weak.
Also, if you're actually interested in writing a paper on "What are the best techniques for place and route" then write a paper "Best Modern Place and Route algorithms" not "Here's why this particular paper my colleagues wrote is wrong".
Personally, I think they should've published his paper - they can continue actually using the work internally to improve their TPU designs whilst he publicly tries to persuade their competitors it's a bad idea.
> Personally, I think they should've published his paper - they can continue actually using the work internally to improve their TPU designs whilst he publicly tries to persuade their competitors it's a bad idea.
This analysis assumes he is wrong. I’ve got no idea if he is right or wrong.
But if he is right, Google would probably want to suppress stories like this for PR purposes. “Google AI published in Nature is actually beaten by old commercial technology” is not the headline Google wants to put out.
Although again - I’m not saying that he is right! Just that Google has commercial reasons for what it decides to pursue/implement/publish.
If he's right they've got a problem because they're actively using the research in their chip designs. Personally, I think if this technique really didn't work they probably would've figured that out when taking it to production.
From a quick scan of the paper, it seems the issue is not that the RL technique doesn't work (the chips do function) but that the claim that RL is just a really bad and inefficient way to do it. Key paragraph:
"We find that RePlAce [non AI] produces 26% better wirelength than RL [reinforcement learning] whilst using 5 orders of magnitude less computation"
If that's true then you can see why he criticized the Nature paper. It's not a small difference in performance. The classical techniques are just crushing the AI in this paper.
Well they said they implemented it in their “pipeline”, which could mean anything from they are using it lots to almost not using it at all. How often have we deployed features in our own software that are barely touched in practice?
Maybe I’m being too cynical about press releases though!
That said, I've also seen plenty of competitive drama in FAANG research labs, so this story is not hard to believe. More senior engineers often will use their seniority to power-grab control of projects. It sounds like Google execs did the right thing in the end.
It also has a weird emphasis on their personal lives - particularly their gender:
> Satrajit Chatterjee, a more senior researcher at Google, used the cover of scientific debate to undermine the women personally, the employees claim. They spoke anonymously because they were not authorized to discuss company matters. Multiple complaints about Chatterjee’s behavior toward the women were made to Google’s personnel department, and he received a written warning, some employees said, but he continued to criticize the women’s results.
I half expect it to carry on: "The male researchman directed his critical male gaze at the delicate ladylike handwritten-in-flowery-pink-biro output of the female women researcheresses.."
I mean, I know I'm out of step with the zeitgeist in finding this irrelevant and patronising, but come on. Female researchers deserve to be treated not as fragile novelties but as regular professionals. Implying that a male researcher launched some kind of inappropriate sexed attack by criticising their work is thoroughly bizarre to me.
> Implying that a male researcher launched some kind of inappropriate sexed attack by criticising their work is thoroughly bizarre to me.
They’re not implying it they are outright claiming that was his motivation for not letting it drop before he got fired.
Most reasonable people would just stop when told “dude, you’re gonna get fired if you keep it up” but some people just can’t do that for whatever reason. Either way they decided back in the 70s women don’t have to deal with harassment as part of their job and whatever your opinion on how this is enforced it is pure and simple fact you will be fired over a corporation going to the mat to defend you against accusations related to this. Well, unless you’re a CEO of one of the Silicon Valley unicorns then you can do whatever you want.
I think there are two separate threads running through this. One is his academic criticism of their work through the academic publishing process. I hope no one would describe that as 'harassment', irrespective of how much his academic output focused on challenging their work. The other is his extremely poor and possibly slightly autistic Slack behaviour, fixating on his disapproval of their work and repeatedly bringing it up in even the most marginally related contexts. That part I would agree rises, or at least arguably rises, to the level of harassment.
I was privy to some discussion with people related to this line of research. In my layman understanding Chatterjee's contention was final placement of components in the accelerator chip was better done with some human aided intervention, after RL had generated a "mostly" complete layout.
Its not hard to imagine why he could be challenging the other paper - The RL program could possibly be generating placements based on optimization related to defined tasks (goals). The Nature paper shows a blurred out image of the same, where the components aren't symmetrical placed. However its understandable that interconnect lengths influence I/O which you want symmetrical from the similar components. That makes the placement more optimal for generalized tasks instead of specific ones.
Do we know if that is done? Maybe or maybe not. But it would be interesting to find out. Anyway this is something we'll better understand if Sat Chatterjee's preprint is released.
My high-level take-away is that Google seems like a cesspool of non-stop drama.
Does anyone there keep work business to technical work only without getting overwrought with emotion? Has Google's hiring practices resulted in a population where neuro-atypical folks are overrepresented, and rational discussion is impossible? In other words people who may be technically excellent but have zero emotional intelligence.
This is also why I think blending activism with work is a terrible idea. People forget how to act professionally.
There are 'difficult' people at every job, but Google seems to be on another level... way too many stories about people freaking out over trivial nonsense like message board posts.
It also seems to be a uniquely American phenomenon. I could not picture a company in Germany or Scandinavia staffed with such immaturity.
I worked at Google for 12 years. I will admit I had a hard time separating my personal emotions and thought-space from my technical work and discussions with coworkers. eventually, I left because I realized that I was truly suffering, mentally (even though many interactions I had with coworkers were excellent, and enjoyed talking to them both technically and personally).
My worst interactions at Google were with Google Research. So were my best- things changed signficantly for the worse as Google Research transitioned (from Alfred Spector to John Gianndrea to Jeff Dean). It's not an org I woudl want to be associated with in the future, for reputational reasons.
It continues to boggle my mind that this guy chose to get fired over this paper, which is the dumbest paper I have read in a long time.
As an example, his paper reports *median* congestion! I literally laughed out loud when I saw that. I don't think a single other chip design paper in the world uses that metric, and for good reason.
For those of you who don't know chip design, congestion is a measure of how many wires will need to go in the same place when you do routing for the chip. When wires overlap, they have to go above / below each other, and your chip only has so many layers. If congestion is too high, it can simply be impossible to make the chip.
Congestion is therefore all about the extreme values. Why in the world would you report median congestion, then? Median ignores extreme values, which are the only ones you care about!
I even checked the paper he cites as being prior work that uses median congestion, to see who else did something so stupid, and... it doesn't! It uses average, which makes way more sense. I mean, you really care about the max, but at least average is sensitive to extremes.
I could go on, but the TLDR is, his paper is hot garbage, and I cannot say I am surprised Google didn't want its name associated with it.
This work is actually legit, and it's really just insane the amount of chaos this one jerk was able to cause. This is not some snowflakes who can't handle criticism. This guy actively destroyed business value left and right.
What's really surprising is how long it took the company to fire this guy.
This sounds like a preemptive hit piece on a researcher who is now free to publish an article that is critical of Google researchers work and perhaps even shows hints of outright fraud.
Nothing is brought forth except to say that this person had persistent claims of issues in the work with no proof provided to disprove these issues, and Google was preventing scientific publishing of the critical work to the point of threatening to fire the person and they now actually did.
The genders of the people involved are irrelevant but are pushed as some argument as to why the criticism is invalid. I’ll look forwards to seem the critique being published so the claims can be weighed against each other honestly.
See my above comment - the choice of metrics in table 3.2 doesn't really make any sense.
Also, they revived some ancient 1998 IBM chip, and report on that, for no clear reason. They call it a 2004 benchmark, but what actually happened is that in 2004 someone made some synthetic variants and put them into a dataset. This is not some widely used benchmark in the field. Given their highly questionable choice of metrics, I would not be surprised if there was some serious cherry-picking on the benchmark as well.
Could I believe an older man might resent the success of two younger women, and seek to undermine their work with baseless criticism? Yes.
Could I believe that two AI researchers overhype or even fabricate their results and that the company which had proudly published those results would not want to hear valid criticism of it? Also yes.
I don't know which side to believe here. And, to be honest, there's not a whole lot in the Wired article to help me. But, I suppose the fact that Google is struggling to deal with such contentious personnel issues is a story in itself, albeit not a very surprising one.
1) Google terminated this guy "with cause" and then said so publicly. Google definitely does not do that lightly.
2) The TPU team used this method to help make TPUs. This is a flagship product, with billions of dollars on the line. There's just no way Google would sacrifice the quality of TPU to help a couple AI researchers look good.
Also, if you're actually interested in writing a paper on "What are the best techniques for place and route" then write a paper "Best Modern Place and Route algorithms" not "Here's why this particular paper my colleagues wrote is wrong".
Personally, I think they should've published his paper - they can continue actually using the work internally to improve their TPU designs whilst he publicly tries to persuade their competitors it's a bad idea.