A couple of hundred GPU's is well within the reach of many even moderately well heeled research institutes. It'd seem that about 3 weeks of compute time with 128 TPU v3's would be about $170,311.68.
But of course that cost would only be for the final model. Anyway, I think I am just living in a different world... :-) We could never compete with that
Yah, big grant money. Now the grad students programming the open source clones will only make approximately $0.56, or 4.2 Ramen packs, for their effort. ;)
Also with keeping in mind that once a good open source model is available, researchers with less resources can still use it to fine tune and get new results for far cheaper than training a new model from scratch.
