Hacker News new | past | comments | ask | show | jobs | submit login

VRAM per GPU isn't such an interesting metric. If it was, everyone would be fine tuning on A100 80gb :)

What matters is steps per $ and to some degree also speed (I'm happy to pay premium sometimes to get the fine tuning results faster).






True, but a TPU v5p is supposedly much closer to an H100 than an A100 (the A100 and TPU v4 were fairly similar) — and you need the RAM as a baseline just to fit the model. I haven't seen super thorough benchmarking done between the two but the Google claims similar numbers. So, $/RAM/hr is all I can really look at without benchmarking sadly.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: