The results i did get from deepseek-r1 on their webpage did not match the results i did get from o1-pro.
I did ask it go to a github repo, find the part where the logic of the “export” button is and explain why it doesn’t work (the whole logic is actually missing, won’t work at all).
O1 pro did get it right in the first try while deepseek r1 was heavily hallucinating.
Maybe i am using the wrong model?
No, you’re not. They explicitly mention in the R1 paper (in the last paragraph before the bibliography) that R1 isn’t a “huge” improvement over DeepSeek-V3 in coding - where “huge” is an academic weasel word.
It’s just a lot of hype. In my coding tests it significantly underperforms o1 (haven’t tried o1-pro), often getting stuck in a reasoning loop because I underspecified something (that I don’t have to with o1).
Same anecdotal experience. Its definitely an improvement and they have made operational improvements at runtime but I am still concerned they are have over fit for the tests.
