Hacker News new | past | comments | ask | show | jobs | submit login

Jokes aside, has anyone tried fine tuning a model on forum conversations?

By which I mean, for chatbots, the data for fine-tuning basically presents it with examples of dialogs. But if you take any given comment on HN and walk its parents, you basically end up with something like that, except there's more than two people involved (usually). So what if you fed that to LLaMA, and then used the same format for the actual chat?




In that vein, how about stackoverflow? That should give at least straightforward ask-and-answer format, and there's plenty on material to work with.


LlaMa was trained on 78 GB of StackExchange (I assume StackOverflow was included in that).


But was it parsed and reformatted specifically in the "chat format" (i.e. the same as inputs later fed to the model when used as a chatbot)? It can make a surprisingly big difference.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: