Hi Everyone, I work at Together. Today we released OpenChatKit: an open-source base to create chatbots for various applications. More than a model release, this is the beginning of an open source project. We are releasing a set of tools and processes for ongoing improvement with community contributions.
1. An instruction-tuned large language model, fine-tuned for chat from EleutherAI’s GPT-NeoX-20B with over 43 million instructions on 100% carbon negative compute available under Apache-2.0 license on Hugging Face.
2. A set of customization recipes to fine-tune the model to achieve high accuracy on your tasks documented and available as open-source under the Apache-2.0 license on Github, along with code to recreate our model results.
3. An extensible retrieval system enabling you to augment bot responses with information from a document repository, API, or other live-updating information source at inference time, with open-source examples for using Wikipedia or a web search API.
4. A moderation model, fine-tuned from GPT-JT-6B, designed to filter which questions the bot responds to, also available under the Apache-2.0 license on Hugging Face.
We collaborated with the tremendous communities at @laion_ai and Ontocord to create the training dataset used for these models, also released as open-source. Read the full details on LAION's blog post!
How did this not get traction? I came upon the model from Twitter by chance. This is the first seemingly valid and functional open-source ChatGPT alternative. Can’t wait for the optimizations which will allow it (or similar) to run on consumer-grade GPUs.
You can try it now on Hugging Face! https://huggingface.co/spaces/togethercomputer/OpenChatKit
OpenChatKit includes 4 key components:
1. An instruction-tuned large language model, fine-tuned for chat from EleutherAI’s GPT-NeoX-20B with over 43 million instructions on 100% carbon negative compute available under Apache-2.0 license on Hugging Face.
2. A set of customization recipes to fine-tune the model to achieve high accuracy on your tasks documented and available as open-source under the Apache-2.0 license on Github, along with code to recreate our model results.
3. An extensible retrieval system enabling you to augment bot responses with information from a document repository, API, or other live-updating information source at inference time, with open-source examples for using Wikipedia or a web search API.
4. A moderation model, fine-tuned from GPT-JT-6B, designed to filter which questions the bot responds to, also available under the Apache-2.0 license on Hugging Face.
We collaborated with the tremendous communities at @laion_ai and Ontocord to create the training dataset used for these models, also released as open-source. Read the full details on LAION's blog post!
Can't wait to hear your feedback!
Thanks, -Together