Been using it for a few days and really like it.

One small thing that bothers me a bit is that the "Distill" button takes up a whole bar at the bottom: https://i.imgur.com/VSbVx4k.png

Very happy to hear! Thank you for your feedback, I will experiment with a different placement for the button, as well as a smaller size.

Any other suggestions?

Great work, looks very simple and neat. I'm gonna use it today to see how does it suit my eyes. Do you have any plans for dark mode?

Thank you, that's the look that I was going for! Dark mode is for sure coming, no ETA yet. Do you use any browser extensions to get automatic dark mode, that work well?

## How did I build this?

The backend is written in Rust, and the frontend is written in TypeScript with SvelteKit. The API is based on gPRC, using Buf Connect on the TS side. On the storage side, it’s using Postgres + Qdrant (self-hosted on dedicated servers) + R2. The reason I am self-hosting on dedicated servers are mostly the high CPU and RAM requirements for the main project which would otherwise make it cost prohibitive.

For the LLM parts (which are central to the main project), I built my own language-agnostic framework and platform to facilitate building LLM powered services. This is based on some lessons learned while working on Google’s Bard and other LLM projects that preceded Bard (those were ultimatelly killed — long story :)).

The framework and platform are currently private, but I might open it up in the future. To give you a glimpse:

The main premise is that building apps that leverage LLMs should not be that different from building regular apps. The main difference is the need for configurability and observability.

Each component (e.g. LLM call, Vector DB lookup, etc.) is just a regular RPC method, with well defined schema for its input & output.

The platform allows you to configure each method, which just means specifying values for some subset of its inputs.

The platform also tracks and stores the inputs and outputs of all the RPC calls and sub-calls. This is invaluable during development to understand and debug the methods, and in production to collect data for evaluation & training.

Thanks to the well defined input & output schemas, all of this can be done universally, rather than creating a bespoke solution for each method. Here are some examples:

- The “LLM” building block: https://www.loom.com/share/bc6fa6b27298420c82fcacca5f84d096

- Higher level JSON → Format → LLM building block: https://www.loom.com/share/d09d38d2c316468fa9f38f5b386fc114

- Example a complex trace from a summarization method: https://screenbud.com/shot/61dc59a6-5c73-4610-8168-753b79062...

Example code for a simple retrieval augmented chat bot in TypeScript and Rust https://gist.github.com/Palmik/42940e3887d6446244f8b74ce0243... — there’s still a lot of room for polish and boilerplate removal, but it gets the job done already.

Happy to answer any questions.

Nice work!

Awesome! Wouldn't it fit in the 1GB free tier plan?

https://cloud.qdrant.io

We're getting complaints about your promoting your project too much in HN threads. Looking at your account history, I can see why users are complaining - it looks like you're breaking this guideline:

"Please don't use HN primarily for promotion. It's ok to post your own stuff part of the time, but the primary use of the site should be for curiosity." - https://news.ycombinator.com/newsguidelines.html

Occasionally linking to your work is fine, of course, where it's relevant, but it's important not to overdo it and to use HN for general curiosity, not promotion.

Hey Andre, huge fan of qdrant, and super impressed by your active support on Discord!

I use qdrant to store all sorts of datasets (wikipedia, podcasts, etc.), but even just hackernews is millions of documents and probably would not perform well without any sort of index.

I've loaded all HN comments with embeddings into ClickHouse, and it performs well without vector indices (and better - with them): https://www.youtube.com/watch?v=hGRNcftpqAk

Slides: https://presentations.clickhouse.com/meetup74/ai/