Youtube talk from Kailash Nadh, CTO of Zerodha: largest retail stock broker in India) Talk slides here: https://nadh.in/files/indiafoss-2024-dungbeetle.pdf

"The recent change also means you can run multiple LLaMA ./main processes at the same time, and they'll all share the same memory resources." So this could have a main and multiple sub-worker llm processes possibly collaborating while sharing same memory footprint?

Yes, if the model is mmap'ed read-only (as I'm sure it is).

There are other bottlenecks than CPU cores though, it might not be very useful to run multiple in parallel..

Thanks!

This sounds like a great advice! Will pickup a domain and get started. Would probably look around for hardware/robotics and machine learning domain specific projects to get started and learn :)

This sounds great!

What else would you recommend? Rust?

But if I have to work with hardware, I believe even today (and perhaps for many years to come) C/C++ is the way to go right?

Over last 10 years have used Freemind & (last few years) Xmind to kind of make superfast flow charts in the tools dressed as mindmap tools. So i can see the appeal of knoted here for quick jotting down of thought flows.

But have also enjoyed making nice diagrams in mermaidjs especially the sequence diagrams. Prefer markdown to digram for code maintainability. Was recently seeing a youtube talk from C4 author on composable diagrams and agree to a lot of what was said in the talk. Src: https://www.youtube.com/watch?v=Za1-v4Zkq5E

Keyboard to diagram is much faster than mouse based diagram tools and have found them better for long term use cases. Quick diagraming in excalidraw or figjam doesn't hurt though when brainstorming especially with not very technical folks as they can easily contribute.

Mermaid? I like mermaid, but it is not fun, it is not fast and it is not comfortable, especially when you have bigger diagrams or must edit them.

Have used sequence diagrams for interactions & state diagram for state machines from mermaid for this. https://mermaid-js.github.io/mermaid/#/ System architecture is probably best in PlantUML as already pointed out in other comments. https://real-world-plantuml.com/ is helpful. But mostly it's a lot of manual effort to keep it updated, but the markdown (in git repo) makes it easier I guess...

=> One thing that worked at times was to have new joinees in team update the docs and the diagram based on their understanding + having this system architecture as part of onboarding sort of ensures it is updated on most occasions. :)

Looks nice :) Have been looking for something as simple as Heroku for a while now. This seems closest as of all the ones I have looked at!

It was delightful to watch him create a Wordle clone with a simple html file coupled with one js file in plain sublime text editor and chrome debugger hacking :D

No complex modules installation or initialisation setup. After having seen the JS complexity grow to from simple jquery stuff to download a whole lot of stuff to just get started creating a simple webpage, I personally found this simplicity absolutely delightful and inspiring. Ended up creating a simple world clone in a couple of hours and quite enjoyed the whole simplicity of it all :D

Thanks!