I'm making a VSCode extension that can record & replay interactions with the IDE: all scrolls, selections, and modifications synced to guiding video, audio, and visual aid tracks.
The result is a much more interactive way to present code than screencasts or blogs. Because at any point we can pause a session and freely explore and experiment with the codebase.
I put together a demo recently [1] and written much more about it here [2].
Really cool! I'm building something with a little bit of overlap.
A personal data collection/archival platform with the goal of focusing on timestamping everything possible.
Basically, I realized that when I have multiple pieces of data (e.g. a note, some commands, a screenshot), almost always this data is naturally connected by time and that's also how I want to query it. So: it's nice to be able to e.g. create wiki links in note taking tools like obsidian but it would be nicer if one didn't even have to do the linking manually (most of the time).
I am building on postgres with a time series extension. On the client side I want there to be able to be multiple "collectors" that can collect data which is then sent to the server. So far I only have a keylogger collector but I have many more ideas such as a screenshot collector à la copilot etc.
I think the data you are recording would also be a candidate for a collector for this system.
I'm gearing all my personal tooling stuff towards collecting a ton of data points—screenshots, logs, notes, todos—that I can then pare down over time to make them useful. I've got a similar setup with a central monolith that can handle anything I throw at it from stdout.
Lately, I've been focusing mainly on the collectors because I figure the state-of-the-art in any middleware I write is going to change quickly. I've been playing around with knowledge graphs for visualization and might consider using them as a data store for this kind of thing. I'm still thinking about it. My watchword has been minimalism when it comes to design—no added steps or latency to existing workflows.
This is very interesting and I agree that creating a knowledge base in obsidian (or org mode) is veey nice but requires a lot of effort. So anything that makes it easier and more automatic is a win.
Having said that, in my experiments with CodeMic, I started to realize more and more how important the editing process is in order to create polished consumable content. My goal is to get as close as possible to 3blue1brown level of clarity but for programming content. So far this has required to focus more on editing part of CodeMic: cut, merge, insert, speed up/down etc.
So if you log everything, then I think the problem becomes how to make sense of all that content from all the noise. I'd love to hear your ideas on that. Feel free to email me :)
The result is a much more interactive way to present code than screencasts or blogs. Because at any point we can pause a session and freely explore and experiment with the codebase.
I put together a demo recently [1] and written much more about it here [2].
[1] https://youtu.be/Qp2GdLO5eSc
[2] https://codemic.io