Interesting! I have also been tickled by incremental map reduce and its possibilities.
> I believe that many of my problem cases above could, in theory, be expressed as a graph of more complex map-reduce operations.
Indeed, and if you can do that, then you are solving these problems in a completely declarative way, with all the advantages that that brings.
Personally I have been wondering whether it is possible to apply the concepts of map-reduce to more complex data structures (like trees) because I think that would open up a lot of possibilities.
I'm glad that other folks think this is important too.
Stay tuned on twitter/medium. That python library also supports a viewabledict (for key-value mappings). I've got an experimental compiler which starts with an AST (stored as a tree of these maps), and then chains type checking and code generation off that to make incrementally updating binaries. Just supporting a toy language to start, but it will make a nice demo I think!
That's great. Why aren't all compilers like that? How comes in visual studio for instance you can have a build that takes a minute, but linting and verification tools that react immediately to code changes. I think we're being short changed!
> I believe that many of my problem cases above could, in theory, be expressed as a graph of more complex map-reduce operations.
Indeed, and if you can do that, then you are solving these problems in a completely declarative way, with all the advantages that that brings.
Personally I have been wondering whether it is possible to apply the concepts of map-reduce to more complex data structures (like trees) because I think that would open up a lot of possibilities.