Interesting, this is the first time I'm reading about it [1]. Well, if anything it looks like a web app would have to be rebuilt from the ground up to fit that model. I haven't yet read much about it, but here's a few questions that pop up immediately:
1) If you have a container per data object, doesn't that mean you also have to start a process every time a user opens a document? So forget about doing any computations in the setup. Even just things like using regex patterns in python (that need to be compiled once) or using anything with a VM (that needs to be started) you'd have to give up. Usecase seems to be extremely limited, but maybe I'm not getting something right here.
2) How do you handle indexes, views and collections over a large set of data objects?
It turns out that converting a web app to Sandstorm is mostly deleting code. You delete your user management, your collection management, your access control, etc. What you have left is the essence of your app -- the UX for manipulating your core data model, of which you now only need to worry about one instance.
> doesn't that mean you also have to start a process every time a user opens a document?
Most apps we've encountered only take a couple seconds to start. But we're working on a trick where we snapshot the process after startup and start each grain from the snapshot, thus essentially optimizing away any startup-time slowness.
> How do you handle indexes, views and collections over a large set of data objects?
Sandstorm is (currently) designed for productivity apps, not for "big data" processing. The data within a single grain is usually small. That said, you can run whatever database you want inside the grain.
> Most apps we've encountered only take a couple seconds to start. But we're working on a trick where we snapshot the process after startup and start each grain from the snapshot, thus essentially optimizing away any startup-time slowness.
This is very interesting. I've been looking for something like this since 2007 for optimizing the startup time of some apps. However I couldn't find any suitable technologies for this purpose; VM memory snapshotting is heavyweight and is slower than starting an app from scratch, OS-level tools like cryopid can't even be called alpha-level. What kind of snapshot technology do you intend to use, and how confident are you that it will work well?
Thank you for your response. Sorry, I'm still not getting it. Not even talking about big data, what I mean is a very simple database application, say with 10k documents in a document based storage model. Now let's say I want to show a list view and a sum over one of the fields in those documents. You'd probably agree that this is a typical usecase for a productivity app. Now how would I do this in the sandstorm model?
> You'd probably agree that this is a typical usecase for a productivity app.
Actually I can't think of any app that matches the abstract problem you describe. Can you give a specific example? Usually the answer to these problems becomes much clearer in the context of a specific app.
Pretty much anything you'd like to do with CouchDB or MongoDB. Replace "document based storage" with table based and you have pretty much any *SQL application.
So I guess my question is: How do you expect people to use your fine grained model with databases? If the answer is "not at all" then I find the scope too limiting. If the answer is "1 grain == 1 db" then I find the claim that you are solving difficult permission problems to be false.
Note: I don't want to be too critical here, I'm just trying to pick holes in your claims of scope so I can categorise the power of sandstorm and how far it could be useful for things I'd like to build.
> Pretty much anything you'd like to do with CouchDB or MongoDB
Wekan is a Trello clone that uses MongoDB for storage. On Sandstorm, each board lives in a grain, so there ends up being one MongoDB per board. This works fine. The only thing stand-alone Wekan ever did that queried multiple boards at once is display the user's list of all boards. On Sandstorm, displaying the user's grain list is Sandstorm's job, not Wekan's -- and indeed, usually the user is more interested in seeing the list of all their grains rather than just the Wekan boards, so delegating this to Sandstorm is a UX win.
If that is not the kind of example you have in mind, then you really need to give a specific example.
1) If you have a container per data object, doesn't that mean you also have to start a process every time a user opens a document? So forget about doing any computations in the setup. Even just things like using regex patterns in python (that need to be compiled once) or using anything with a VM (that needs to be started) you'd have to give up. Usecase seems to be extremely limited, but maybe I'm not getting something right here.
2) How do you handle indexes, views and collections over a large set of data objects?
[1]https://sandstorm.io/how-it-works