It’s a tough problem! I think 90% of it is solved with GraphQL/ protobuf like rules (“only ever introduce new fields”)
There are some edge cases where you might loose easy access to some data if two clients migrate concurrently, but we’re hoping to provide patterns to mitigate these
Edit: Right now it all depends on you to implement migrations “the right way” but we hope to provide guardrails soon
I've been thinking about this as well and wondering if one possible approach to avoid the eventual messy database state—where fields are "append-only" and never deleted—might be to include the up/down migration logic as part of the payload.
This approach may require a central authority (with no access to user data) responsible solely for providing the schema and migration patterns as code transformations.
Since running arbitrary code from the payload introduces potential security risks, the migration code could be cryptographically signed, ensuring that only valid, trusted transformation code is executed.
A possible additional security layer would be to have the transformation code execute in a sandbox which can only output JSON data. (keeping a possible full before-migration version as backup in case something went wrong would always be a good idea)
Another option would be to use a transformation library for migrations, but in this case, the approach would only describe (as JSON) the functions and parameters needed to transition the schema from one version to another.
> I think 90% of it is solved with GraphQL/ protobuf like rules (“only ever introduce new fields”)
Agreed, that’s the only
sensible thing to do. Not sure it’s 90% though.
> but we’re hoping to provide patterns to mitigate these
Hope is not confidence inspiring for the most difficult problem at the heart of the system. That doesn’t mean it has to be impossible, but it needs to be taken seriously and not an afterthought.
Another thing you have to think about is what happens when data of a new schema is sent to a client on an older schema. Does the “merging” work with unknown fields? Does it ignore and drop them? Or do you enforce clients are up to date in some way so that you don’t have new-data-old-software?
Partykit is mostly an abstraction on top of Cloudflare Durable Objects and offers simplified Websockets for room-like multiplayer.
Jazz is a framework for building apps around locally mutable collaborative values which are granularly synced (you don't have to worry about data transport or coordinating sync between collaborators, you just set permissions and edit data)
Right now, we're not doing complex validation very well, we basically rely on all clients having the same schema or backwards-compatible schemas and prevent errors at the type level.
But because the schemas are runtime constructs as well, we might be able to do more complex stuff in the future. Basically any rule you can express as a function of (previous history, new transaction) -> is new transaction valid? can be a rule in theory, and each client will verify it in an eventually consistent way.
For the time being, the Group abstraction with it's reader, writer and admin roles, plus composition of CoValues in different groups is sufficient for most permission-like rules, which is what most apps need most of the time.
Hey, thanks for the response. This does make sense. The group roles do seem relatively powerful.
I guess what I'm probably getting at is how does it hold from a threat perspective? What is to stop an actor distributing something extra large or unscrupulous between nodes. TS is notoriously hard to use for validation within types (e.g. string of max length, number between), but clearly covalues go beyond this.
Anyway, I'm writing this having not yet tried the framework. So I'll do that first and then do some further reading, and then jump onto the discord if I have more questions. Cheers.
Yes, plus it’s actually local-first (works offline) and you don’t have to trust the syncing & persistence infrastructure because only encrypted edits are synced!
So it if I already have an app using instantdb, how hard would it be to switch to jazz? Does it work in a very similar way? Or would it likely require a lot of code modification?
We want to support as many environments as possible and are doing them in order of easiest & most requested - you are the first one to ask about Dart/Flutter
React Native is released (experimental)
After that will be really good support for Next.JS (Jazz for both SSR and client)
A bit later will probably be a Rust port, which then also makes bindings to Swift/Kotlin/Go/... easy
- Jazz is built with granular syncing in mind. CoValues are meant to be small (think one level of a JSON tree) and then reference each other. You only resolve references and sync as far down as you need / in lists you can do pagination.
- CoValues can reference CoValues that belong to a different group and thus have different permissions. In your example the title would need to live in its own CoValue and group compared to the content, which is a tiny bit more setup, but totally viable
- no resources for data modelling yet, but the guide + example apps should give you a very good idea. If you have any questions, please ask on Discord! https://discord.gg/sesjU2W5
- deletion is typically soft-deletion with tombstones. Total erasure of entire CoValues is coming soon but should only be needed in compliance situations (such as GDPR)
