For something that's open-source, production ready thanks to the lovely folks at Mozilla, and ready to host on your own infrastructure check out Kinto[1]. Replicache and Kinto have overlapping feature sets, but my guess is that Replicache is probably a bit more optimized (especially if the delta updates are truly as small as possible). That being said, of all the problems I've run into implementing sync for my applications with Kinto, the size of update requests hasn't been one of them. Also while Kinto can support live subscriptions, the only plugin implementing them currently relies on a third-party.
The biggest difference I see between them is that Replicache sits in front of your server, whereas Kinto _is_ your server.
Disclosure: I'm a contributor to Kinto, primarily working on transitioning the JavaScript libraries to TypeScript.
I'm sorry to say that I didn't know about Kinto. Thanks for the link!
Kinto seems, is in some ways, the opposite design approach to Replicache. Replicache answers the question: I have this existing web service with a whole complex backend stack, how do I make it work offline-first?
Kinto answers the question: I have this website, and I don't want a backend stack. Where can I store data and have it work offline-first?
I think the problem Kinto is solving is also important, thanks for sharing it.
I'm starting a new company attempting to solve the offline-first / mobile-sync problem once and for all.
Replicache makes your mobile, web, or desktop application blazingly responsive, by buffering all reads and writes to a local cache (aka "offline-first").
This is something I've been thinking about on and off for over fifteen years, on major projects at Google as well as my previous vc-backed startup. I previously thought there was no satisfying general solution, but happy to say I think I was wrong: Replicache is pretty dang easy, and it works with any existing stack.
We're looking for groups to partner with would like early access. If that sounds interesting, please reach out!
Sorry, no technical comments but a UI one: Please change the font/style. Depending on monitor color settings, the paragraph text in Retina Thin is really hard to read, but even on my other monitor it’s still a problem with the letter "s" [0]. I think it’s a problem with the thinness and white on dark background. It was hurting my eyes and permanently distracting me while trying to read the copy.
Thank you for this feedback, I'll ask our designer for his advice. Perhaps we can detect high density monitors and only do Retina in that case or something.
- Doesn't guarantee convergence. It's up to the developer to provide a feed of deltas to the client and ensure they have the correct effect when applied to the client. This is the one of the hardest parts!
- Provides no help with conflict resolution. Push replication just pushes the new version of each document. The server must figure out how the document differs from the stored version and what that must mean in terms of logical operations.
- Implemented in JavaScript. Difficult to use in native apps.
--
I'm not up to date with Gun (it has gone through many iterations), but I think it is currently trying to be a masterless system which means the same concerns as with CRDTs apply.
You can do conflict-resolution by subscribing to the changeFeed and when multiple versions with the same revision-height exist, you create a new version with a 'merged' document. Depending on your dataset this can be tricky and painful.
RxDB can only be used with javascript and will likely ever be.
This looks really interesting. I have been digging deep into sync for the past few months and am very interested in how this works. I am currently trying to understand automerge and delta patching trying to implement from scratch. I have looked at Gun and couch db + pouch db but both did not feel like the right answer.
Be aware that CRDTs like automerge are solving a different (and harder) problem than Replicache. They are trying to implement convergence in an asynchronous system where there is no central authority. See the excellent article https://www.inkandswitch.com/local-first.html for more on this type of application.
Most classic web services don't have this requirement because they do in fact have a central authority -- the service itself.
Moreover, for web services, it is crucial that the central authority actually be authoritative. You don't want client and server state kind of gets smooshed together arbitrarily, but for the client's view of the state to be a mere suggestion - one which the server always overrides.
So my view is that CRDTs are not really an appropriate basis for building this kind of feature in a web service.
However I think the tech is awesome (Replicache actually started out as a true CRDT and moved to its current design after extensive iteration with customers).
- Using couch as your backend db ends up being a nonstarter for most applications. A distributed multitenant database is a big big thing and a hugely important technical decision. Most orgs are not going to go with couch just to get sync. See https://medium.com/wandering-cto/mobile-syncing-with-couchba... for an example of this.
- The couchdb replication protocol offers no help with conflict resolution. It just tells you there was a conflict and gives you two conflicting documents. This isn't practical for most applications.
PouchDB author here, your project looks great good job.
I certainly agree that switching backends to CouchDB has made it hard for people to adopt Pouch/Couch. I have often considered how I could make Pouch work with arbitrary data sources, but as you well know its a tricky problem.
However I dont understand from your website or comment here about how conflict resolution is easier? Given the situation of having 2 clients recieving an initial data of {key: value}, the clients go offline, 1 client writes {key: foo} and the other writes {key: bar}, the clients then both reconnect. What is the new state?
Thank you for the comment. I think there is a technical difference and an ergonomic difference:
1. The technical difference is that when you do conflict resolution with Replicache you have more information, specifically the intent of the mutations. Consider something very simple like a positive-only counter. The parent is `1` and the forks are `2` and `0`. Is the correct resolution `2`? Is it `0`? Or is it `1`? There's no way to know because we don't know what the intent of those changes was. Was fork 1 incrementing? Was it multiplying? Was fork 2 decrementing? By how much? Now multiply this simple example by real applications with many developers, many features, and many client versions in the wild. Having the intent of each change travel with the change is crucial.
2. The ergonomic difference is that conflict resolution in Replicache isn't something separate that is done after-the-fact. Replicache applies mutations to the server by calling normal HTTP APIs, just with potentially old arguments. This forces developers to consider conflict resolution at the point they are writing APIs, and keeps conflict resolution code colocated with the corresponding services.
> This forces developers to consider conflict resolution at the point they are writing APIs
Would an example of what you have in mind be writing an atomic "/increment" endpoint instead of a "/set?value=possiblyOldValue+1" endpoint? (using GET notation instead of POST just to make illustration easier)
From what I can tell from the website, replicache retries HTTP requests periodically until they succeed. With an non-idempotent request like `/increment` how does replicache know when to stop retrying?
if request succeed, but the response fails, could replicache increment twice?
The APIs Replicache calls at customer service must be idempotent. Replicache passes a version vector to the customer that customer uses to ensure this, and to enforce causal consistency.
Customers must make several relatively small changes to their backends to support Replicache, including this one.
After scanning that doc it looked to me like a developer could create a way to alert a user of conflicts and provide options to resolve them. On the client side it would seem like PouchDB might be used to create a friendly way to detect and help implement those decisions.
I just started looking into conflict resolution this month, and more out of curiosity than need. PouchDB's "Live Sync" for feature CouchDB works amazinly well and that alone reduces the opportunities for conflicts substantially by insuring when you go offline you always have the latest data.
As a rule of thumb I'd say that when using an app offline the user should avoid editing living documents. PouchDB makes it easy enough to create a new document with that data for others to review and merge into the living document when you come back online.
But reality is conflicts will happen and we do need tools to deal with them, so it's great to see the Replicache team working on that.
I'm curious how well Replicache composes together with web state management libraries such as Redux and Apollo GraphQL. My closest experience with something like Replicache's offline view is optimistic responses in GraphQL[1]. Maybe some more web-specific examples can be added to the site?
Apollo's optimistic responses are similar Replicache's local writes. The difference is that Replicache's offline writes automatically refresh every affected subscription and ensures the entire UI is consistent with the latest changes. You don't need to manually modify affected queries: https://www.apollographql.com/docs/react/performance/optimis....
How Replicache composes with Apollo needs more thought. Off the top of my head, I think that Replicache replaces the need for Apollo's unified cache, but not with many of its other features. For example, it's easy to imagine Replicache being one of several data sources backing Apollo. In fact we have a customer who has done something like that already, and it's quite nice because the consuming code doesn't even need to know whether a GraphQL query is coming from offline data or the server. That can even change over the life of the product as more features in an application become offline-enabled.
The biggest difference I see between them is that Replicache sits in front of your server, whereas Kinto _is_ your server.
Disclosure: I'm a contributor to Kinto, primarily working on transitioning the JavaScript libraries to TypeScript.
[1] https://docs.kinto-storage.org/en/latest/