> Our actual solution is to store the state in our server-side database. This feels wrong at first glance: we're storing a client-side React component's state as an opaque JSON blob in our server-side database, and we're doing that knowing that some of those JSON blobs will go out of date and be unusable in the future. But io-ts and the type system keep us honest here! And this is a convenience feature, so we can always change it later, even if that means throwing away all of the saved states. (When a user finishes a lesson, that record is stored in a separate part of the database.)
That seems very... bold. That said, I could see this being useful for more ephemeral situations where it's okay to not be entirely in sync with the server. Like the user signup example: multi-step forms, or some create-only or override-always sort of arrangement, or maybe some filters for some sorting/slicing data tables, etc.
It's then useful for not being jittery on load + not storing half-baked data just to have semi-persistence of incomplete forms. Quickly rendering existing state is always a win, without resetting it then loading the updated position.
Otherwise I could see this introducing a lot of race conditions and additional work being done post-load to clean up the data.
Since there's a relatively short line here where it just makes sense to rebuild the state, it's probably preferable to utilize this for smaller isolated parts of the site, like a single form's data, instead of the wider state. But still an interesting approach none-the-less, made easier with the reducer pattern...
We only use it for resuming partially-finished lessons, so we're using it in basically the way that you describe. Overall course progress is tracked in a totally different way.
Exactly, the ideal solution is the one they dismiss first.
> Solution 1: Store the current step index
Every issue they mention can be solved by assigning each step a UUID and doing proper database migrations when deleting a step so people get reassigned to a neighboring step.
Every step has had a unique ID since day one. Database migrations would be wildly more complex than our actual solution and involve a huge increase in maintenance and (depending on what you're imagining) increase database server load by multiple orders of magnitude.
Today, we can edit lessons in completely arbitrary ways, including adding and removing steps, and the system adapts dynamically. With your proposed solution, lesson content changes that take seconds today would require writing and running an entire database migration.
Your scheme doesn't address metadata at all, but metadata is one of the biggest complications discussed in the post. If you're imagining a database schema that knows about the structure of the metadata, broken into separate tables, then your scheme would amplify a single write per step (as implemented today) into hundreds of separate writes per step (with your change). Those writes would happen for every step advancement in a lesson, so a user completing a single lesson would result in thousands or possibly tens of thousands of database writes instead of the roughly 25 writes that happen today.
But regardless of whether you're imagining the metadata in an opaque blob or separate tables, it doesn't actually solve the metadata migration problem! Imagine this situation: You have tens of thousands of half-finished lesson sessions sitting in the database. Now you add a new piece of lesson metadata to the system. What do you do with those old sessions when you run your database migration? What if the relevant metadata can't be reconstructed for those old sessions (which will be the normal case)? Do you make up a fake, incorrect value for the metadata? Do you make that metadata field nullable, forever, which over time will result in all metadata being nullable, defeating the type system guarantees? Our scheme solves all of this with a relatively simple code change, and it works automatically in all situations, and it has minimal runtime overhead or database load, and it requires no ongoing maintenance whatsoever.
The metadata is mostly used for analytics. I didn't even get into this in the post, but "throw the incomplete lessons away when the metadata structure changes" is equivalent to saying "don't fabricate analytics data that will generate incorrect analytics results, which will cause us to optimize the business incorrectly."
I think it’s a common issue: i.e. if you decide to be bold, then you also may need to be bold in other situations or otherwise you are kind of missing the point.
But often people will decide they can only be bold once (forgetting that the risk of bold tends to overlap with the other bold).
The idea is great. But it also requires one to accept other ideas to fully work.
>1. Why marry the state stored in the database with any particular implementation of the front-end?
They seem to have only one front-end implementation to worry about for now, and the client state is most likely pretty generic (they don't really make this clear, but I can't think of anything react-specific you'd want to store in a reducer state).
This is right. You'd have to read a ton of our source code to see this first hand, but the reducer state has nothing about React in it. It's a lot of stuff like "what answers did the user give?"; "was the answer correct or not?"; "what were the start and end timestamps for each attempt?"; etc. If we maintained multiple clients, we could reuse that structure because it's all about the application, not about React.
We did the same at a previous company. This was Redux, so maybe a little different, but it went very smoothly.
We stored a subset of the redux store locally, and a further subset of that was sent to the server. This data is quite prone to changing, so a JSON field in the database is fine. You're not going to do any advanced querying / migrations on it.
Restoring was pretty easy:
- Try to grab from local storage
- Try to read from remote
- Merge those (let local overwrite remote)
- Pass the merged object through a migration function
- Pass the migrated data to `createStore`
The biggest pain point was dealing with TypeScript and migrations. The app only knows if the most recent store interface, so migration functions looked a lot like "old" javascript (`if (foo && foo.bar)`, `typeof foo === 'string'`, that sort of stuff).
Easy persistence / restoring is one of the many things I like about Redux.
You can express a type as one of many interfaces, or use discriminated unions (https://www.typescriptlang.org/docs/handbook/advanced-types....) by using some unique version key to identify each interface. Then you'll have all the goodness of types back :)
That seems very... bold. That said, I could see this being useful for more ephemeral situations where it's okay to not be entirely in sync with the server. Like the user signup example: multi-step forms, or some create-only or override-always sort of arrangement, or maybe some filters for some sorting/slicing data tables, etc.
It's then useful for not being jittery on load + not storing half-baked data just to have semi-persistence of incomplete forms. Quickly rendering existing state is always a win, without resetting it then loading the updated position.
Otherwise I could see this introducing a lot of race conditions and additional work being done post-load to clean up the data.
Since there's a relatively short line here where it just makes sense to rebuild the state, it's probably preferable to utilize this for smaller isolated parts of the site, like a single form's data, instead of the wider state. But still an interesting approach none-the-less, made easier with the reducer pattern...