Updating balances using an RDBMS is like managing your finances with pencils and erasers. Unless you somehow ban the UPDATE statement.
Updating balances with Kafka is like working in pen. You can't[1] change the ledger lines, you can only add corrections after the fact.
[1] Yes, Kafka records can be updated/deleted depending on configuration. But when your codebase is written around append-only operations, in-place mutations are hard and corrections are easy, so your fellow programmers fall into the 'pit of success'.
The original sin of SQL is that it begins without memory.
The original sin of Kafka is that it begins with only memory.
To me the right middle way is relational temporal tables[0]. You get both the memoryless query/update convenience and the ability to travel through time.
[0] SQL:2011 introduced temporal data, but in a slightly janky just-so-happens-to-fit-Oracle's-preference kind of way.
Blockchain databases take this model to the extreme - it is a database that is append only and immutable, with cryptographic guarantees of the ledger.
I'm not selling any particular tool but Amazon's QLDB is an interesting example of a blockchain-based database. I am interested to see how things like Kafka and this might come together somehow.
In my opinion, we have evolved to a point where storage is not a concern for temporal use cases - i.e. we can now store every change in an immutable fashion. When you think about this being an append only transaction log that you never have to purge, and you make observable events on that log (which is what most CDC systems do)... yeah it works. Now you have every change, cryptographically secure, with events to trigger downstream consumers and you can really rethink the whole architecture of monolithic databases vs. data platforms.
I would just put the ledger in a database table if it's that important and maintain the current state of the account in a separate table. ACID transactions and database constraints make this kind of consistency easier to achieve than many alternatives. It's also easier to prove correctness since you can run queries that return consistent results thanks to the isolation guaranteed by ACID. (Modulo some corner cases that are not hard to work around.)
> ACID transactions and database constraints make this kind of consistency easier to achieve than many alternatives.
If your company only runs one database.
> It's also easier to prove correctness
RDBMSs are wonderful and I don't consider them unreliable at all. But I can't prove the correctness of my teammate's actions. I want them to show their working by putting their updates onto an append-only ledger.
> I think the same argument can be made with "only one Kafka cluster"
No it can't. Kafka events for the same partition will always be processed in order. If you have a stream transformation that reads from a topic that lives in one cluster and writes to a topic that lives in another cluster, then (as long as you chose the right partition key) everything will work correctly. Even if you have a sequence of transformations that zig-zag between two different clusters, it will do the right thing. You can't achieve that with traditional databases.
