> there's not a great story for adding a second machine if the first one can't handle the load
I mean this is basically your redis situation right? Just with a very specialized "redis".
You could scale this out, even after the pretty massive ability to scale up is exceeded. Have some front-end servers that act as a connection pooler to your datastore. Or shard the datastore, and have clients only request from the shards that they are currently looking at.
Right, and then the question is "is my specialized datastore gonna be faster than Redis" right? And it seems totally reasonable that you could make something faster eventually - but I think it's not a reasonable goal within the timeframe of the go rewrite (one Sunday afternoon and evening). Especially if you want to extend that system such that other services could talk to it!
The entire timeframe of this project was 2 weeks, and the critical period (most activity / new eyes) was a couple of days.
Sorry, I'm talking hypothetically about how this would be designed, not in the context of your specific timeframe!
> "is my specialized datastore gonna be faster than Redis"
Absolutely! With how efficient this code would be, you'd likely never need to scale horizontally, and in that case it is extremely easy to compete with a network hop (at least 1ms latency) versus L1 cache (<50ns)
The comparison with redis otherwise only applies once you do need to scale horizontally.
There's also the fact that redis must be designed to support any query pattern efficiently; a much harder problem then supporting just 1-2 queries efficiently.
I mean this is basically your redis situation right? Just with a very specialized "redis".
You could scale this out, even after the pretty massive ability to scale up is exceeded. Have some front-end servers that act as a connection pooler to your datastore. Or shard the datastore, and have clients only request from the shards that they are currently looking at.