I’ve got to disagree here, especially with memoization and streaming, deriving metric from structured logs is extremely flexible, relatively fast, and can be configured to be as cheap as you need it to be. With streaming you can literally run your workload on a raspberry pi. Granted, you need to write the code to do so yourself, most off-the-shelf services probably are expensive
memoization isn't free in logs, you're basically deduping an unbounded queue and its difficult to scale from one machine. Its both CPU and Memory heavy. I mean sure you can use scuba, which is great, but that's basically a database made to look like a log store.
> deriving metric from structured logs is extremely flexible
Assuming you can actually generate structured logs reliably. but even if you do, its really easy to silently break it.
> With streaming you can literally run your workload on a raspberry pi
no, you really can't. Streaming logs to a centralised place is exceptionally IO heavy. If you want to generate metrics from it, its CPU heavy as well. If you need speed, then you'll also need lots of RAM, otherwise searching your logs will cause logging to stop. (either because you've run out of CPU, or you've just caused the VFS cache to drop because you're suddenly doing no predictable IO. )
greylog exists for streaming logs. hell, even rsyslog does it. Transporting logs is fairly simple, storing and generating signal from it is very much not.
