Okay, You never log logs in DB in the first place.
You never fill table with non capped/infinitly growing records (capped = collections with an upper limit).
You use at best rotating collections (like circular buffer ring). But anyway, if you have success the log flow should always grow more than your number of customers (coupling) thus, it grows more than linearly. So the upper limit will grow too.
Tools have software complexity in retrieving, inserting and deleting. There is not a tool that can be log(n) for all cases and be ACID.
The big data fraud is about letting business handling growing set of datas that are inducing diminishing returns in OPEX.
In software theory the more data, the more resource you need that is a growing function of size of your data. Size that grows more than your customers, and linearly other time.
The more customers you have, the longer you keep them, the more they cost you. It is in terms of business stupid.
Storing ALL your logs is like being an living being that refuses to poo. It is not healthy.
Solutions lies in sampling or reducing datas after an amount of time and scheme like round robin databases.
You never fill table with non capped/infinitly growing records (capped = collections with an upper limit).
You use at best rotating collections (like circular buffer ring). But anyway, if you have success the log flow should always grow more than your number of customers (coupling) thus, it grows more than linearly. So the upper limit will grow too.
Tools have software complexity in retrieving, inserting and deleting. There is not a tool that can be log(n) for all cases and be ACID.
The big data fraud is about letting business handling growing set of datas that are inducing diminishing returns in OPEX.
In software theory the more data, the more resource you need that is a growing function of size of your data. Size that grows more than your customers, and linearly other time.
The more customers you have, the longer you keep them, the more they cost you. It is in terms of business stupid.
Storing ALL your logs is like being an living being that refuses to poo. It is not healthy.
Solutions lies in sampling or reducing datas after an amount of time and scheme like round robin databases.