If one does not need petabytes of scale, but am otherwise interested in the data lineage / observability / workflow being sold by databricks, what would you suggest?
Some evaluation Criteria:
- Ease of maintenance and operation is almost paramount.
- It's fine if the solution never lives anywhere but 1 single virtual server that scales vertically (data might grow to a couple TB, but not PETA BYTES)
- Similarly, 20 9's is not a criteria. If the machine fails and it takes an hour till someone goes and re-deploys, that's fine.
- Declarative, reproducible deployment with an easy upgrade story would be great
- Ideal if the deployment can be run locally for quick developmnet
Some evaluation Criteria:
- Ease of maintenance and operation is almost paramount.
- It's fine if the solution never lives anywhere but 1 single virtual server that scales vertically (data might grow to a couple TB, but not PETA BYTES)
- Similarly, 20 9's is not a criteria. If the machine fails and it takes an hour till someone goes and re-deploys, that's fine.
- Declarative, reproducible deployment with an easy upgrade story would be great
- Ideal if the deployment can be run locally for quick developmnet