Hacker News new | past | comments | ask | show | jobs | submit login

> You can also use a wildcard in the path (e.g. s3://mybucket/hits/*.parquet) to query a list of files.

So is this an FDW that provides a similar function to Amazon Athena?

Does every query fetch everything again or does it somehow cache the downloaded data files?

Running something like this on EC2 is interesting as you wouldn’t pay the intermediate transfer costs. Only the final results would egress.




It caches the files on locally attached NVMe drives in the background, and continues to use range requests to S3 for queries until the full download completes.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: