No, you're right. I use cloud front and you have to pay for the day at to go from S3 to cloud front servers where it gets cached. Then the costs are $0.14/gb in the cheapest region.

If your cache gets deleted or expired, you'll have to pay to move the data (at s3 rates) to the CDN again when someone requests it.