I’m shocked that this hasn’t been a thing from the start. That seems like table ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		jonhohle 30 days ago \| parent \| context \| favorite \| on: Prompt caching for cheaper LLM tokens I’m shocked that this hasn’t been a thing from the start. That seems like table stakes for automating repetitive tasks.

qeternity 30 days ago [–]

It has been a thing. In a single request, this same cache is reused for each forward pass.

It took a while for companies to start metering it and charging accordingly.

Also companies invested in hierarchical caches that allow longer term and cross cluster caching.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact