It's not "forever", it's "for as long as we want to offer it", they can stop offering it or terminate you whenever, and if you go under any of these for 7 days they will "reclaim" it:
> CPU utilization for the 95th percentile is less than 20%
> Network utilization is less than 20%
> Memory utilization is less than 20% (applies to A1 shapes only)
I interpret this as the machine is deemed idle and reclaimed if ALL of these are true. Not Any.
Am I wrong?
Have a an instance that I used for some experimenting. It is doing quite a bit of CPU and RAM but almost zero network. It has been for months. But has not been reclaimed...
I've got one. I'm using it to power an LLM for a chatbot me and a few friends use.
The memory and disk speeds are really slow, but there's plenty of memory to keep whatever you need loaded. I can mmap massive models and get decent inferencing performance without spending a dime.
One of these days I feel like we'll draw too much attention to the program, and they'll have to shut it down. Until then...
It's not "forever", it's "for as long as we want to offer it", they can stop offering it or terminate you whenever, and if you go under any of these for 7 days they will "reclaim" it:
> CPU utilization for the 95th percentile is less than 20%
> Network utilization is less than 20%
> Memory utilization is less than 20% (applies to A1 shapes only)