lower compute cost especially over longer sequence length. Depending on context ...

pico_creator 24 days ago | parent | context | favorite | on: RWKV Language Model

lower compute cost especially over longer sequence length. Depending on context length, its 10x, 100x, or even 1000x+ cheaper. (quadratic vs linear cost difference)