I use redis to replace RabbitMQ as well, but it's all abstracted away in Celery. A question: Don't you need polling when you use redis in this way? Does it have some sort of notification functionality, apart from the recently added pubsub?
Our queues are implemented using BLPOP which blocks until items arrive, so no polling is needed.
The newer pub/sub stuff promises to be very useful, too, but we didn't have a use-case for that, yet. Our app generally needs messages to persist until they are consumed.
What I can say however is that none of the concerns we initially had about performance/scalability held any water.
We are still running on a single redis instance (plus a slave) on a moderately sized server and it happily processes 100 messages/sec average between 4 producers and a varying number of consumers (20-100).
To add insult to injury our monitoring metrics show that this isn't even a worthwhile load for redis. The server is nowhere near breaking a sweat, the CPUs barely drop below 90% idle, there's no disk i/o to speak of, and the memory usage is more than reasonable (plenty headroom for our purposes).
Thus "just throw it at redis" has long become a common stopgap meme in this particular project. And so far we didn't have to replace any of these supposed stopgaps with something else.
