For anyone interested in this subject, the presentation has clean, easy to understand drawings which make the reading so much pleasant. I read all 154 pages mainly because of that, and I must admit it gave me a cleaner picture of how ZeroMQ can be integrated in distributed systems. Thanks for sharing
People interested in these slides might like to check out the Brubeck framework. It can use gevent for concurrency and ZeroMQ to communicate with Mongrel2, it's webserver.
So Mongrel2 sits in front of Brubeck and proxies requests back?
I've been playing with ZeroMQ quite a bit the last few weeks because I needed a fast way to pump data from a Python app into the Neo4j Batch Loader, which runs on the JVM. I wrote a lightweight ZeroMQ server in Jython called Lightsocket and a Python client that (with multiple clients) was pushing about 10,000 inserts per second into Neo4j on my dev system.
Jython was the bottleneck, and I could have rewritten it in Java or Scala, but the data was already loaded. However, I went looking around to see if there was a ZeroMQ-based container or if anyone had modified Tomcat or Jetty to run as a ZeroMQ server, but I couldn't find much on it. Does anyone know of a ZeroMQ server/container for the JVM?
Haven't read the preso yet but I just wanted to chime in and say that I love ZeroMQ - it's one of those simple libs that are so cool you want to run out and build something with it the moment you see a four line example.
I've used it to build some cool stuff, and can't wait to use it more often.
Ah thanks, it must be something like that, though later on in the FAQ there they also say:
> ØMQ gets you more throughput than TCP has using intelligent batching algorithms.
yes, intelligent, sounds like a good idea. ;-)
EDIT: ok here's a little more detail as to what they mean by that in the next FAQ entry:
> ØMQ batches messages in opportunistic manner. Rather than waiting for a predefined number of messages and/or predefined time interval, it sends all the messages available at the moment in one go. Imagine the network interface card is busy sending data. Once it is ready to send more data it asks ØMQ for new messages. ØMQ sends all the messages available at the moment. Does it harm the latency of the first message in the batch? No. The message won't be sent earlier anyway because the networking card was busy. On the contrary, latency of subsequent messages will be improved because sending single batch to the card is faster then sending lot of small messages. On the other hand, if network card isn't busy, the message is sent straight away without waiting for following messages. Thus it'll have the best possible latency.
> We are working on delivering ØMQ over alternative networking stacks, thus having advantage of features like kernel bypass, avoiding TCP/IP overhead, using high-performance networking hardware etc. That way we can get the latency as low as 10 microseconds.
