PingPong test for CCR
October 31, 2006
Erlang recently added support for native threads in their runtime. To test performance, someone posted a simple ping-pong benchmark on a mailing list. It spawns N processes, which (1) pings all the other processes, (2) replies to all pings with a pong message, and (3) sends a completion message to a receiver process. I duplicated this demo using the .NET CCR library. The guy who posted the benchmark ran it on a dual-processor machine with 2.4GHz Xeons. Running 700 processes on a single thread took 27 seconds. The same benchmark with 4 Erlang schedulers (i.e., 4 OS threads) took 9.3 seconds. I ran this equivalent benchmark on my pathetic Pentium M (1.6GHz) laptop. Remember, I’ve only got 1 processor. It took 2.6 seconds. The CCR version ran 10X faster than Erlang’s single threaded version on a much faster machine (maybe 50% faster). There are 2*700*700 messages = 980,000 messages (+ 1400 init and complete messages). I’m awfully impressed by these very rough numbers. What other kinds of tests should I run?
Admittedly, Erlang is a far nicer language to program in then .NET & CCR. It is very tedious to construct the dataflow layout with the existing API. I can simplify this greatly with a higher-level API. However, it won’t be as cool as Erlang. Another thing is that Erlang has processes which run as soon as you spawn them. On the CCR (AFAIK), it’s entirely a reactive dataflow model. So you need write your programs differently to map them onto the CCR’s model. But the result is really the same: one could (I think) automatically translate Erlang programs to .NET CCR. So far I’m very excited about the CCR for general concurrency programming.
[edit: I missed a pong reply in my previous post. I updated the numbers in this post.]