Infiniband and the FSMLabs Drill Down design approach

06/14/2011

TimeKeeper on Infiniband.

FSMLabs has just validated TimeKeeper performance on two high frequency trading systems that are based on Infiniband - the super low-latency networking technology.. Performance was superb. To get Infiniband working in the test systems took about 15 minutes. In both sites, TimeKeeper was inserted into a working NTP system as both server and client and just worked. In one system the TimeKeeper Server was itself accepting time from a PTP master clock on an Ethernet network and acted as a bridge. In a second system, the TimeKeeper server ran on a device that contained a GPS time-clock PCI card.

The high performance and “no muss, no fuss” operation validates FSMLabs “drill down” approach to reconciling the conflicting requirements of standards and high performance.  Our design approach relies on drilling down through layers of general purpose software to get raw hardware performance on highly optimized purpose built software for critical functions. Essentially we take on a big part of the effort of balancing standards against performance in our software design/implementation project - so it’s not a problem for over-worked IT staffs.  IT departments want standardized hardware and software that is widely compatible with a large range of devices, applications, and programs. So a special purpose operating system or even one that has been modified to support a special API or for functionality, rapidly becomes a huge expense as it is adapted to support rapidly changing system compute servers, drivers, devices, and software. On the other hand, standard platforms have to be all things to all people and necessarily sacrifice some performance/reliability/security. You can’t get “general purpose” and “finely honed for purpose” in the same box.  Well, you can, if you can bypass or override generic functions in just those places where you need specific performance. Doing that drill down while keeping the rest of the system safe is a pretty difficult technical play, but it pays big dividends.  That’s why we get microsecond timing accuracy over Ethernet - and that’s why our Timekeeper software can leverage the Linux general purpose networking support and the Infiniband drivers to get submicrosecond accuracy over the rock steady Infiniband interconnect. (photo from Chris Dag)