Measurement of TimeKeeper

06/16/2011

FSMLabs has a rigorous, highly automated, test and regression system that is absolutely necessary to deliver high performance.

Time Synchronization solutions are hard to test and often are provided with “self-test” components that report numbers which border on wishful thinking. Usually, the time synchronization solution test component reports the “offset” from the correct time - or at least an estimate of that offset. But how accurate is that estimate? If you think about it, if the synchronizer really “knew” the offset, it could correct its time and reduce the
error to zero. Actually, our tests show that those self-reported numbers are often wildly off.  What we do is build various test systems where the “synchronized time” can be compared in some way to an external reference - we want to compare actual time to computed time. For example, when TimeKeeper is trying to lock client system time to a reference time coming from some network time server that collects GPS time, we can run the “pulse per second” generated by that time server into a cable that connects to the client computer directly and then run special software that waits for a pulse and reads the “synchronized time” as the pulse shows up in real-time. TimeKeeper uses data that comes over the network connection, but our test software compares that to the hardware generated pulse-per-second. What you want to see is that the synchronized time is nearly exactly on a second boundary or perhaps a little past depending on how long the signal takes to propagate down the wire.  These measurements allow us to both to evaluate our algorithms for time synchronization and our test tool.

There are three quantities tracked on this graph: “raw”, “offset”, and “local” (click on it for a larger view). “Raw” and “offset” are our estimates of error where the second one is smoothed out by the algorithm. “Localtest” is actual error - using the pulse-per-second hardware. As you can see, these times converge rapidly.  “Raw” is the time TimeKeeper computes to be its instantaneous variance (error) from the reference time.  The IEEE PTP protocol is designed for no-traffic networks that do not introduce much if any variation in transfer time, but in the real-world we cannot rely on having such networks. So TimeKeeper has some sophisticated algorithms to synthesize a correct time from both sources - and we keep a running calculation of what we think the worst case error may be. “Offset” is the error we compute in a further smoothed time that TimeKeeper reports to users - because we want to avoid rapid “corrections” whenever possible. And “localtest” is a time error computed from a GPS “pulse per second” signal run directly into the client for cross-check purposes. That is, “localtest” crosschecks the time TimeKeeper computes against the pulses directly generated by the GPS clock hardware. It’s important to note that not only do we converge on correct time, but we do so very quickly and then lock onto it.