Seamus's notes and animations related Clock Syncronization and NTP.
You also may wish to see my Clock Sources and System Time page.
Index:
- Definitions of the terms used in this document
- Timelines, Offsets, and Clock Synchronization
- NTP - Netowk Time Protocol
- Sources for this document
Definitions of the terms used in this document:
| Term | Definition |
|---|---|
| Wall Clock Time | Time displayed in a format understandable to humans, such as "12:00 am, Jan 1st, 1970." |
| Real-Time Clock | A hardware-based clock found on system motherboards, functions independent of a computer's power state. |
| System Time | A software-based clock or time line initialized during system boot from the real-time clock |
| Reference Clock | A highly precise and stable time source used to synchronize and calibrate other clocks. |
| Clock Cycles | A measure of the transitions in an electronic circuit, akin to the movement of a production line. |
| Counter | A device that increments based on specific events or time periods, independent of CPU-executed code. |
| Timer | A device that tracks the number of increments occurring between two points in time. |
| Clock Source | A device queried by CPU-executed code to obtain the current value of a counter. |
| Monotonic Counter | A counter that increments at regular intervals, such as once per second, analogous to a metronome. |
| System | A self-contained computing entity, such as a Linux server, Windows desktop, or Mac laptop. |
| Time Line | A graphical or numerical depiction of the passage of time. |
| Time Scale | A representation of time, either graphical or numerical, adjusted to illustrate drift or skew |
Timelines, Offsets and Clock Synchronization
Single Source of Time
If we have a single watch, can we can be sure of what the time is?
There are number of problems with using a single watch to determine the time, such as...
- How can we can confirm that the watch is displaying the correct time. It could be...
- correct
- behind the correct time
- ahead of the correct time
- running too fast
- running too slow
- drifting,
- sometimes running too fast, and/or sometimes running too slow.
Multiple Sources of Time
If we got a 2nd watch (B) in addition to our 1st watch (A), could we be more sure of what the time is?
Is having two watches better than having just one?
If the 2nd watch (B) shows the same time as our 1st watch (A), then we might assume that both are correct.
Clock Skew / Offset
If we got a 2nd watch (B) in addition to our 1st watch (A), it might display a different time.
The time displayed on the 2nd watch (B) might be ahead of the 1st watch (A), or it might be behind.
"A man with a watch knows what time it is. A man with two watches is never sure." Segal's Law
In the example below the time on Watch (B) is 5 minutes ahead of Watch (A)...
How should we set the correct time on the 2 watches?
Should we set watch (A) to the time displayed on watch (B)?
Should we set watch (B) to the time displayed on watch (A)?
Should we (split the difference) by setting both watches to the average time displayed on both watches?
With just 2 ordanary watches it is impossible to determine if the time is correct on either of the Watches.
Clock Drift
If we set the watches to display the same time, they may drift apart as time passes.
Watch (B) is Running FASTER than Watch (A)
Watch (B) is Running SLOWER than Watch (A)
Correcting Clock Drift
There are no perfect solutions, but we can take various actions to regularly syncronize 2 watches.
One simple, but time consuming solution is to...
Designate watch (A) to act as the reference time source and regularly adjust the time on watch (B) to bring it inline with watch (A)
If Watch (B) consistently drifts at a fixed rate, such as being 5 seconds slow per day:
- This implies a steady deviation in its timekeeping accuracy.
- By recognizing this consistent drift pattern, we can adjust the time on Watch (B) periodically.
- These adjustments account for the observed drift, eliminating the need to consult Watch (A) for each correction.
Watch (B)'s Timekeeping is Unstable
If Watch (B) is drifting at an inconsitant rate such as in the example below.
For example,
- Watch (B) was initially in sync with Watch (A)
- Watch (B) started running SLOWER than Watch (A)
- Watch (B) started running FASTER than Watch (A)
- Watch (B) is once again in sync with Watch (A)
Watch (A) would need to be referenced for each adjustment of Watch (B).
Synchronizing Watches from a Remote Clock Source
In the preceding examples, we had 2 watches that we were manualy keeping in Sync with each other.
How do we check if our 2 watches are in sync with the real world time?
How do we correct our 2 watches if they are out of sync with the real world time?
We could make a phone call to a person or service that has access to an accurate reference clock, such as an atomic clock.
There are many issues with this solution including the following...
- The delay for the person or service to read the clock and to say the time.
- The delay for the electrical signal to travel through the telephone network.
- The delay between when you have heard the message and understood it.
- The delay for setting the time. (pressing buttons etc..)
If we were just setting the time on some ordanary watches, we could account for these delays and they would not cause much of a problem. However, there are circumstances where more accuracy is needed.
NTP - Network Time Protocol
NTP (Network Time Protocol), is a system to automatically and accurately synchronise the clocks in computers, routers, and numerous other devices from remote NTP servers.
NTP was invented by Dr. David L. Mills in 1985 and is the most common networking protocol used for clock synchronization between computer systems.
NTP operates on a hierarchical structure of timekeeping devices.
The NTP devices with a stratum level of 1 provide the most accurate reference time.
The Time is propagated through various layers of NTP devices to end devices such as Desktop Computers, Laptops, home routers, etc...
NTP Client to NTP Server Polling Mechanism
The NTP polling mechanism uses the UDP packet format descriped below and 4 different TimeStamps to calculate the Round Trip Deley and the Clock Offset.
NTP Packet Format
+-------------------------------------------+ | LI | VN | Mode | Stratum | Poll |Precision| +-------------------------------------------+ | Root Delay | +-------------------------------------------+ | Root Dispersion | +-------------------------------------------+ | Reference Identifier | +-------------------------------------------+ | | | Reference TimeStamp | | | +-------------------------------------------+ | | | Origin TimeStamp | | | +-------------------------------------------+ | | | Recieve TimeStamp | | | +-------------------------------------------+ | | | Transmit TimeStamp | | | +-------------------------------------------+ | | | Optional Fields / Digest | | | +-------------------------------------------+
NTP Packet Fields
- LI (Leap Indicator) 2-bits
- VN (NTP Version Number) 3-bits
- Mode (Work Mode) 3-bits code that indicates the work mode, can be set to either of these values:
- 0 - reserved
- 1 - symmetric active
- 2 - symmetric passive
- 3 - client
- 4 - server
- 5 - broadcast or multicast
- 6 - NTP control message
- 7 - reserved for private use.
- Stratum (Statum Level) 8-bits. An integer ranging from 1 to 16.
- Note: Stratum 1 clock has the highest precision, and a stratum 16 clock is not synchronized and cannot be used as a reference clock.
- Poll ( Poll Interval ) - 8-bits maximum interval between successive messages.
- Precision (precision of the local clock) - 8-bit signed integer.
- Root Delay - Roundtrip delay to the primary reference source.
- Root Dispersion - The maximum error of the local clock relative to the primary reference source.
- Reference Identifier - Identifier of the particular reference source.
- Reference Timestamp - Local time at which the local clock was last set or corrected.
- Originate Timestamp - Local time at which the request departed from the client for the service host.
- Receive Timestamp - Local time at which the request arrived at the service host.
- Transmit Timestamp - Local time at which the reply departed from the service host for the client.
- Authenticator - Authentication information.
NTP Request Flow
As the NTP packet traverses from the client, across the netowk, theough the server, across the network, and back to the client. 4 different timestamps are added to the packet.
- T1 - the timestamp of when the packet was sent from the NTP Client
- T2 - the timestamp of when the packet was received on the NTP Server
- T3 - the timestamp of when the packet was sent from the NTP Server
- T4 - the timestamp of when the packet was received on the NTP Client
Formulas for calculating the RTD and Offset
Round Trip Delay = (T4 - T1) - (T3 - T2)
Clock Offset = [(T2 - T1) + (T3 - T4)] / 2
3 example calculations for RTD and Offset.
-----------------------------------------
Example A
TimeStamps T1=0, T2=2, T3=12, and T4=14
RTD = ( T4 + T1 ) + ( T3 + T2 )
( 14 + 0 ) | ( 12 + 2 )
( 14 ) + ( 10 )
_______
4
~~~~~~~
Offset = [( T2 + T1 ) + ( T3 + T4 )] /2
[( 2 + 0 ) | ( 12 | 14 )] /2
[( 2 ) + ( +2 )] /2
[ 0 ] /2
_______
0
~~~~~~~
-----------------------------------------
Example B
TimeStamps T1=0, T2=4, T3=6, and T4=10
RTD = ( T4 + T1 ) + ( T3 + T2 )
( 10 + 0 ) | ( 6 + 4 )
( 10 ) + ( 2 )
_______
8
~~~~~~~
Offset = [( T2 + T1 ) + ( T3 + T4 )] /2
[( 4 + 0 ) | ( 6 | 10 )] /2
[( 4 ) + ( +4 )] /2
[ 0 ] /2
_______
0
~~~~~~~
-----------------------------------------
Example C
TimeStamps T1=10, T2=2, T3=12, and T4=24
RTD = ( T4 + T1 ) + ( T3 + T2 )
( 24 + 10 ) | ( 12 + 2 )
( 14 ) + ( 10 )
_______
4
~~~~~~~
Offset = [( T2 + T1 ) + ( T3 + T4 )] /2
[( 2 | 10 ) | ( 12 | 24 )] /2
[( +8 ) | ( +12 )] /2
[ 20 ] /2
_______
10
~~~~~~~
------------------------------------------
Round Trip Delay and Offset Calculator
Enter sample values for T1, T2, T3, and T4 (in the format hh:mm:ss):
NTP Client
NTP Server
Networks and NTP Precision and Accuracy
The precision and accuracy of system time achievable and maintainable by an NTP (Network Time Protocol) client depend significantly on the network infrastructure between the NTP client and the NTP servers. * In a Local Area Network (LAN), the accuracy of NTP synchronization can typically reach the order of 100 microseconds (us).
-
Within a Wide Area Network (WAN), the accuracy may decrease slightly, with synchronization achievable in the order of several tens of milliseconds.
-
Over the Internet, the accuracy of NTP synchronization depends heavily on many different factors. Typically, selecting NTP servers closer to the client location rather than those farther away results in more precise time synchronization.
However, it's important to note that accuracy in all these scenarios is significantly influenced by factors such as jitter and asymmetric network paths.
If you need a higher level of clock precision than NTP can provide, the PTP system may be better suited for your needs.
https://en.wikipedia.org/wiki/Precision_Time_Protocol
Sources for this document:
Intel CPU specifications and developer guides
- Intel 64-ia-32-architectures-software-developer-vol-2a-manual.pdf
- Intel 64-ia-32-architectures-software-developer-manual-325462.pdf
- Intel 64-ia-32-architectures-software-developer-vol-3b-part-2-manual
- Intel timestamp-counter-scaling-virtualization-white-paper.pdf
- Intel xeon-e5-v2-spec-update.pdf
Linux Kernel and Linux Distribution documentation
- Linux kernel Documentation kvm timekeeping
- access.redhat.com/solutions/18627
- kb.vmware.com
- RedHat - Virtualization-KVM_guest_timing_management
- bugzilla.redhat.com/show_bug.cgi?id=698842
- blog.cr0.org time-stamp-counter-disabling-oddities.html
- cs.virginia.edu/~evans/cs216/guides/x86.html
- man7.org/linux/man-pages/man2/prctl.2.html
- chromium-os/how-tos-and-troubleshooting/tsc-resynchronization
- wiki.archlinux.org CPU_frequency_scaling
- btorpey.github.io/blog/2014/02/18/clock-sources-in-linux/
- lkml.org/lkml/2005/11/4/173
- [Manual page clock_gettime]
NTP (Official Documentation)
- www.ntp.org/documentation/
- www.eecis.udel.edu/~mills/ntp/html/index.html
- Computer-Network-Time-Synchronization-Protocol By David L Mills
- Computer-Network-Time-Synchronization-Protocol 2nd By David L Mills
- Leap second (XKCD always counts as official ;)
Wikipedia links
- Segals Law
- List_of_Intel_microprocessors
- P5 microarchitecture
- Intel SpeedStep
- Wall-clock_time
- Time_Stamp_Counter
- Linux_kernel
- Allan_variance
Extra Info