Leap second is applied at the 59th TAI second instead of 59th UTC second

[szqtc]

I run the following code:

fn main() {
    let utc0 = Epoch::from_gregorian_utc_hms(2016, 12, 31, 23, 59, 0);
    let t0 = utc0.to_tai_seconds();
    println!("T0={}, TAI0={}", utc0.to_gregorian_str(hifitime::TimeScale::UTC), t0);

    let mut t1 = t0;
    for dt in 1..62 {
        t1 = t1 + 1.;
        let utc1 = Epoch::from_tai_seconds(t1);
        let utc2 = Epoch::from_utc_seconds(utc0.to_utc_seconds() + (dt as f64));
        println!("+{} sec, TAI={}, T1={}, T2={}", dt, t1,
            utc1.to_gregorian_str(hifitime::TimeScale::UTC),
            utc2.to_gregorian_str(hifitime::TimeScale::UTC));
    }
}

And the output is

T0=2016-12-31T23:59:00 UTC, TAI0=3692217576
+1 sec, TAI=3692217577, T1=2016-12-31T23:59:01 UTC, T2=2016-12-31T23:59:01 UTC
...
+22 sec, TAI=3692217598, T1=2016-12-31T23:59:22 UTC, T2=2016-12-31T23:59:22 UTC
+23 sec, TAI=3692217599, T1=2016-12-31T23:59:23 UTC, T2=2016-12-31T23:59:23 UTC
+24 sec, TAI=3692217600, T1=2016-12-31T23:59:23 UTC, T2=2016-12-31T23:59:24 UTC   <-- leap second for T1
+25 sec, TAI=3692217601, T1=2016-12-31T23:59:24 UTC, T2=2016-12-31T23:59:25 UTC
...
+59 sec, TAI=3692217635, T1=2016-12-31T23:59:58 UTC, T2=2016-12-31T23:59:59 UTC
+60 sec, TAI=3692217636, T1=2016-12-31T23:59:59 UTC, T2=2017-01-01T00:00:00 UTC
+61 sec, TAI=3692217637, T1=2017-01-01T00:00:00 UTC, T2=2017-01-01T00:00:01 UTC

When I initiate Epoch using from_tai_seconds, leap second appears at 2016-12-31T23:59:23 UTC or 2017-01-01T00:00:00 TAI. But if I use from_utc_seconds, there is no leap second.

I use hifitime = "4.1.3".

[ChristopherRabotin]

Hi there,

What output are you expecting? At first glance, this looks correct to me because you're calling the to_utc_seconds which accounts for leap seconds when initializing T1.

[szqtc]

Thanks for the reply. I am expecting the leap second at 2016-12-31T23:59:60 UTC (or there are two 23:59:59). T1 is created with Epoch::from_tai_seconds(t1), does it mean to_utc_seconds is called implicitly inside from_tai_seconds function?

Sorry, I just wonder what I should do if I want to get the UTC time which is some seconds after a reference time.

• If I add a Duration to the Epoch using Epoch::from_gregorian_utc_hms(2016, 1, 1, 0, 0, 0)+31622401.seconds(), I will get 2017-01-01T00:00:01 UTC.
• If I convert to TAI, add some seconds and convert back, the leap second seems at 2016-12-31T23:59:23 UTC.

[ChristopherRabotin]

I believe the leap second happens at the TAI time, not at the UTC time.

From data.iana.org/time-zones/data/leap-seconds.list:

UTC differs from TAI by an integer number of seconds; it is the basis of all activities in the world.
(...)
DTAI: The difference DTAI= TAI-UTC in units of seconds
It is the quantity to add to UTC to get the time in TAI

The 60th second in UTC is only supported in the initialization of UTC epochs, but it is not printed back out because the second didn't actually happen in the UTC clock: the leap second allows us to stop counting the UTC time for one second. So operations on UTC time should not account for it; in other words, the leap second only matters when converting from one time scale to another.

Another way to explain it is that the 60th second on the leap second day in UTC does not exist: it corresponds to the exact same time as the 59th second in TAI clocks.

For this reason, initializing on the leap second will actually be printed back out as the 59th second, not the 60th.

I lightly modified your test case as follows to demonstrate how I recommend adding seconds to epochs, and the output:

#[test]
fn gh_417() {
    use hifitime::{Epoch, Unit};
    let utc0 = Epoch::from_gregorian_utc_hms(2016, 12, 31, 23, 59, 0);
    let tai0 = utc0.to_time_scale(hifitime::TimeScale::TAI);

    for dt in 1..62 {
        let tai_next = tai0 + Unit::Second * dt;
        let utc_next = utc0 + Unit::Second * dt;
        println!("+{dt} s => TAI = {tai_next}\tTAI as UTC = {tai_next:?}\tUTC = {utc_next}");
    }

    let e = Epoch::from_gregorian_utc(2016, 12, 31, 23, 59, 60, 0);
    println!("On leap second: {e}");
    println!(
        "Equivalent in TAI: {}",
        e.to_time_scale(hifitime::TimeScale::TAI)
    );
}

The output is the following:

+1 s => TAI = 2016-12-31T23:59:37 TAI   TAI as UTC = 2016-12-31T23:59:01 UTC    UTC = 2016-12-31T23:59:01 UTC
+2 s => TAI = 2016-12-31T23:59:38 TAI   TAI as UTC = 2016-12-31T23:59:02 UTC    UTC = 2016-12-31T23:59:02 UTC
+3 s => TAI = 2016-12-31T23:59:39 TAI   TAI as UTC = 2016-12-31T23:59:03 UTC    UTC = 2016-12-31T23:59:03 UTC
+4 s => TAI = 2016-12-31T23:59:40 TAI   TAI as UTC = 2016-12-31T23:59:04 UTC    UTC = 2016-12-31T23:59:04 UTC
+5 s => TAI = 2016-12-31T23:59:41 TAI   TAI as UTC = 2016-12-31T23:59:05 UTC    UTC = 2016-12-31T23:59:05 UTC
+6 s => TAI = 2016-12-31T23:59:42 TAI   TAI as UTC = 2016-12-31T23:59:06 UTC    UTC = 2016-12-31T23:59:06 UTC
+7 s => TAI = 2016-12-31T23:59:43 TAI   TAI as UTC = 2016-12-31T23:59:07 UTC    UTC = 2016-12-31T23:59:07 UTC
+8 s => TAI = 2016-12-31T23:59:44 TAI   TAI as UTC = 2016-12-31T23:59:08 UTC    UTC = 2016-12-31T23:59:08 UTC
+9 s => TAI = 2016-12-31T23:59:45 TAI   TAI as UTC = 2016-12-31T23:59:09 UTC    UTC = 2016-12-31T23:59:09 UTC
+10 s => TAI = 2016-12-31T23:59:46 TAI  TAI as UTC = 2016-12-31T23:59:10 UTC    UTC = 2016-12-31T23:59:10 UTC
+11 s => TAI = 2016-12-31T23:59:47 TAI  TAI as UTC = 2016-12-31T23:59:11 UTC    UTC = 2016-12-31T23:59:11 UTC
+12 s => TAI = 2016-12-31T23:59:48 TAI  TAI as UTC = 2016-12-31T23:59:12 UTC    UTC = 2016-12-31T23:59:12 UTC
+13 s => TAI = 2016-12-31T23:59:49 TAI  TAI as UTC = 2016-12-31T23:59:13 UTC    UTC = 2016-12-31T23:59:13 UTC
+14 s => TAI = 2016-12-31T23:59:50 TAI  TAI as UTC = 2016-12-31T23:59:14 UTC    UTC = 2016-12-31T23:59:14 UTC
+15 s => TAI = 2016-12-31T23:59:51 TAI  TAI as UTC = 2016-12-31T23:59:15 UTC    UTC = 2016-12-31T23:59:15 UTC
+16 s => TAI = 2016-12-31T23:59:52 TAI  TAI as UTC = 2016-12-31T23:59:16 UTC    UTC = 2016-12-31T23:59:16 UTC
+17 s => TAI = 2016-12-31T23:59:53 TAI  TAI as UTC = 2016-12-31T23:59:17 UTC    UTC = 2016-12-31T23:59:17 UTC
+18 s => TAI = 2016-12-31T23:59:54 TAI  TAI as UTC = 2016-12-31T23:59:18 UTC    UTC = 2016-12-31T23:59:18 UTC
+19 s => TAI = 2016-12-31T23:59:55 TAI  TAI as UTC = 2016-12-31T23:59:19 UTC    UTC = 2016-12-31T23:59:19 UTC
+20 s => TAI = 2016-12-31T23:59:56 TAI  TAI as UTC = 2016-12-31T23:59:20 UTC    UTC = 2016-12-31T23:59:20 UTC
+21 s => TAI = 2016-12-31T23:59:57 TAI  TAI as UTC = 2016-12-31T23:59:21 UTC    UTC = 2016-12-31T23:59:21 UTC
+22 s => TAI = 2016-12-31T23:59:58 TAI  TAI as UTC = 2016-12-31T23:59:22 UTC    UTC = 2016-12-31T23:59:22 UTC
+23 s => TAI = 2016-12-31T23:59:59 TAI  TAI as UTC = 2016-12-31T23:59:23 UTC    UTC = 2016-12-31T23:59:23 UTC
+24 s => TAI = 2017-01-01T00:00:00 TAI  TAI as UTC = 2016-12-31T23:59:23 UTC    UTC = 2016-12-31T23:59:24 UTC
+25 s => TAI = 2017-01-01T00:00:01 TAI  TAI as UTC = 2016-12-31T23:59:24 UTC    UTC = 2016-12-31T23:59:25 UTC
+26 s => TAI = 2017-01-01T00:00:02 TAI  TAI as UTC = 2016-12-31T23:59:25 UTC    UTC = 2016-12-31T23:59:26 UTC
+27 s => TAI = 2017-01-01T00:00:03 TAI  TAI as UTC = 2016-12-31T23:59:26 UTC    UTC = 2016-12-31T23:59:27 UTC
+28 s => TAI = 2017-01-01T00:00:04 TAI  TAI as UTC = 2016-12-31T23:59:27 UTC    UTC = 2016-12-31T23:59:28 UTC
+29 s => TAI = 2017-01-01T00:00:05 TAI  TAI as UTC = 2016-12-31T23:59:28 UTC    UTC = 2016-12-31T23:59:29 UTC
+30 s => TAI = 2017-01-01T00:00:06 TAI  TAI as UTC = 2016-12-31T23:59:29 UTC    UTC = 2016-12-31T23:59:30 UTC
+31 s => TAI = 2017-01-01T00:00:07 TAI  TAI as UTC = 2016-12-31T23:59:30 UTC    UTC = 2016-12-31T23:59:31 UTC
+32 s => TAI = 2017-01-01T00:00:08 TAI  TAI as UTC = 2016-12-31T23:59:31 UTC    UTC = 2016-12-31T23:59:32 UTC
+33 s => TAI = 2017-01-01T00:00:09 TAI  TAI as UTC = 2016-12-31T23:59:32 UTC    UTC = 2016-12-31T23:59:33 UTC
+34 s => TAI = 2017-01-01T00:00:10 TAI  TAI as UTC = 2016-12-31T23:59:33 UTC    UTC = 2016-12-31T23:59:34 UTC
+35 s => TAI = 2017-01-01T00:00:11 TAI  TAI as UTC = 2016-12-31T23:59:34 UTC    UTC = 2016-12-31T23:59:35 UTC
+36 s => TAI = 2017-01-01T00:00:12 TAI  TAI as UTC = 2016-12-31T23:59:35 UTC    UTC = 2016-12-31T23:59:36 UTC
+37 s => TAI = 2017-01-01T00:00:13 TAI  TAI as UTC = 2016-12-31T23:59:36 UTC    UTC = 2016-12-31T23:59:37 UTC
+38 s => TAI = 2017-01-01T00:00:14 TAI  TAI as UTC = 2016-12-31T23:59:37 UTC    UTC = 2016-12-31T23:59:38 UTC
+39 s => TAI = 2017-01-01T00:00:15 TAI  TAI as UTC = 2016-12-31T23:59:38 UTC    UTC = 2016-12-31T23:59:39 UTC
+40 s => TAI = 2017-01-01T00:00:16 TAI  TAI as UTC = 2016-12-31T23:59:39 UTC    UTC = 2016-12-31T23:59:40 UTC
+41 s => TAI = 2017-01-01T00:00:17 TAI  TAI as UTC = 2016-12-31T23:59:40 UTC    UTC = 2016-12-31T23:59:41 UTC
+42 s => TAI = 2017-01-01T00:00:18 TAI  TAI as UTC = 2016-12-31T23:59:41 UTC    UTC = 2016-12-31T23:59:42 UTC
+43 s => TAI = 2017-01-01T00:00:19 TAI  TAI as UTC = 2016-12-31T23:59:42 UTC    UTC = 2016-12-31T23:59:43 UTC
+44 s => TAI = 2017-01-01T00:00:20 TAI  TAI as UTC = 2016-12-31T23:59:43 UTC    UTC = 2016-12-31T23:59:44 UTC
+45 s => TAI = 2017-01-01T00:00:21 TAI  TAI as UTC = 2016-12-31T23:59:44 UTC    UTC = 2016-12-31T23:59:45 UTC
+46 s => TAI = 2017-01-01T00:00:22 TAI  TAI as UTC = 2016-12-31T23:59:45 UTC    UTC = 2016-12-31T23:59:46 UTC
+47 s => TAI = 2017-01-01T00:00:23 TAI  TAI as UTC = 2016-12-31T23:59:46 UTC    UTC = 2016-12-31T23:59:47 UTC
+48 s => TAI = 2017-01-01T00:00:24 TAI  TAI as UTC = 2016-12-31T23:59:47 UTC    UTC = 2016-12-31T23:59:48 UTC
+49 s => TAI = 2017-01-01T00:00:25 TAI  TAI as UTC = 2016-12-31T23:59:48 UTC    UTC = 2016-12-31T23:59:49 UTC
+50 s => TAI = 2017-01-01T00:00:26 TAI  TAI as UTC = 2016-12-31T23:59:49 UTC    UTC = 2016-12-31T23:59:50 UTC
+51 s => TAI = 2017-01-01T00:00:27 TAI  TAI as UTC = 2016-12-31T23:59:50 UTC    UTC = 2016-12-31T23:59:51 UTC
+52 s => TAI = 2017-01-01T00:00:28 TAI  TAI as UTC = 2016-12-31T23:59:51 UTC    UTC = 2016-12-31T23:59:52 UTC
+53 s => TAI = 2017-01-01T00:00:29 TAI  TAI as UTC = 2016-12-31T23:59:52 UTC    UTC = 2016-12-31T23:59:53 UTC
+54 s => TAI = 2017-01-01T00:00:30 TAI  TAI as UTC = 2016-12-31T23:59:53 UTC    UTC = 2016-12-31T23:59:54 UTC
+55 s => TAI = 2017-01-01T00:00:31 TAI  TAI as UTC = 2016-12-31T23:59:54 UTC    UTC = 2016-12-31T23:59:55 UTC
+56 s => TAI = 2017-01-01T00:00:32 TAI  TAI as UTC = 2016-12-31T23:59:55 UTC    UTC = 2016-12-31T23:59:56 UTC
+57 s => TAI = 2017-01-01T00:00:33 TAI  TAI as UTC = 2016-12-31T23:59:56 UTC    UTC = 2016-12-31T23:59:57 UTC
+58 s => TAI = 2017-01-01T00:00:34 TAI  TAI as UTC = 2016-12-31T23:59:57 UTC    UTC = 2016-12-31T23:59:58 UTC
+59 s => TAI = 2017-01-01T00:00:35 TAI  TAI as UTC = 2016-12-31T23:59:58 UTC    UTC = 2016-12-31T23:59:59 UTC
+60 s => TAI = 2017-01-01T00:00:36 TAI  TAI as UTC = 2016-12-31T23:59:59 UTC    UTC = 2017-01-01T00:00:00 UTC
+61 s => TAI = 2017-01-01T00:00:37 TAI  TAI as UTC = 2017-01-01T00:00:00 UTC    UTC = 2017-01-01T00:00:01 UTC
On leap second: 2016-12-31T23:59:59 UTC
Equivalent in TAI: 2017-01-01T00:00:35 TAI

You will note that on the line that starts with +24, the TAI clock and the UTC clock start diverging by one second because the leap second happened in UTC. There are two identical UTC epochs for the same TAI epoch, which I believe is the correct implementation.

Let me know if that makes sense.

[ephraim71]

Pardon my ignorance, still reading up all about time, but isn't leap second supposed to happen at 23:59:59 UTC, why is it happening at 23:59:23 UTC?

https://en.wikipedia.org/wiki/Leap_second

[szqtc]

Thank you very much for the clarification and kind help.

I also remember leap second happens at UTC. I find some documents to support the point:

1. hpiers.obspm.fr/eoppc/bul/bulc/bulletinc.52

    The difference between UTC and the International Atomic Time TAI is:
     from 2015 July 1, 0h UTC, to 2017 January 1 0h UTC   : UTC-TAI = - 36s
     from 2017 January 1, 0h UTC, until further notice    : UTC-TAI = - 37s 
   

2. www.nist.gov/pml/time-and-frequency-division/time-realization/leap-seconds

   All leap seconds listed in the table are positive leap seconds, which means an extra second is 
   inserted into the UTC time scale. The sequence of events is:
   23h 59m 59s - 23h 59m 60s - 00h 00m 00s
   

3. In data.iana.org/time-zones/data/leap-seconds.list, the last leap second happens at the NTP Time of 3692217600 sec.

   #NTP Time      DTAI    Day Month Year
   ...
   3692217600      37      # 1 Jan 2017
   

   According to https://tickelton.gitlab.io/articles/ntp-timestamps, the convertion of NTP Timestamp to unix time is UNIX=NTP-2208988800. So NTP=3692217600 <=> UNIX=1483228800 <=> 2017-01-01 00:00:00 UTC.

   $ date -u -d @1483228800 +"%Y-%m-%d %H:%M:%S"
   2017-01-01 00:00:00

Besides, I also find astropy.time add leap second at the end of the date in UTC:

>>> from astropy import time, units
>>> t0 = time.Time('2016-12-31T23:59:00', scale='utc')
>>> for dt in range(1, 62):
...    t1 = t0+dt*units.second
...    print(f'+{dt} sec, {t1.utc}')

+1 sec, 2016-12-31T23:59:01.000
...
+59 sec, 2016-12-31T23:59:59.000
+60 sec, 2016-12-31T23:59:60.000
+61 sec, 2017-01-01T00:00:00.000

[ChristopherRabotin]

You've convinced me that hifitime applies the leap second too early. The original IERS file, a copy of which can be found here, state this:

The first column shows an epoch as a number of seconds since 1 January 1900, 00:00:00 (1900.0 is also used to indicate the same epoch.)

For this reason, the LeapSecond structure refers to that offset as timestamp_tai_s.

The IANA leap second file clearly marks that column as "NTP Time", which is UTC based.

To fix this, I think that all that's need is:

1. Modify this line to check the UTC seconds
2. Rename that field to timestamp_utc_s
3. Add regression tests
4. Version bump to 4.2 because we renamed the field.

This might be related to #255.

cc @gwbres for awareness

[google-labs-jules]

Jules is on it. When finished, you will see another comment and be able to review a PR.

[gwbres]

@ChristopherRabotin,

I confirm the fix is needed

[ChristopherRabotin]

@szqtc , I'm working on the fix to this bug. I just want to check that the following output looks good to you as well:

+58 s => TAI = 2017-01-01T00:00:34 TAI	UTC = 2016-12-31T23:59:58 UTC
+59 s => TAI = 2017-01-01T00:00:35 TAI	UTC = 2016-12-31T23:59:59 UTC
+60 s => TAI = 2017-01-01T00:00:36 TAI	UTC = 2017-01-01T00:00:00 UTC
+61 s => TAI = 2017-01-01T00:00:37 TAI	UTC = 2017-01-01T00:00:00 UTC
test test_regression_gh_417_leap_utc ... ok

In Hifitime, leap seconds are not printed as "60 seconds" because the conversion to another time scale follows that clock's tick. Hence the repetition here of midnight on the first of January 2017.

[szqtc]

@ChristopherRabotin

The output is good to me.

I have some questions on the latest version of hifitime:

• Will 2016-12-31T23:59:60 UTC and 2017-01-01T00:00:00 UTC map to different TAI times during the initialization?
• Are there 2 seconds between 2017-01-01T00:00:00 UTC and 2016-12-31T23:59:59 UTC? Do I still need to convert to TAI if I want to calculate the time interval with leap seconds?

Thank you.