[Android] Timezone APIs on Android

[Yqwed]

This is a spin-off from mono/mono#21682, as @mdh1418 suggested to do.

tl;dr:

1. As I understand, current implementation reads Android's tzdata directly. Here are few problems with that: neither its location nor its structure is documented or guaranteed. I don't know what .NET's API surface is, but maybe ICU4C would suffice. Overall it endangers users as PR above illustrates.
2. How does current .NET implementation handles TZif4 files?
3. Do you have tests like this? I think old Mono implementation would fail on Asia/Gaza as it has predefined transition up to 2086 (does not fit into 32-bit section of TZif ver. 1)
4. We usually make tzdata changes early in the release cycle. Can you include Android beta builds into testing matrix?

Happy to answer your questions!

Thanks,
Almaz

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This is a spin-off from mono/mono#21682, as @mdh1418 suggested to do.

tl;dr:

1. As I understand, current implementation reads Android's tzdata directly. Here are few problems with that: neither its location nor its structure is documented or guaranteed. I don't know what .NET's API surface is, but maybe ICU4C would suffice. Overall it endangers users as PR above illustrates.
2. How does current .NET implementation handles TZif4 files?
3. Do you have tests like this? I think old Mono implementation would fail on Asia/Gaza as it has predefined transition up to 2086 (does not fit into 32-bit section of TZif ver. 1)
4. We usually make tzdata changes early in the release cycle. Can you include Android beta builds into testing matrix?

Happy to answer your questions!

Thanks,
Almaz

Author:	Yqwed
Assignees:	-
Labels:	area-System.Globalization, untriaged
Milestone:	-

Tagging subscribers to 'arch-android': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This is a spin-off from mono/mono#21682, as @mdh1418 suggested to do.

tl;dr:

1. As I understand, current implementation reads Android's tzdata directly. Here are few problems with that: neither its location nor its structure is documented or guaranteed. I don't know what .NET's API surface is, but maybe ICU4C would suffice. Overall it endangers users as PR above illustrates.
2. How does current .NET implementation handles TZif4 files?
3. Do you have tests like this? I think old Mono implementation would fail on Asia/Gaza as it has predefined transition up to 2086 (does not fit into 32-bit section of TZif ver. 1)
4. We usually make tzdata changes early in the release cycle. Can you include Android beta builds into testing matrix?

Happy to answer your questions!

Thanks,
Almaz

Author:	Yqwed
Assignees:	-
Labels:	area-System.Globalization, os-android, untriaged
Milestone:	-

[steveisok]

Appreciate the feedback! I'll try to answer your questions to the best of my knowledge.

1. As I understand, current implementation reads Android's tzdata directly. Here are few problems with that: neither its location nor its structure is documented or guaranteed. I don't know what .NET's API surface is, but maybe ICU4C would suffice. Overall it endangers users as PR above illustrates.

Up until this point, we've felt that was an acceptable risk reading from tzdata as it has the least impact on startup performance and I don't believe ICU4C covers everything. As an alternative, we could call the java API's via JNI, but I would like to avoid that unless it's absolutely necessary.

2. How does current .NET implementation handles TZif4 files?

It appears we only support up to v3 and we will fall back to v1 in the case of a unsupported version

runtime/src/libraries/System.Private.CoreLib/src/System/TimeZoneInfo.Unix.cs

Lines 1324 to 1327 in a5461cb

	Version =
	version == '2' ? TZVersion.V2 :
	version == '3' ? TZVersion.V3 :
	TZVersion.V1; // default/fallback to V1 to guard against future, unsupported version numbers

@tarekgh are there any plans on supporting v4?

3. Do you have tests like this? I think old Mono implementation would fail on Asia/Gaza as it has predefined transition up to 2086 (does not fit into 32-bit section of TZif ver. 1)

The way dotnet/runtime parses tzdata appears to be a bit more robust than what we had in mono/mono. I'm not sure if Asia/Gaza is part of our test cases, but it would be interesting to try it and see if there's an issue.

Good question re: the TimeApisConsistencyTest test. @tarekgh, would you happen to know if we have an equivalent.

4. We usually make tzdata changes early in the release cycle. Can you include Android beta builds into testing matrix?

Yes, we have the capability of adding different emulators to our CI. This is something we should do more aggressively.

[Yqwed]

Thanks for quick answer!

Up until this point, we've felt that was an acceptable risk reading from tzdata as it has the least impact on startup performance and I don't believe ICU4C covers everything. As an alternative, we could call the java API's via JNI, but I would like to avoid that unless it's absolutely necessary.

Yeah, I understand why you took that path :) Like Java APIs, do you also need to know information about past and upcoming transitions?

@tarekgh are there any plans on supporting v4?

It is available as a draft only, but AFAIU current zic can produce TZif4 files. But we don't support it either. I think it's better to be vocal about such things, because otherwise it will take a while to find the root cause.

I don't know whether it is a good idea to fallback to V1 - it has 32-bit data only and slim format skips it entirely, producing valid TZif file (that's exactly what happened in mono/mono#21682). And even if it has 32-bit data we are getting closer to 2038, were it may not work correctly.

Yes, we have the capability of adding different emulators to our CI. This is something we should do more aggressively.

SGTM!

Tagging subscribers to this area: @dotnet/area-system-datetime
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This is a spin-off from mono/mono#21682, as @mdh1418 suggested to do.

tl;dr:

1. As I understand, current implementation reads Android's tzdata directly. Here are few problems with that: neither its location nor its structure is documented or guaranteed. I don't know what .NET's API surface is, but maybe ICU4C would suffice. Overall it endangers users as PR above illustrates.
2. How does current .NET implementation handles TZif4 files?
3. Do you have tests like this? I think old Mono implementation would fail on Asia/Gaza as it has predefined transition up to 2086 (does not fit into 32-bit section of TZif ver. 1)
4. We usually make tzdata changes early in the release cycle. Can you include Android beta builds into testing matrix?

Happy to answer your questions!

Thanks,
Almaz

Author:	Yqwed
Assignees:	-
Labels:	os-android, area-System.DateTime
Milestone:	-

[tarekgh]

It is available as a draft only, but AFAIU current zic can produce TZif4 files. But we don't support it either. I think it's better to be vocal about such things, because otherwise it will take a while to find the root cause.

I am seeing the draft is expired and it is not clear to me if this is approved spec now or just abandoned? It is not a good idea to take dependency on such expired draft till we have confirmation it is going to be supported.

Good question re: the TimeApisConsistencyTest test. @tarekgh Tarek Mahmoud Sayed FTE, would you happen to know if we have an equivalent.

This test according to the comments there is testing different Java implementation when using ICU or when bionic. This is irrelevant to .NET as we don't use ICU anyway and only depend on reading the tzdata. We treat Asia/Gaza as any other time zone and not special casing it in any way.

On Android, java.time is implemented using ICU4J, bionic and java.util.TimeZone read transitions
 * from tzdata file. These tests ensure that they have consistent view on what offset was on a given
 * time zone at a given time.

[Yqwed]

I am seeing the draft is expired and it is not clear to me if this is approved spec now or just abandoned? It is not a good idea to take dependency on such expired draft till we have confirmation it is going to be supported.

Yes. What I am saying is that in the wild you might observe it with the relatively recent zic version.

This test according to the comments there is testing different Java implementation when using ICU or when bionic. This is irrelevant to .NET as we don't use ICU anyway and only depend on reading the tzdata. We treat Asia/Gaza as any other time zone and not special casing it in any way.

What I am suggesting is you can compare libc vs .NET. It might have caught mono/mono#21682 if dates outside of 2038 were tested. I am mentioning Asia/Gaza because it observes DST, but due to the way transitions are set they are pregenerated and there is no TZ string in its TZif files.