ZonedDateTime not isbits

[Keno]

@oxinabox and I were discussing some performance problems in the presence of large numbers of ZonedDateTime objects and I noted that them not being isbits is a problem if you put lots of them in a vector, because the GC time taken to mark such vectors will be O(number of elements) not O(number of vectors). Now, arguably this is a prime use case for a partially pooled vector that just pools the TimeZone objects, but keeps the raw DateTime inline, but I'm also wondering if ZonedDateTime itself couldn't be made to be isbits and not have this issue in the first place. In particular, it seems like there's probably only ever gonna be a small handful of TimeZones globally in the system, so we could just globally intern them and only refer to them by small integer identifiers (reconstituting them into the full object on access). Does that sound like a feasible thing to do? I think it would dramatically increase the performance of ZonedDateTime.

[omus]

I have briefly looked into this and I think it's a feasible thing to do. My main concern is how this will work with serialization specifically if a user had defined a custom time zone instead of a predefined time zone.

[oxinabox]

Could we fix that by defining custom serialization, so that it still serializes the same as it does now, even though it just stores a integery identifier both pre-serialization and after deserialization ?

[kcajf]

https://github.com/kcajf/TimeZones.jl
I have a branch that was working towards this - i.e. generate code for all the timezones, each one taking up just 16 bits (VariableTimeZoneF). I then made VariableTimeZone parametric in the timezone type. It's a bit hacky, and I never had time to fully finish it, but the proof of concept seemed to work.
See #182 for more discussion too

kcajf@4caacd3#diff-5c5b3616b976774dc6e6c77d00227ba3
This is the commit where I added most of the code generation. When the package is built, it'll write a zones.jl file with the type definitions and a few methods (each implemented as a big if/else block switching on the timezone).

[omus]

I'll be working on this next week.

[kcajf]

@omus what did you think of the idea (discussed in #182) of making ZonedDateTime parametric in the timezone type? Too much change? It would allow, for example, ZonedDateTime{UTC} to be only 8 bytes

[omus]

Interface-wise I like the idea of a parametric ZonedDateTime. Adding the type parameter is definitely breaking which we can do but need to do the experimentation to be sure it's with it. The change in #287 doesn't break the external API so it's a good alternative implementation to compare to.

[oxinabox]

I thought worth an update on some more thoughts i have had on using ShortStrings.jl
Particularly since making overloaded serialization is not as easy as it at first seems (see #323 (comment))

I did some checked, and if we represented timezone name not as a single (short) string,
but as a tuple/as three fields, after splittign the name on / then we have options.
That would be Region, Locality1, Locality2 (where the later two are optional and can be replaced with "" if not present).
In particular, all TimeZones name parts would be 15 characters or less.
Which means they would fit into UInt128, Which is much faster than larger types.

So updates: since: #287 (comment)

I've been doing a bunch of experimentation and I'll provide an overview of some of bits of knowledge I've discovered:

• BitIntegers.jl had a bug where only IOBuffer could be used with serialization (rfourquet/BitIntegers.jl#19)

Would not be a problem as UInt128 is part of Base, it doesn't need BitIntegers.jl
But that bug is now fixed

• ShortStrings.jl could only be constructed with String and not SubString{String} (JuliaString/ShortStrings.jl#13)

This has now been resolved.

• ShortStrings.jl require constructing a String again for most operations including hashing which results worse performance when combined with storing the fields externally via a Dict/Vector

This has now been resolved, pretty much all operations now don't require constructing a String anymore.
Though there is a open bug in hashing on 32bit. JuliaString/MurmurHash3.jl#12

• Attempting to make ZonedDateTime an isbitstype by making all subfields isbitstype is challenging.

  • A FixedTimeZone can be made isbitstype by using ShortStrings.jl but VariableTimeZone requires using ShortString and StaticArrays.jl (to replace the Vector{Transition}) which performed very poorly (never actually finished).
  • Updating VariableTimeZone to no longer precompute transitions (thereby eliminating Vector{Transition}) and instead use zones/rules from the tzsource may be feasible but would require a tuple of zones/rules as well as require a reference to an function

Nothing has changed with this AFAIK.

So ShortString is more viable than before, but still might not be viable.

Though it might be worth considering, is making FixedTimeZone isbits itself going to get a lot of the way to solving the practical problems with bitstypes.
That would mean if we made ZonedDateTime parameteric on the type of its timezone field then any one with a FixedTimeZone would be isbits.
And any VariableTimeZone wouldn't be isbits but the vector of transitions that it carries with in it would be a vector of isbits which are much nicers on the GC.
(niceness to the GC is the primary problem with not being isbits)