`Cldr.Number.to_string` performs expensive `validate_options`

[tomtaylor]

I've been tracing a hot path in our app, and found a surprisingly expensive call to Cldr.Number.to_string. A call to this is dominated by Cldr.Number.Format.Options:validate_options/3, specifically, a call to [Backend].Cldr.Number:default_options/0 calling Cldr:get_locale/1 in turn. I've attached a flame graph you can open in https://www.speedscope.app/

1774277542281657-eflambe-output.bggg.zip

I think this could be solved by lazily evaluating the default options on a per-key basis. Do you have any particular preference about how to approach this?

[kipcole9]

@tomtaylor thanks for the report. Yes, it certainly could be better. I will have a chance to look at the flame graph soon. In the mean time, a couple of thoughts:

1. If you're passing the locale as anything but a Cldr.LanguageTag.t then there is a full parse of the locale ID and it's not cheap. Ideally you would do a Cldr.validate_locale/2 at the boundary of your app and only pass the result of that around. That doesn't fully solve it, but it helps a lot.

2. Secondly, if you're doing repeated formatting with the same options (like you would be a table of a list) then you would desirably call Cldr.Number.Format.Options.validate_options/1 on your keyword list. That returns an options struct which you can pass as options to Cldr.Number..to_string/2.

I was just working on exactly this issue today in the localise implementation (ex_cldr version 3, alpha preview in a couple of weeks). I know options parsing can be improved - probably by re-implementing. There is a a lot going on to resolve the format under the covers and it is definitely a hot path.

[kipcole9]

My preference is a re-implementation of the options module using a more declarative approach. Which requires that I formalise the options specification which can only be a good thing. I'm not sure that will radically change things but it's worth the effort. Always very happy to receive suggestions, help and advice.

[tomtaylor]

Thanks @kipcole9, makes sense. I wonder if in the short-term this could be fixed by using Keyword.get_lazy to evaluate the default options, avoiding an expensive call to Cldr.get_locale when merging the maps?

[kipcole9]

voiding an expensive call to Cldr.get_locale

This is the bit that perplexes me because typically all get_locale/0 does is Process.get/1.

Is there any code calling Cldr.put_locale/1 prior to number formatting. ie From a plug if it's a Phoenix app?
Any chance you can share your backend module configuration?

[kipcole9]

@tomtaylor, I have back-ported the much improved locale-matching code from localize to ex_cldr_numbers. I believe if you upgrade to ex_cldr version 2.47.4 you'll see substantial improvement in the situation you outline.

If not, I will certainly take a look. I recognise that options handling being such a large amount of the total formatting time is surprising. However I hope the balance is now restored. I'll hold this open for a while to await your feedback.