Caching support in Phrase CLI

[realdadfish]

We integrated PhraseCLI in our CI pipeline and stumble upon the known rate limiting issues. The Phrase Translate API seems to offer ETag support to hit Phrase's servers with fewer requests, however I don't see an option in the CLI for that. Is there any?

[jablan]

Hey Thomas Keller (@realdadfish) we haven't had such a request so far (and caching would be pretty big feature to implement). Can you elaborate in which way are you using the CLI so that it would benefit from caching? Why does it perform the same requests multiple times?

[jablan]

I presume you are aware of our OTA feature? https://support.phrase.com/hc/en-us/articles/5804059067804-Over-the-Air-Strings

[realdadfish]

Well, right now we automated the process of pulling strings from Phrase in our CI pipeline and also created a small check for it whether there are updates to strings that are not checked in. Reason for that is that we want a developer to look over the string changes before they're actually checked in, because things like missing / malformed placeholders and the like are not automatically detected by Phrase on input. Also, sometimes strings might not fit inside the UI and there needs to be extra communication between the translator and the developer.

All in all this automated process uses Phrase CLI and is executed as part of a bigger pipeline and several times a day by the developer on many different Gradle modules (many in parallel), because also other checks run afterwards (like linting the translated sources), and more often than not we run into the problem that we get the "too many requests" issue. Now an obvious (but work-heavy) way on our side would be to skip using the CLI altogether and manually call the Phrase API, but then we're basically rebuilding a client that is already there.

I also think people using the CLI would greatly benefit from a caching implementation, because repeated pulling of strings would not count towards the request limit and in effect lessen the burden of Phrase's infrastructure.

[jablan]

I see. I will open an internal feature request ticket to implement CLI pull caching, but I don't think we could tackle it any time soon.

I know your workflow could be wildly different from ours, but for us, checking for new translations once per day centrally works. Could you perhaps decouple it from your standard CI pipeline and perform it independently less frequently?

[realdadfish]

I see. I will open an internal feature request ticket to implement CLI pull caching, but I don't think we could tackle it any time soon.

Hrm... ok. Thanks!

I know your workflow could be wildly different from ours, but for us, checking for new translations once per day centrally works. Could you perhaps decouple it from your standard CI pipeline and perform it independently less frequently?

It's not so much that it happens so many times of the day, but rather that things happen in parallel. We have dozens, soon hundreds of different Gradle modules containing translations for UI components. These need translations from a dozen different Phrase projects, each again subdivides its strings via tags. Naturally each module takes care of it's own translations (divide and conquer), so at some point dozens of parallel requests might come up.

I don't really want to change that implementation because of some rate-limiting API where I even have to track the rate-limit part myself (rate-limiting IMHO is nowhere else seen these days, especially not that kind of harsh limiting, especially for a paid service even). Rather I'd deploy some smart caching mechanisms and that's why I asked for the above.

[tetienne]

Hello, I just created this PR to address this question. The improvement is not so impressive, but it avoid to be rate limited.