Add xcstrings support

[fthdgn]

I did a simple development for xcstrings support.

It is backward compatible for for .xcstrings files converted from .strings files. It means the same R.string codes will be generated.

However, it is not backward compatible for .xcstrings files converted from .stringsdict files, because of named arguments.

About named arguments

My implementation strips names of substitutions from generated arguments. I found using them problematic on some cases.

In my opinion, named arguments does not worth the implementation. Xcode generates substituons only if there are more than one pluralable parameter on new xcstrings files. The most of the string values will not be have any information to generate named arguments.

Problematic case 1

Some key information is lost on the xcstring convertion of stringsdict files.
Original stringsdict content.

	<key>x_users</key>
	<dict>
		<key>NSStringLocalizedFormatKey</key>
		<string>%#@users@</string>
		<key>users</key>
		<dict>
			<key>NSStringFormatSpecTypeKey</key>
			<string>NSStringPluralRuleType</string>
			<key>NSStringFormatValueTypeKey</key>
			<string>d</string>
			<key>one</key>
			<string>%d user</string>
			<key>other</key>
			<string>%d users</string>
		</dict>
	</dict>

Generated xcstrings content

{
  "x_users" : {
    "localizations" : {
      "en" : {
        "variations" : {
          "plural" : {
            "one" : {"stringUnit" : {"value" : "%d user"}},
            "other" : {"stringUnit" : {"value" : "%d users"}
}}}}}}}

R.swift creates .x_users(users: Int) for stringsdict file. However, xcstrings file does not have the argument name information any more.

If string value of the original content is <string>Add %#@users@</string>, generated xcstrings would keep the substitution information.

Problematic case 2

{
  "example": {
    "localizations": {
      "en": {
        "substitutions": {
          "device_iphone": {
            "argNum": 1,
            "formatSpecifier": "lld",
            "variations": {
              "plural": {
                "one": {"stringUnit": {"value": "%arg iPhone"}},
                "other": {"stringUnit": {"value": "%arg iPhones"}}
              }}},
          "device_mac": {
            "argNum": 1,
            "formatSpecifier": "lld",
            "variations": {
              "plural": {
                "one": {"stringUnit": {"value": "%arg Mac"}},
                "other": {"stringUnit": {"value": "%arg Macs"}}
              }}},
          "input_iphone": {
            "argNum": 2,
            "formatSpecifier": "lld",
            "variations": {
              "plural": {
                "one": {"stringUnit": {"value": "%arg touch"}},
                "other": {"stringUnit": {"value": "%arg touches"}}
              }}},
          "input_mac": {
            "argNum": 2,
            "formatSpecifier": "lld",
            "variations": {
              "plural": {
                "one": {"stringUnit": {"value": "%arg key"}},
                "other": {"stringUnit": {"value": "%arg keys"}}
              }}},
        "variations": {
          "device": {
            "iphone": {
              "stringUnit": {
                "value": "%#@device_iphone@ and %#@input_iphone@"
              }},
            "mac": {
              "stringUnit": {
                "value": "%#@device_mac@ and %#@input_mac@"
              }}}}}}}}

This strings value is "1 iPhone and 10 touches" on iPhones and "1 Mac and 10 keys" on Mac.
There are 2 substitutions for each device variation.
Which signature should we use?
.example(device_iphone: Int, input_iphone: Int)
.example(device_mac: Int, input_mac: Int)
.example(_ arg1: Int, _ arg2: Int)
.example(device Int, input: Int) //With some extra coding to detect shared parts of the names.

About algorithm

The alghorithm tries to convert localization of source language to single string with basic format parameters, then it uses FormatPart.formatParts(formatString:)` on this string to extract parameters.

This convertion works like that:

• If localization has stringUnit, get its value, and replace the substitutions inside if.
• Else if localization has variations for device or plural, convert each variation value to string with the same algorithm and select the string with most parameters. (To work correctly, all variations should have same parameters, but on some cases such as plural rule one or zero, parameter may not be used)

Substitutions replacement works like that:

• Substitutions generated by Xcode contain plural variations with %arg parameter. Each substitutions has argNum and formatSpecifier variable to define real properties of this %arg parameter.
• The value of a substitution is calculated by the convertion algorith above.
• Then, %arg parameters on the calculated value is replaced according to argNum and formatSpecifier values.

Example 1

  "account": {
    "localizations": {
      "en": {
        "stringUnit": {"value": "Account"}
      }}}

• Localization has stringUnit, get its value.
• Localization does not have any substitutions. Nothing replaced.
• Get paramters of "Account". []

Example 2

  "x_sets": {
    "localizations": {
      "en": {
        "variations": {
          "plural": {
            "one": {"stringUnit": {"value": "a set"}},
            "other": {"stringUnit": {"value": "%d sets"}}
          }}}}}

• Localization does not have stringUnit.
• Localization does not have device variations.
• Localization has plural variations. Get value of each variations with parameter count.
  • one: Value: "a set", parameters []
  • other: Value: "%d sets", parameters [%d]
• Select the variation with most parameters. "%d sets"
• Get paramters of "%d sets". [%d]

Example 3

{
  "example": {
    "localizations": {
      "en": {
        "stringUnit": {
          "value": "%#@books@ and %#@pens@"
        },
        "substitutions": {
          "books": {
            "argNum": 1,
            "formatSpecifier": "lld",
            "variations": {
              "plural": {
                "one": {"stringUnit": {"value": "%arg book"}},
                "other": {"stringUnit": {"value": "%arg books"}}
              }}},
          "pens": {
            "argNum": 2,
            "formatSpecifier": "lld",
            "variations": {
              "plural": {
                "one": {"stringUnit": {"value": "%arg pen"}},
                "other": {"stringUnit": {"value": "%arg pens"}}
              }}}}}}}}

• Localization has stringUnit.
• Value is %#@books@ and %#@pens@
• Localization has substitutions.
• For "books" substitution
• Get the value like Example 2: %arg book
• Replace %arg according to argNum and formatSpecifier: %1$lld book
• Replace substitution on the value: %1$lld book and %#@pens@
• For "pens" substitution
• Get the value like Example 2: %arg pen
• Replace %arg according to argNum and formatSpecifier: %2$lld pen
• Replace substitution on the value: %1$lld book and%2$lld pen
• Get paramters of %1$lld book and %2$lld pen. [%lld, %lld]

[Beer-Koala]

It algorithm has a problem if in xcstrings file have few diffeerents formats. For example:

{
"sourceLanguage" : "en",
"strings" : {
"continue" : {
"extractionState" : "manual",
"localizations" : {
"en" : {
"stringUnit" : {
"state" : "translated",
"value" : "Continue"
}
}
}
},
"Diagnose" : {
"extractionState" : "manual"
},
"Explore" : {
"extractionState" : "manual"
},
"Garden" : {
"extractionState" : "manual"
},
"Home" : {
"extractionState" : "manual"
},
"noLocationLabel" : {
"extractionState" : "manual",
"localizations" : {
"en" : {
"stringUnit" : {
"state" : "translated",
"value" : "Get customized planting plan for your zone"
}
}
}
},
......

[Flatout73]

Any updates on this? Let's merge mb?

[Flatout73]

Bump

[fthdgn]

Sorry I couldn't read this PR for a long time.

@Beer-Koala On my code I assumed that each key to have a localizations object, which contains a key for "sourceLanguage". However, my assumption is wrong. Just after a new key is created, it does not have any localization object until a value is entered. I am using my own breach since November, and I didn't encounter this problem, because I immediately enter an English value. I will update my branch.

[ulkomar]

Any updates about merging this PR?

[fthdgn]

I couldn't update source branch of this PR bacause it has references on my own apps.

I created a new PR #886

Whats new on #886

• Rebased to version 7.5.0.
• Fixed error on keys without values