Improving consistency of glossaries across languages

[hturner]

The glossary used on Weblate isn't consistent across languages. E.g.

• Polish: https://translate.rx.studio/projects/r-project/glossary/pl/ - 7 strings
• Catalan: https://translate.rx.studio/projects/r-project/glossary/ca/ - 9 strings
• Brazilian Portuguese: https://translate.rx.studio/projects/r-project/glossary/pt_BR/ - 57 strings!

For more consistency we could do the following:

• Create a combined list of terms by reviewing the glossary for each language and possibly adding further terms. Check the terms are actually frequently used in the source strings by using the Weblate search (https://docs.weblate.org/en/latest/user/search.html).
• Contact the team leaders to share the combined glossary and ask if they would be happy for the glossary to be managed globally, so that the glossary is common across languages and new terms can only be added by the Weblate admins. Any translator could suggest a new term by contacting the team leader or posting on the #core-translation channel of the R Contributors Slack.

Note: only English skills are required for this task!

[SaranjeetKaur]

Attaching a combined glossary of from Weblate as of now: glossary_combined.csv

[SaranjeetKaur]

Code snippet for generating glossary_combined.csv:

library(magrittr)
library(tidyverse)

# language_stat <- read.csv("stats-r-project-glossary.csv")
# View(language_stat)
# files <- list.files('language/', pattern = ".csv$", recursive = TRUE, full.names = TRUE)
# glossary_merged <- read_csv(files, show_col_types = FALSE) %>% bind_rows()
# View(glossary_merged)

filenames_language <- list.files(path = "language/", pattern = ".csv", all.files = FALSE, full.names = FALSE)
# View(filenames_language)

myfiles = lapply(paste0("language/", filenames_language), read.delim, sep = ",")  
for (i in 1:length(filenames_language)){
  colnames(myfiles[[i]])[which(names(myfiles[[i]]) == "target")] <- paste0("target", str_sub(filenames_language[i], 19, -5))
}

for (i in 1:length(filenames_language)){
  myfiles[[i]] <- subset(myfiles[[i]], select = c(2, 3))
}

glossary_combined <- Reduce(function(x,y) merge(x, y, by = "source", all.x = TRUE, all.y = TRUE), myfiles)
write.csv(glossary_combined, "glossary_combined.csv")

[SaranjeetKaur]

The code above need to modified to clean up the glossary that it generates. Some glossary rules that were discussed with @hturner are listed below:

• Avoid using plurals for glossary terms, singular terms are preferred
• Use small letters for the glossary terms unless proper noun
• Avoid big sentences
• Conduct sanity checks to ensure relevant terms are included in the glossary

[daroczig]

What we have discussed with @hturner @lawremi and @azeloc at the R Dev Day PLUS:

• we follow up on the great work started by @SaranjeetKaur above to have a synchronized table of the terms from the language-specific glossaries
• @azeloc and @bjungbogati are working on cleaning up the synchronized list
• We import the cleaned-up list into the English glossary in Weblate
• Create group / add users (who to add?) with permissions to manage the English glossary in Weblate, and keep it updated based on feedback from the language teams
• Update the translation guide so that folks use the English glossary as a starting point

[azeloc]

Another version of SaranjeetKur script to build a unified list of terms: https://gist.github.com/azeloc/07d5ab53c97226862d55b8a33b91dc97

Based on this list we built a synchronized list of terms for the "template" glossary

General principles on building the synchronized list, based on the translation guidelines conventions https://contributor.r-project.org/translations/Conventions_for_Languages/

• (This is not in the guidelines, but seems useful to define as a principle for the glossaries) Keep only infinitive verbs. e.g. the "coerce" and "coerced" should not be different terms. "coerce" is the preffered term
• Technical acronyms/terms such as file extensions, language names, packages names should not be translated etc. e.g. xz, HTML
• Some large expressions were kept on the final synchronized list as it might help on automatic translations by weblate

sample removals:

• coerced
• coercion
• dropped
• dropping
• embedded
• embeddings
• HTML
• xz
• BibTex

[bjungbogati]

Below is the list of terms extracted from the above code.

lots <- readxl::read_excel("list_of_terms.xlsx")

lots_source <- lots %>% filter(
  !source %in% c("coerced", "coercion", "dropped", "dropping", "embedding", "embeddings",
              "HTML", "xz", "BibTeX", "bzip2", "PDF", "LaTeX", "users", "methods dispatch", "lags",
              "bytecode",
              "invalid 'name' argument",
              "invalid value of %s",
              "file '%s' not found",
              "dim(X) must have a positive length",
              "argument to 'La.svd' must be numeric or complex",
              'See help("Deprecated")',
              'See help("Defunct") and help("%s-defunct").',
              'See help("Deprecated") and help("%s-deprecated").',
              "'%s'",
              "'%s' is not a valid choice",
              "'MARGIN' does not match dim(X)",
              "'coef' must not be negative"
              )
) %>%
  mutate(source = str_remove(source, "version string"), "") %>%
  filter(is.na(plural_flag)) %>%
  select(source)

write_csv(lots_source, "list_of_glossary_terms.csv")

list_of_terms.xlsx

list_of_glossary_terms.csv

[bjungbogati]

The above list of glossaries are updated in Weblate glossaries marked as terminology.
Thanks @azeloc and @daroczig for feedbacks and spot checking.

[hturner]

The second point about communication with translation team leads was large covered by a PR to the translations documentation (r-devel/translations#41). It would be nice to have this PR completed before closing this issue.