Skip to content

front: extract i18n keys with TypeScript #11554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: dev
Choose a base branch
from
Open

Conversation

emersion
Copy link
Member

@emersion emersion commented Apr 22, 2025

i18next-parser uses the TypeScript compiler only for lexing. In
other words, it has no contextual information about the translation
calls. This makes extracting translation keys more fragile and
error-prone. For instance, when TFunctions are passed around, the
namespace is lost. As another example, "i18n.t" is not recognized
as a translation function. In general, this is a fool's errand:
operating on tokens without type information will always result in
a lot of unhandled corner cases.

Instead, use the TypeScript compiler API to extract type information
and iterate over all calls to the translation function.

(This should probably be extracted into a re-usable NPM package.)

See individual commits.

Typescript compiler API debugging notes
  //console.log(node.expression.escapedText);
  //console.log(checker.typeToString(type));
  //console.log(type.symbol?.escapedName);

  //if (node.expression.escapedText !== 't') {
  //  console.log('A', file.fileName, file.getLineAndCharacterOfPosition(node.pos));
  //}

    //console.log(checker.typeToString(namespaceType));

      //console.log(namespaceType.resolvedTypeArguments.map(checker.typeToString));
      //console.log(ts.Debug.formatSyntaxKind(symbol.valueDeclaration.kind));

    const symbol = checker.getSymbolAtLocation(keyNode);
    const type = symbol ? checker.getTypeOfSymbol(symbol) : null;
    if (!ts.isTemplateExpression(keyNode)) {
      //console.log('E', file.fileName, file.getLineAndCharacterOfPosition(node.pos));
      //console.log(ts.Debug.formatSyntaxKind(keyNode.kind));
      //console.log(type);
    }

@github-actions github-actions bot added the area:front Work on Standard OSRD Interface modules label Apr 22, 2025
@emersion emersion self-assigned this Apr 22, 2025
@emersion emersion changed the title front: extract keys with TypeScript front: extract i18n keys with TypeScript Apr 22, 2025
@emersion emersion force-pushed the emr/ts-i18next-extract branch from cc66535 to 3913537 Compare April 23, 2025 16:04
@emersion emersion requested review from Synar and clarani April 23, 2025 16:06
@emersion emersion force-pushed the emr/ts-i18next-extract branch from 3913537 to 8fdc600 Compare April 23, 2025 16:11
@emersion emersion marked this pull request as ready for review April 23, 2025 16:13
@emersion emersion requested a review from a team as a code owner April 23, 2025 16:13
@emersion emersion force-pushed the emr/ts-i18next-extract branch 2 times, most recently from 0758246 to a20acb6 Compare April 24, 2025 15:09
@emersion
Copy link
Member Author

For what it's worth, this has been extracted into https://github.com/emersion/i18next-typescript-parser

Copy link
Contributor

@Synar Synar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Reviewed all commits but the script one. I still need some time to completely understand that one ^^''.


import { glob } from 'glob';
// TODO : remove eslint-disable once https://github.com/i18next/i18next-parser/issues/1000 gets fixed
// eslint-disable-next-line import/no-unresolved
import * as I18nextParser from 'i18next-parser';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we drop i18next-parser from package.json?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Done.

import vfs from 'vinyl-fs';
import * as ts from 'typescript';

const LANGUAGES = ['en', 'fr'];

const IGNORE_MISSING: RegExp[] = [
// key used by a t function in modules/trainschedule/components/ManageTrainSchedule/helpers/checkCurrentConfig.ts
/translation:errorMessages\..*/,
/translation:error/,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw you removed t('error') at the profit of t('default') in one commit, is this still used somewhere else?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, this can be dropped.

/translation:unspecified/,
// key used by upsertMapWaypointsInOperationalPoints
/translation:requestedPoint/,
// key used by checkStdcmConfigErrors
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove this comment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, done!

emersion added 13 commits April 29, 2025 13:56
Ensures our i18n-checker script can recognize that translations
accessed from inside this function are in use.

Signed-off-by: Simon Ser <[email protected]>
Otherwise translations aren't live-updated when switching the
language.

Signed-off-by: Simon Ser <[email protected]>
We don't need to interpolate variables here. Makes it easier to
statically extract the constant string in the i18n checker script.

Signed-off-by: Simon Ser <[email protected]>
Helps with static analysis in the i18next checker script.

Signed-off-by: Simon Ser <[email protected]>
'error' is not a valid translation key, it doesn't exist. Use
'default' just like getErrorMessage(), which exists.

Signed-off-by: Simon Ser <[email protected]>
We don't need to pass multiple of these.

Signed-off-by: Simon Ser <[email protected]>
i18next-parser uses the TypeScript compiler only for lexing. In
other words, it has no contextual information about the translation
calls. This makes extracting translation keys more fragile and
error-prone. For instance, when TFunctions are passed around, the
namespace is lost. As another example, "i18n.t" is not recognized
as a translation function. In general, this is a fool's errand:
operating on tokens without type information will always result in
a lot of unhandled corner cases.

Instead, use the TypeScript compiler API to extract type information
and iterate over all calls to the translation function.

Signed-off-by: Simon Ser <[email protected]>
These reference non-existing keys or are unnecessary.

Signed-off-by: Simon Ser <[email protected]>
This is a hook, we can just call useTranslation() there.

Signed-off-by: Simon Ser <[email protected]>
Ensure we don't have unused keys in translation files, to reduce
the workfload for community translators and keep our files neat
and tidy.

Signed-off-by: Simon Ser <[email protected]>
@emersion emersion force-pushed the emr/ts-i18next-extract branch from a20acb6 to c89a934 Compare April 29, 2025 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:front Work on Standard OSRD Interface modules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants