Ensure that Symfony Translation always works reliably with Twig templates

[miguelvaara]

Reasons for creating this PR

Symfony extract did not work with Twig templates

Link to relevant issue(s), if any

• Closes Ensure that Symfony Translation always works reliably with Twig templates #1907

Description of the changes in this PR

Added a paths setting to the configuration

How to review

Edit: src/view/about.twig

After
<h1 class="mb-5">{{"About" | trans }}</h1>
add the following:
<p>{{ "This is a test translation key 1"|trans }}</p>
and save

Use
bin/console debug:translation en | grep -i "test translation"
to see that the key is used in the code but missing from the translation files.

Extract:
bin/extract-translations

Re-run the following
bin/console debug:translation en | grep -i "test translation"
to see that it is not missing anymore

Verify:
grep 'This is a test translation key' resource/translations/messages.en.json

Known problems or uncertainties in this PR

Checklist

• phpUnit tests pass locally with my changes
• I have added tests that show that the new code works, or tests are not relevant for this PR (e.g. only HTML/CSS changes)
• The PR doesn't reduce accessibility of the front-end code (e.g. tab focus, scaling to different resolutions, use of .sr-only class, color contrast)
• The PR doesn't introduce unintended code changes (e.g. empty lines or useless reindentation)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.32%. Comparing base (ff07932) to head (2a20ee5).
⚠️ Report is 38 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1908      +/-   ##
============================================
- Coverage     70.38%   70.32%   -0.07%     
  Complexity     1658     1658              
============================================
  Files            34       34              
  Lines          4356     4364       +8     
============================================
+ Hits           3066     3069       +3     
- Misses         1290     1295       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[osma]

In my view, the problem is not that extraction from Twig templates doesn't work at all but that the extraction is unreliable. It works for some Twig templates but not for others, and I can't figure out what makes the difference.

If I run the command

bin/console debug:translation en

it shows a number of messages as unused, including these:

   unused    messages   Download this vocabulary:                                                                                 Download this vocabulary:                 
   unused    messages   Vocabularies                                                                                              Vocabularies       

yet they are clearly used in Twig templates:

src/view$ grep 'Download this vocabulary' *
vocab-info.inc.twig:          <h2><i class="fa-solid fa-download"></i> {{ "Download this vocabulary:" | trans }}</h2>

src/view$ grep Vocabularies *
base-template.twig:                   <i class="fa-solid fa-house"></i>&nbsp;{{ "Vocabularies" | trans }}
landing.twig:      <h2 class="fs-3 fw-bold pb-3">{{ "Vocabularies" | trans }}</h2>

This PR makes no difference to the behavior. These two messages, and many others, are still listed as "unused" even though they clearly are being used and translated in the Twig templates.

[miguelvaara]

How to verify Symfony Translation limitations

Skosmos uses custom Twig filters (global_url, link_url) and a "non-standard" template structure: src/view/ instead of templates/. While bin/console translation:extract correctly finds all translations, bin/console debug:translation produces false "unused" warnings due to known Symfony TwigExtractor limitations.

GitHub: symfony/symfony#29085

@xabbuh (Symfony Member) on Nov 5, 2018:
Unfortunately, that's expected. As the docs state the command is only able to find translation usages in templates.

@nicolas-grekas (Symfony Member) on Nov 6, 2018:
Until now yes. In theory it could be possible to extract more messages by analyzing the code, but someone would need to contribute the code :)

These are not configuration errors. They are architectural limitations in Symfony's translation tools (at least in the version 6.4 we are using).

The following tests you can follow/repeat aims to prove limitations:

1. Custom Twig Extensions

TwigExtractor fails to parse templates containing custom filter expressions skipping all subsequent translations.

### Create test templates
cat > src/view/test-simple.twig << 'EOF'
<h1>{{ "Translation A" | trans }}</h1>
<a href="/link">{{ "Translation B" | trans }}</a>
<p>{{ "Translation C" | trans }}</p>
EOF

cat > src/view/test-complex.twig << 'EOF'
<h1>{{ "Translation X" | trans }}</h1>
<a href="{{ 'home' | global_url(request.lang, request.contentLang, null, false) }}">{{ "Translation Y" | trans }}</a>
<p>{{ "Translation Z" | trans }}</p>
EOF

Extract translations

bin/console translation:extract --dump-messages en 2>&1 | grep "Translation [A-Z]"

Simple template: All 3 translations found (A, B, C)
Complex template: 0 translations found (X, Y, Z missing)

rm -f src/view/test-simple.twig src/view/test-complex.twig

2. Template Inheritance + Custom Filters

Debug:translation fails to detect translation usage in templates with custom filters showing false "unused" warnings.

Find templates using inheritance

grep -l "{% extends" src/view/*.twig

Verify base-template.twig has custom filters and translations

grep -c "global_url" src/view/base-template.twig
grep -c "| trans" src/view/base-template.twig

Confirm extraction works (finds all translations)

bin/console translation:extract --dump-messages en 2>&1 | grep -E "(Vocabularies|About|Feedback)"

But debug:translation shows false "unused" warnings

bin/console debug:translation en --only-unused 2>&1 | grep -E "(Vocabularies|About|Feedback)"

Expected result:

• 8 child templates extend base-template.twig
• base-template.twig has 5 global_url + 3 trans filters
  - translation:extract finds: "About", "Feedback", "Vocabularies" OK
• debug:translation marks some as "unused" despite being in templates FAILS

Why?
translation:extract works correctly, but debug:translation maybe(?) uses a different parser that fails on custom filters. This creates false "unused" warnings for translations that are actually used in templates.

3. Non-Standard Template Structure

Symfony propably expects templates directory but Skosmos uses src/view. Extraction requires explicit configuration.

! This could not be verified for 100 precent sure but the custom filters are still a problem.

Check current configuration

cat config/packages/twig.yaml

Create test in "standard" location

mkdir -p templates
echo '<h1>{{ "Test Standard" | trans }}</h1>' > templates/test.twig

Before modifications:

twig:
    default_path: '%kernel.project_dir%/src/view'
    file_name_pattern: ['*.twig']

Attempt extraction

bin/console translation:extract --dump-messages en 2>&1 | grep "Test Standard"

Result: templates/ directory not scanned, 0 translations found

After the modifications:

Edit the config/twig.yaml (add the "paths")

twig:
    default_path: '%kernel.project_dir%/src/view'
    file_name_pattern: ['*.twig']
    paths:
        '%kernel.project_dir%/templates': templates

Re-attempt extraction

bin/console translation:extract --dump-messages en 2>&1 | grep "Test Standard"

templates directory scanned, translations found OK

Explanation: Templates outside the "Symfony bundle" require explicit paths configuration. Maybe: Even with configuration, debug:translation has hardcoded assumptions about bundle structures.

Does standard structure fix custom filter issues?

Test if standard location + paths config helps with custom filters:

Create template in standard location with custom filter

cat > templates/test-filter.twig << 'EOF'
<h1>{{ "Before Custom Filter" | trans }}</h1>
<a href="{{ 'home' | global_url(request.lang, request.contentLang, null, false) }}">Link</a>
<p>{{ "After Custom Filter" | trans }}</p>
EOF

Extract with paths configured

bin/console translation:extract --dump-messages en 2>&1 | grep -E "(Before Custom|After Custom)"

Expected result: Still 0 translations found

Re-create template in standard location withOut custom filter

cat > templates/test-filter.twig << 'EOF'
<h1>{{ "Before Custom Filter" | trans }}</h1>
<p>{{ "After Custom Filter" | trans }}</p>                                                  
EOF 

Extract

bin/console translation:extract --dump-messages en 2>&1 | grep -E "(Before Custom|After Custom)"

WHY?: Custom filter parse failures occur regardless of template location. Standard structure with paths config does bot fix the custom filter limitation - they are separate issues.

Cleanup:

rm -rf templates
git restore config/packages/twig.yaml

Summary

All three limitations were at least partly verified (custom filter case 100 percently):

1. Custom Twig Extensions -> 0 percent extraction rate with when complex filters
2. Template Inheritance -> False positives when combined with custom filters (incorrectly reports a translation as "unused" when it is actually used in a template)
3. Non-Standard Structure -> Requires explicit paths, still has "edge cases"

Summa summarum

• translation:extract works correctly and finds all translations
• debug:translation fails to detect usage in templates with custom filters
• Adding paths configuration maybe helps scan multiple directories but does not fix custom filter parse errors
• The issues are independent: fixing template location does not fix custom filter parsing

Conclusion

The false "unused" warnings in debug:translation are almost certainly caused by Symfony TwigExtractor limitations, not Skosmos configuration issues. No configuration changes will fix the custom filter parsing problem becayse it likely is an architectural limitation in Symfony's template parser or caused by some hidden and silent logic.

The recommended approach: rely on translation:extract for accurate extraction and ignore false positives from debug:translation -> and document it clearly to the community.