Ftr/users lcp

[jgoddard19]

🚀 Pull Request

🎯 PR Type

• Feature: Adds new functionality or user-facing change.
• Bug Fix: Solves a defect or incorrect behavior.
• Refactor: Improves code structure/readability without changing external behavior.
• Documentation: Updates to README, Wiki, or internal docs.
• Chore: Non-code changes (e.g., CI configuration, dependency bumps).

1. Issue Tracking & Reference

• Ticket Number(s): it's out there somewhere

2. Summary of Changes

LCP A/B Testing Implementation

Added two mutually exclusive LCP slowness scenarios for performance monitoring:

Backend Changes

• Scenario Service: Added two A/B testing endpoints for LCP slowness
  • /api/ab-testing/lcp-slowness-percentage - Affects configurable % of all users (hash-based cohort assignment)
  • /api/ab-testing/lcp-slowness-cohort - Affects 11 hardcoded test users only
  • Updated scenario UI to display and toggle both scenarios
• Accounts Service: Enhanced /browser-user endpoint to return lcp_delay_ms based on active scenario
  • Defined 11 test users (Alice through Smough) as slow cohort
  • Deterministic cohort assignment via MD5 hash for percentage-based scenario
  • Graceful fallback (0ms delay) when scenario service unavailable

Frontend Changes

• Root Layout: Fetch and store lcp_delay_ms in sessionStorage, set New Relic custom attributes (lcp_delay_ms, lcp_treatment)
• Dashboard: Apply delay by showing loading spinners for 3 seconds before rendering LCP elements (account balance cards)
  • Ensures actual LCP measurement captures the delay (not blocked by full-page spinner)

Testing

• 10 tests in test_ab_testing_scenarios.py (cohort-based only)
  • Validates all 11 slow users get 3000ms delay
  • Validates random users get 0ms delay
  • Tests enable/disable/reset functionality
  • Tests deterministic cohort assignment

Monitoring

• New Relic dashboards can now facet LCP metrics by lcp_treatment (slow/normal)
• Enables A/B testing impact analysis on Core Web Vitals

3. Testing

Tested via the UI

sessionStorage.setItem('browserUserId', 'f5e8d1c6-2a9b-4c3e-8f1a-6e5b0d2c9f1a'); // Bob
sessionStorage.setItem('lcpDelayMs', '3000');
location.reload();

After setting this in the console, I validated the response

Test suite run here with new tests included
https://github.com/newrelic/relibank/actions/runs/22162298427

4. Evidence

Above

Also checked with this query

  SELECT
    average(largestContentfulPaint) as 'Avg LCP (ms)',
    percentile(largestContentfulPaint, 50, 95, 99) as 'LCP Percentiles'
  FROM PageViewTiming
  WHERE appName = 'Relibank - Customer Portal'
    AND pageUrl LIKE '%/dashboard%'
    AND largestContentfulPaint > 0
  SINCE 3 hours ago

5. Review and Merge Checklist

Confirm the following prerequisites have been met by checking the boxes:

• All blocking review comments from the latest round are resolved.
• This branch has been rebased against the main branch and all merge conflicts are resolved.
• The code includes sufficient comments, particularly in complex or non-obvious areas.
• The code adheres to project coding standards (linting, style guides, best practices).
• Relevant documentation and runbooks have been updated, if necessary.
• My changes generate no new warnings or errors.
• My commits are squashed into a single, descriptive commit.
• My test results are uploaded as a text file, and new tests were created where required.

In general, to fix this class of issue, you add a permissions block either at the root of the workflow (to apply to all jobs) or inside specific jobs, and set the minimal scopes necessary. For a workflow that only calls external APIs and writes to $GITHUB_STEP_SUMMARY, you typically only need contents: read (the minimal default for most workflows) and no write permissions.

For this specific workflow in .github/workflows/flow-lcp-ab-test.yml, the simplest and least invasive fix is to add a top-level permissions block right under the name: (and before on:). The job does not perform any GitHub write actions (no actions/checkout, no pushes, no issue/PR operations), so we can safely lock the token down to contents: read. No additional imports or steps are required; this change only affects token scopes. Concretely, in the region around lines 1–8, add:

permissions:
  contents: read

and keep the rest of the workflow unchanged.

In general, the fix is to avoid MD5 (and other weak hashes like SHA‑1) for hashing identifiers or other sensitive data. For this specific use—deterministic bucketing of a user ID—you can replace MD5 with a stronger, standard hash from the SHA‑2 family such as SHA‑256. The numeric mapping logic (int(hash, 16) % 100) can remain the same, so functionality (deterministic but pseudo-random bucket assignment) is preserved.

Concretely, in accounts_service/accounts_service.py around line 526, change:

user_hash = int(hashlib.md5(browser_user_id.encode()).hexdigest(), 16)

to:

user_hash = int(hashlib.sha256(browser_user_id.encode()).hexdigest(), 16)

No additional imports are required since hashlib is already imported at the top of the file. This is the only necessary code change; all other logic (reading the header, using modulo 100, logging, and returning lcp_delay_ms) should remain as-is to avoid altering existing behavior apart from the internal distribution of users across buckets, which is acceptable for an A/B test.

General fix: Replace the use of the broken MD5 hash with a strong, modern hash function. Since this hashing is not for passwords but for deterministic bucketing, using hashlib.sha256 (or any SHA-2/SHA-3 function) is appropriate and requires minimal code change.

Best way in this code: In accounts_service/accounts_service.py, at the line where user_hash is computed, change hashlib.md5(user_id.encode()).hexdigest() to hashlib.sha256(user_id.encode()).hexdigest(). This preserves the same overall “hash-to-int-then-mod-100” pattern and keeps functionality (deterministic mapping of each user_id to a 0–99 bucket) unchanged in spirit, while avoiding the insecure MD5 algorithm. No new imports are needed because hashlib is already imported at the top of the file.

Concrete change:

• File: accounts_service/accounts_service.py
• Around line 571, replace:
  • user_hash = int(hashlib.md5(user_id.encode()).hexdigest(), 16)
• With:
  • user_hash = int(hashlib.sha256(user_id.encode()).hexdigest(), 16)

No additional methods or definitions are required; the standard library already provides hashlib.sha256.

---