Skip to content

feat(ab-testing): add Vercel Flags SDK integration with Matomo#17265

Draft
pettinarip wants to merge 10 commits intodevfrom
ab-testing
Draft

feat(ab-testing): add Vercel Flags SDK integration with Matomo#17265
pettinarip wants to merge 10 commits intodevfrom
ab-testing

Conversation

@pettinarip
Copy link
Copy Markdown
Member

Summary

  • Adds GDPR-compliant, cookie-less A/B testing using Vercel Flags SDK
  • Integrates with Matomo for experiment configuration and analytics tracking
  • Uses middleware to precompute flag variants and rewrite URLs (e.g., /en//en/abc123)
  • Implements deterministic variant assignment via IP + User-Agent fingerprinting
  • Adds comprehensive skill documentation for adding new experiments

Key Changes

  • New flags system: src/lib/ab-testing/flags.ts with Matomo adapter
  • Middleware integration: Precomputes variants, handles URL rewrites
  • Precomputed route: app/[locale]/[code]/page.tsx for A/B tested homepage
  • Components: ABTest, TestDebugPanel, TestTracker for variant rendering and tracking
  • Documentation: Skill file at .claude/skills/ab-testing/SKILL.md

Test plan

  • Set USE_MOCK_EXPERIMENTS=true and verify mock experiments work locally
  • Verify homepage renders correctly with precomputed routes
  • Check debug panel shows correct variant assignments (when enabled)
  • Confirm Matomo events fire correctly for variant tracking
  • Test that different IP/UA combinations get different variants

- Add middleware precomputation for A/B test routes
- Create matomo adapter for flags sdk
- Add ABTest component for precomputed flag values
- Add [code] route for static variant generation
- extract error message only in logging to avoid exposing internals
- replace middleware regex with string operations for performance
- delete unused getAbTestFlagKeys and getMatomoExperimentConfig functions
- consolidate duplicate ABTestVariants type definition
- simplify double-function adapter factory pattern
- add barrel exports for ab-testing module and AB components
- add permutation growth warning when exceeding threshold
@netlify
Copy link
Copy Markdown

netlify Bot commented Feb 10, 2026

Deploy Preview for ethereumorg ready!

Name Link
🔨 Latest commit 3367790
🔍 Latest deploy log https://app.netlify.com/projects/ethereumorg/deploys/698c4679adf881ae4b957555
😎 Deploy Preview https://deploy-preview-17265.ethereum.it
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
7 paths audited
Performance: 61 (🟢 up 6 from production)
Accessibility: 94 (no change from production)
Best Practices: 100 (no change from production)
SEO: 98 (🔴 down 2 from production)
PWA: 59 (no change from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot added dependencies 📦 Changes related to project dependencies documentation 📖 Change or add documentation tooling 🔧 Changes related to tooling of the project labels Feb 10, 2026
@pettinarip pettinarip marked this pull request as draft February 10, 2026 16:15
…irect

The trailingSlash: true config in next.config.js was causing Next.js to
308 redirect after the middleware rewrite, exposing the internal precomputed
code URL in the browser address bar.

Also enables debug overrides unconditionally to allow testing on all deploys.
@pettinarip
Copy link
Copy Markdown
Member Author

Major blocker found for this setup that base its solution around NextResponse.rewrite() call in the middleware.

Summary: The Vercel Flags SDK A/B testing implementation works correctly on Vercel but fails on Netlify due to a platform bug.

The Issue: Netlify converts NextResponse.rewrite() calls into 308 redirects when i18n (next-intl) is enabled, exposing internal URLs to users instead of silently serving rewritten content.

Evidence:

  • Middleware correctly generates rewrite: / -> /en/eyJhbG.../
  • x-middleware-rewrite header is set properly
  • Netlify responds with 308 redirect instead of serving content internally
  • Same code works perfectly on Vercel

Related GitHub Issues:

@pettinarip pettinarip added the Status: Blocked 🛑 This is blocked label Feb 11, 2026
@github-actions
Copy link
Copy Markdown
Contributor

This issue is stale because it has been open 30 days with no activity.

@github-actions github-actions Bot added the Status: Stale This issue is stale because it has been open 30 days with no activity. label Mar 14, 2026
@wackerow
Copy link
Copy Markdown
Member

@pettinarip Still holding onto this PR or could we close out?

@github-actions github-actions Bot removed the Status: Stale This issue is stale because it has been open 30 days with no activity. label Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies 📦 Changes related to project dependencies documentation 📖 Change or add documentation Status: Blocked 🛑 This is blocked tooling 🔧 Changes related to tooling of the project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants