Skip to content

Commit e8f8f49

Browse files
authored
[turbopack] fix feature usage telemetry (#93100)
## Report Turbopack feature-usage telemetry Turbopack never reported `NEXT_BUILD_FEATURE_USAGE` telemetry for production builds. This PR wires it up and fixes a correctness bug in how the counts were computed, then cleans up the API surface that carried them across the napi boundary. ### Changes - **JS**: `turbopackBuild()` now records `EVENT_BUILD_FEATURE_USAGE` events after `writeAllEntrypointsToDisk` via a new `eventBuildFeatureUsageFromTurbopackDiagnostics` helper. Dev is out of scope — webpack's `TelemetryPlugin` is `!dev && isClient` too. - **Rust**: aligned feature names with the JS `EventBuildFeatureUsage['featureName']` union — SWC triple is now `swc/target/<triple>`; dropped `persistentCaching` (redundant with `turbopackFileSystemCache`) and `turbotrace: false` (hardcoded). ### Correctness fix: count unique importers, not resolves Previously feature-usage counts for module imports (`next/image`, `next/font/google`, …) were computed from a `BeforeResolvePlugin` that emitted one event per resolve. Turbopack caches resolves, so the emission fired at most **once per unique request** — the count was effectively `1` for every feature that was imported anywhere. Webpack's equivalent counts unique importing modules via `moduleGraph.getIncomingConnections(module).size`. This PR replaces the resolve-plugin emission with a single whole-app module-graph traversal on `Project`. For each tracked feature, we accumulate the set of unique parent modules of each matching node (mirroring webpack's "unique origin modules" semantics). Fonts are matched against their synthesized `/target.css?…` virtual modules produced by the SWC font-loader transform — matching webpack's `FEATURE_MODULE_REGEXP_MAP` approach. Paths are matched via `phf_map!` tables in `next_telemetry.rs`. ### Incidental simplifications While in here, the `Diagnostic` collectibles subsystem got right-sized and then removed entirely, since feature usage was its only consumer: - `Project::project_feature_usage()` returns a structured `Vc<ProjectFeatureUsageSummary>` instead of emitting diagnostics. Surfaced to JS as a dedicated `project.featureUsage(): Promise<BuildFeatureUsage[]>` napi method, called once at build's end. - `TurbopackResult<T>` loses its `diagnostics: BuildFeatureUsage[]` field — it's now just `{ result, issues }`. Every napi result type and ~10 construction sites are correspondingly simpler. - Deleted `turbopack_core::diagnostics` entirely (`Diagnostic` trait, `DiagnosticExt`, `DiagnosticContextExt`, `CapturedDiagnostics`, `PlainBuildFeatureUsage`). Deleted `FeatureUsageTelemetry`, `ModuleFeatureReportResolvePlugin`, `get_diagnostics()` aggregation, the `feature_usage`/`diagnostics` fields on `AllWrittenEntrypointsWithIssues`/`OperationResult`/`EntrypointsWithIssues`/`WrittenEndpointWithIssues`/`HmrUpdateWithIssues`/`HmrChunkNamesWithIssues`/`EndpointIssuesAndDiags`/`WriteAnalyzeResult`, and the defensive `drop_collectibles::<Box<dyn Diagnostic>>()` scrub in `entrypoints_without_collectibles_operation`. Feature-usage telemetry now flows as a plain return value end-to-end: `Project::project_feature_usage()` → napi `projectFeatureUsage()` → JS `project.featureUsage()` → `telemetry.record()`. No collectibles, no peeking, no emission-as-side-effect. ### Tests Un-skipped four previously webpack-only integration tests in `test/integration/telemetry/test/config.test.ts`: `image/script/dynamic`, `next/legacy/image`, `transpilePackages`, and middleware options. All pass under Turbopack. The remaining three skipped tests (`swc` flags, `@vercel/og`, `useCache`) cover features Turbopack doesn't emit yet — left skipped with TODOs. Added unit test for the helper at `packages/next/src/telemetry/events/build.test.ts`. Updated the Turbopack `next-rs-api` snapshot to reflect the new diagnostic shape. <!-- NEXT_JS_LLM_PR -->
1 parent fdd0499 commit e8f8f49

25 files changed

Lines changed: 532 additions & 889 deletions

File tree

crates/next-api/src/operation.rs

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,10 @@ use anyhow::Result;
22
use bincode::{Decode, Encode};
33
use turbo_rcstr::RcStr;
44
use turbo_tasks::{
5-
CollectiblesSource, FxIndexMap, NonLocalValue, OperationValue, OperationVc, ResolvedVc,
6-
TaskInput, Vc, debug::ValueDebugFormat, take_effects, trace::TraceRawVcs,
5+
FxIndexMap, NonLocalValue, OperationValue, OperationVc, ResolvedVc, TaskInput, Vc,
6+
debug::ValueDebugFormat, take_effects, trace::TraceRawVcs,
77
};
8-
use turbopack_core::{diagnostics::Diagnostic, issue::CollectibleIssuesExt};
8+
use turbopack_core::issue::CollectibleIssuesExt;
99

1010
use crate::{
1111
entrypoints::Entrypoints,
@@ -31,14 +31,13 @@ pub struct EntrypointsOperation {
3131
pub pages_error_endpoint: OperationVc<OptionEndpoint>,
3232
}
3333

34-
/// Removes diagnostics, issues, and effects from the top-level `entrypoints` operation so that
35-
/// they're not duplicated across many different individual entrypoints or routes.
34+
/// Removes issues and effects from the top-level `entrypoints` operation so that they're not
35+
/// duplicated across many different individual entrypoints or routes.
3636
#[turbo_tasks::function(operation)]
3737
async fn entrypoints_without_collectibles_operation(
3838
entrypoints: OperationVc<Entrypoints>,
3939
) -> Result<Vc<Entrypoints>> {
4040
let _ = entrypoints.resolve().strongly_consistent().await?;
41-
entrypoints.drop_collectibles::<Box<dyn Diagnostic>>();
4241
entrypoints.drop_issues();
4342
let _ = take_effects(entrypoints).await?;
4443
Ok(entrypoints.connect())

crates/next-api/src/project.rs

Lines changed: 174 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ use next_core::{
2424
get_server_chunking_context_with_client_assets, get_server_compile_time_info,
2525
get_server_module_options_context, get_server_resolve_options_context,
2626
},
27-
next_telemetry::NextFeatureTelemetry,
27+
next_telemetry::ProjectFeatureUsageSummary,
2828
parse_segment_config_from_source,
2929
segment_config::ParseSegmentMode,
3030
util::{NextRuntime, OptionEnvMap},
@@ -35,7 +35,7 @@ use tracing::{Instrument, field::Empty};
3535
use turbo_rcstr::{RcStr, rcstr};
3636
use turbo_tasks::{
3737
Completion, Completions, FxIndexMap, NonLocalValue, OperationValue, OperationVc, ReadRef,
38-
ResolvedVc, State, TaskInput, TransientInstance, TryFlatJoinIterExt, Vc,
38+
ResolvedVc, State, TaskInput, TransientInstance, TryFlatJoinIterExt, TryJoinIterExt, Vc,
3939
debug::ValueDebugFormat, fxindexmap, trace::TraceRawVcs,
4040
};
4141
use turbo_tasks_env::{EnvMap, ProcessEnv};
@@ -56,7 +56,6 @@ use turbopack_core::{
5656
},
5757
compile_time_info::CompileTimeInfo,
5858
context::AssetContext,
59-
diagnostics::DiagnosticExt,
6059
environment::NodeJsVersion,
6160
file_source::FileSource,
6261
ident::Layer,
@@ -1690,81 +1689,184 @@ impl Project {
16901689
}
16911690
}
16921691

1693-
/// Emit a telemetry event corresponding to [webpack configuration telemetry](https://github.com/vercel/next.js/blob/9da305fe320b89ee2f8c3cfb7ecbf48856368913/packages/next/src/build/webpack-config.ts#L2516)
1694-
/// to detect which feature is enabled.
1692+
/// Computes the project's feature-usage telemetry summary.
1693+
///
1694+
/// Includes:
1695+
/// - The SWC target triple (`swc/target/...`, always on).
1696+
/// - Boolean config and compiler-option flags, mirroring the webpack [`TelemetryPlugin`](https://github.com/vercel/next.js/blob/9da305fe320b89ee2f8c3cfb7ecbf48856368913/packages/next/src/build/webpack-config.ts#L2516)
1697+
/// shape.
1698+
/// - Per-feature-module import counts (e.g. `next/image`, `next/font/google`) computed by
1699+
/// walking the whole-app module graph and counting **unique importing modules** per feature.
1700+
/// This replaces an earlier `before_resolve` plugin that emitted telemetry per resolve;
1701+
/// because Turbopack caches resolves, the earlier approach under-counted to at most one per
1702+
/// feature.
1703+
///
1704+
/// Returns `bail!` if the project is not in build mode — `whole_app_module_graphs` drops
1705+
/// issues in development and the graph may not reflect the full project, so reporting
1706+
/// telemetry from dev would produce misleading counts.
1707+
///
1708+
/// The returned summary is sorted by feature name for determinism.
16951709
#[turbo_tasks::function]
1696-
async fn collect_project_feature_telemetry(self: Vc<Self>) -> Result<()> {
1697-
let emit_event = |feature_name: &str, enabled: bool| {
1698-
NextFeatureTelemetry::new(feature_name.into(), enabled)
1699-
.resolved_cell()
1700-
.emit();
1701-
};
1702-
1703-
// First, emit an event for the binary target triple.
1704-
// This is different to webpack-config; when this is being called,
1705-
// it is always using SWC so we don't check swc here.
1706-
emit_event(env!("VERGEN_CARGO_TARGET_TRIPLE"), true);
1710+
pub async fn project_feature_usage(
1711+
self: ResolvedVc<Self>,
1712+
) -> Result<Vc<ProjectFeatureUsageSummary>> {
1713+
if !self.next_mode().await?.is_production() {
1714+
bail!("project_feature_usage() may only be called during `next build`");
1715+
}
17071716

1708-
// Go over config and report enabled features.
1709-
// [TODO]: useSwcLoader is not being reported as it is not directly corresponds (it checks babel config existence)
1710-
// need to confirm what we'll do with turbopack.
1717+
// (public feature specifier, path suffix) pairs. The suffix identifies the resolved
1718+
// feature module; we match via `module.ident().path.path.ends_with(suffix)`. Mirrors
1719+
// the webpack `FEATURE_MODULE_MAP` + `FEATURE_MODULE_REGEXP_MAP` in
1720+
// `packages/next/src/build/webpack/plugins/telemetry-plugin/telemetry-plugin.ts`.
1721+
//
1722+
// Font specifiers (`next/font/*`, `@next/font/*`) are matched against the synthesized
1723+
// `target.css` virtual module produced by the Next.js font loader transform
1724+
// (`crates/next-custom-transforms/src/transforms/fonts`). That transform rewrites
1725+
// `import { Inter } from 'next/font/google'` into
1726+
// `import inter from 'next/font/google/target.css?{...}'` — the original specifier never
1727+
// appears in the module graph, but the synthesized `target.css` module's path suffix does.
1728+
// `ident.path.path` does not include the query string (that lives on `ident.query`), so
1729+
// `ends_with` is the correct matcher here.
1730+
static FEATURE_MODULE_PATH_SUFFIXES: &[(&str, &str)] = &[
1731+
("next/image", "/next/image.js"),
1732+
("next/future/image", "/next/future/image.js"),
1733+
("next/legacy/image", "/next/legacy/image.js"),
1734+
("next/script", "/next/script.js"),
1735+
("next/dynamic", "/next/dynamic.js"),
1736+
("next/font/google", "/next/font/google/target.css"),
1737+
("next/font/local", "/next/font/local/target.css"),
1738+
("@next/font/google", "/@next/font/google/target.css"),
1739+
("@next/font/local", "/@next/font/local/target.css"),
1740+
];
1741+
1742+
// TODO: useSwcLoader is not being reported as it is not directly corresponds (it checks
1743+
// babel config existence) — need to confirm what we'll do with turbopack.
17111744
let config = self.next_config();
1712-
1713-
emit_event(
1714-
"skipProxyUrlNormalize",
1715-
*config.skip_proxy_url_normalize().await?,
1716-
);
1717-
1718-
emit_event(
1719-
"skipTrailingSlashRedirect",
1720-
*config.skip_trailing_slash_redirect().await?,
1721-
);
1722-
emit_event(
1723-
"persistentCaching",
1724-
*self.is_persistent_caching_enabled().await?,
1725-
);
1726-
1727-
emit_event(
1728-
"modularizeImports",
1729-
!config.modularize_imports().await?.is_empty(),
1730-
);
1731-
emit_event(
1732-
"transpilePackages",
1733-
!config.transpile_packages().await?.is_empty(),
1734-
);
1735-
emit_event("turbotrace", false);
1736-
1737-
// compiler options
17381745
let compiler_options = config.compiler().await?;
1739-
let swc_relay_enabled = compiler_options.relay.is_some();
1740-
let styled_components_enabled = compiler_options
1741-
.styled_components
1742-
.as_ref()
1743-
.map(|sc| sc.is_enabled())
1744-
.unwrap_or_default();
1745-
let react_remove_properties_enabled = compiler_options
1746-
.react_remove_properties
1747-
.as_ref()
1748-
.map(|rc| rc.is_enabled())
1749-
.unwrap_or_default();
1750-
let remove_console_enabled = compiler_options
1751-
.remove_console
1752-
.as_ref()
1753-
.map(|rc| rc.is_enabled())
1754-
.unwrap_or_default();
1755-
let emotion_enabled = compiler_options
1756-
.emotion
1757-
.as_ref()
1758-
.map(|e| e.is_enabled())
1759-
.unwrap_or_default();
1746+
let mut features: Vec<(RcStr, u32)> = vec![
1747+
// SWC target triple is prefixed with `swc/target/` to match the webpack
1748+
// `swc/target/${SWC_TARGET_TRIPLE}` variant in `EventBuildFeatureUsage`.
1749+
(
1750+
format!("swc/target/{}", env!("VERGEN_CARGO_TARGET_TRIPLE")).into(),
1751+
1,
1752+
),
1753+
(
1754+
rcstr!("skipProxyUrlNormalize"),
1755+
(*config.skip_proxy_url_normalize().await?) as u32,
1756+
),
1757+
(
1758+
rcstr!("skipTrailingSlashRedirect"),
1759+
(*config.skip_trailing_slash_redirect().await?) as u32,
1760+
),
1761+
(
1762+
rcstr!("modularizeImports"),
1763+
!config.modularize_imports().await?.is_empty() as u32,
1764+
),
1765+
(
1766+
rcstr!("transpilePackages"),
1767+
!config.transpile_packages().await?.is_empty() as u32,
1768+
),
1769+
(rcstr!("swcRelay"), compiler_options.relay.is_some() as u32),
1770+
(
1771+
rcstr!("swcStyledComponents"),
1772+
compiler_options
1773+
.styled_components
1774+
.as_ref()
1775+
.is_some_and(|sc| sc.is_enabled()) as u32,
1776+
),
1777+
(
1778+
rcstr!("swcReactRemoveProperties"),
1779+
compiler_options
1780+
.react_remove_properties
1781+
.as_ref()
1782+
.is_some_and(|rc| rc.is_enabled()) as u32,
1783+
),
1784+
(
1785+
rcstr!("swcRemoveConsole"),
1786+
compiler_options
1787+
.remove_console
1788+
.as_ref()
1789+
.is_some_and(|rc| rc.is_enabled()) as u32,
1790+
),
1791+
(
1792+
rcstr!("swcEmotion"),
1793+
compiler_options
1794+
.emotion
1795+
.as_ref()
1796+
.is_some_and(|e| e.is_enabled()) as u32,
1797+
),
1798+
];
1799+
1800+
// Module-usage counts: two passes over the module graph.
1801+
// 1. Iterate all nodes, classify each in parallel, keep only feature-module matches.
1802+
// 2. Walk edges, for each edge whose target is a classified feature module, add the parent
1803+
// to that feature's unique-importer set.
1804+
let module_graph = self.whole_app_module_graphs().await?.full.await?;
1805+
1806+
let matching: FxHashMap<ResolvedVc<Box<dyn Module>>, &'static str> = module_graph
1807+
.iter_nodes()
1808+
.map(async |node| {
1809+
let ident = node.ident().await?;
1810+
let path = &ident.path.path;
1811+
for &(feature, suffix) in FEATURE_MODULE_PATH_SUFFIXES {
1812+
if path.ends_with(suffix) {
1813+
return Ok(Some((node, feature)));
1814+
}
1815+
}
1816+
Ok(None)
1817+
})
1818+
.try_flat_join()
1819+
.await?
1820+
.into_iter()
1821+
.collect();
1822+
1823+
// Collect (feature, parent) pairs for every edge whose target is a feature module.
1824+
//
1825+
// We count every such edge regardless of whether the import is eventually tree-shaken.
1826+
// This matches webpack's `TelemetryPlugin`, which hooks `finishModules` (before DCE).
1827+
// We could filter via `BindingUsageInfo` to only count edges that survive tree-shaking,
1828+
// but staying parallel to webpack lets dashboards compare counts across the two bundlers
1829+
// directly.
1830+
let mut pairs: FxHashSet<(&'static str, ResolvedVc<Box<dyn Module>>)> =
1831+
FxHashSet::default();
1832+
module_graph.traverse_edges_unordered(|parent, node| {
1833+
if let Some((parent_node, _)) = parent
1834+
&& let Some(&feature) = matching.get(&node)
1835+
{
1836+
pairs.insert((feature, parent_node));
1837+
}
1838+
Ok(())
1839+
})?;
17601840

1761-
emit_event("swcRelay", swc_relay_enabled);
1762-
emit_event("swcStyledComponents", styled_components_enabled);
1763-
emit_event("swcReactRemoveProperties", react_remove_properties_enabled);
1764-
emit_event("swcRemoveConsole", remove_console_enabled);
1765-
emit_event("swcEmotion", emotion_enabled);
1841+
// Dedupe parents by their source location (path + query + fragment), ignoring
1842+
// `ident().layer` and other modifiers. In Turbopack the same user file often appears as
1843+
// separate modules per layer (e.g. SSR, client, edge), but webpack counts one "importer"
1844+
// per source file — this matches that semantics.
1845+
let parent_source_keys = pairs
1846+
.into_iter()
1847+
.map(async |(feature, parent)| {
1848+
let ident = parent.ident().await?;
1849+
let key = (
1850+
ident.path.path.clone(),
1851+
ident.query.clone(),
1852+
ident.fragment.clone(),
1853+
);
1854+
Ok((feature, key))
1855+
})
1856+
.try_join()
1857+
.await?;
1858+
1859+
let mut importers: FxHashMap<&'static str, FxHashSet<(RcStr, RcStr, RcStr)>> =
1860+
FxHashMap::default();
1861+
for (feature, key) in parent_source_keys {
1862+
importers.entry(feature).or_default().insert(key);
1863+
}
1864+
for (feature, unique_sources) in importers {
1865+
features.push((RcStr::from(feature), unique_sources.len() as u32));
1866+
}
17661867

1767-
Ok(())
1868+
features.sort_by(|a, b| a.0.cmp(&b.0));
1869+
Ok(ProjectFeatureUsageSummary { features }.cell())
17681870
}
17691871

17701872
/// Scans the app/pages directories for entry points files (matching the
@@ -1779,8 +1881,6 @@ impl Project {
17791881
self: Vc<Self>,
17801882
app_route_filter: Option<Vec<RcStr>>,
17811883
) -> Result<Vc<Entrypoints>> {
1782-
self.collect_project_feature_telemetry().await?;
1783-
17841884
let this = self.await?;
17851885
let mut routes = FxIndexMap::default();
17861886
let app_project = self.app_project();

crates/next-core/src/next_client/context.rs

Lines changed: 6 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ use crate::{
5151
get_next_client_resolved_map,
5252
},
5353
next_shared::{
54-
resolve::{ModuleFeatureReportResolvePlugin, NextSharedRuntimeResolvePlugin},
54+
resolve::NextSharedRuntimeResolvePlugin,
5555
transforms::{
5656
emotion::get_emotion_transform_rule,
5757
react_remove_properties::get_react_remove_properties_transform_rule,
@@ -180,18 +180,11 @@ pub async fn get_client_resolve_options_context(
180180
resolved_map: Some(next_client_resolved_map),
181181
browser: true,
182182
module: true,
183-
before_resolve_plugins: vec![
184-
ResolvedVc::upcast(
185-
ModuleFeatureReportResolvePlugin::new(project_path.clone())
186-
.to_resolved()
187-
.await?,
188-
),
189-
ResolvedVc::upcast(
190-
NextFontLocalResolvePlugin::new(project_path.clone())
191-
.to_resolved()
192-
.await?,
193-
),
194-
],
183+
before_resolve_plugins: vec![ResolvedVc::upcast(
184+
NextFontLocalResolvePlugin::new(project_path.clone())
185+
.to_resolved()
186+
.await?,
187+
)],
195188
after_resolve_plugins: vec![ResolvedVc::upcast(
196189
NextSharedRuntimeResolvePlugin::new(project_path.clone())
197190
.to_resolved()

crates/next-core/src/next_edge/context.rs

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ use crate::{
2727
next_font::local::NextFontLocalResolvePlugin,
2828
next_import_map::{get_next_edge_and_server_fallback_import_map, get_next_edge_import_map},
2929
next_server::context::ServerContextType,
30-
next_shared::resolve::{ModuleFeatureReportResolvePlugin, NextSharedRuntimeResolvePlugin},
30+
next_shared::resolve::NextSharedRuntimeResolvePlugin,
3131
util::{
3232
NextRuntime, OptionEnvMap, defines, foreign_code_context_condition,
3333
free_var_references_with_vercel_system_env_warnings, worker_forwarded_globals,
@@ -109,22 +109,19 @@ pub async fn get_edge_resolve_options_context(
109109
.to_resolved()
110110
.await?;
111111

112-
let mut before_resolve_plugins = vec![ResolvedVc::upcast(
113-
ModuleFeatureReportResolvePlugin::new(project_path.clone())
114-
.to_resolved()
115-
.await?,
116-
)];
117-
if matches!(
112+
let before_resolve_plugins = if matches!(
118113
ty,
119114
ServerContextType::Pages { .. }
120115
| ServerContextType::AppSSR { .. }
121116
| ServerContextType::AppRSC { .. }
122117
) {
123-
before_resolve_plugins.push(ResolvedVc::upcast(
118+
vec![ResolvedVc::upcast(
124119
NextFontLocalResolvePlugin::new(project_path.clone())
125120
.to_resolved()
126121
.await?,
127-
));
122+
)]
123+
} else {
124+
vec![]
128125
};
129126

130127
let after_resolve_plugins = vec![ResolvedVc::upcast(

0 commit comments

Comments
 (0)