Skip to content

Perf: Index collections by tag instead of re-sorting and re-deriving tags per lookup - #4332

Open
j9t wants to merge 1 commit into
11ty:mainfrom
j9t:perf/collection-tag-index
Open

Perf: Index collections by tag instead of re-sorting and re-deriving tags per lookup#4332
j9t wants to merge 1 commit into
11ty:mainfrom
j9t:perf/collection-tag-index

Conversation

@j9t

@j9t j9t commented Jul 29, 2026

Copy link
Copy Markdown

As mentioned in prior conversation with @zachleat, a major performance improvement, and an AI-assisted result of performance optimizations on one of my Eleventy projects. I took the liberty to use AI (Claude Code) for the following summary, too.

I vouch for doing due diligence and already, successfully using the same optimizations in my own projects (via patch-package)—a massive speed-up.

Problem

TemplateCollection.getFilteredByTag() is called once per collection (tag) per build, from
TemplateMap.getTaggedCollection(). Each call did two things that scale badly:

  1. getAllSorted() re-sorted the entire collection (Sortable.sort() has no cache), so a site with n templates and t tags paid t × O(n log n) comparisons.
  2. The filter callback called TemplateData.getIncludedTagNames(item.data) for every item, which rebuilds (and deduplicates) the tag array from scratch—t × n array allocations.

On a large site this dominates the build. On a 22,000-template site with ~1,100 tags that is roughly
24 million sort comparisons and 23 million tag-array rebuilds per build.

Change

TemplateCollection now keeps two caches, both invalidated in add()—the only entry point that
mutates items, and the same signal the existing _filteredByGlobsCache relies on via _dirty:

  • the sorted item list, so the collection is sorted once per mutation rather than once per lookup;
  • a Map of tag name to items (built by walking the sorted list once), so getFilteredByTag() is a Map.get() instead of a full scan.

Because the index is built from the sorted list, each tag’s items are already in the same order the
filter produced, so results are unchanged. getAllSorted(), getFilteredByTag() and
getFilteredByTags() all keep returning fresh arrays, so callers can still sort or reverse the
result in place (#352).

getFilteredByTags() (plural) now intersects the first tag’s index entry with the remaining tags
instead of scanning the whole collection.

Results

Synthetic benchmark, one getFilteredByTag() call per unique tag (as a build does), 6 tags per item:

Templates Tags Before After
500 25 14ms 3ms
2,000 100 134ms 3ms
10,000 500 3,359ms 20ms
22,000 1,100 17,288ms 31ms

On a real 22,000-template site, this plus the 4.0 inputPathMap change for
getMapEntryForInputPath() took a production build from ~120–150s to ~48s, with byte-identical
output.

Testing

  • Full test suite passes (1,439 tests).
  • Added regression tests in test/TemplateCollectionTest.js: cache invalidation when templates are added after a lookup, mutability/independence of returned arrays, and the no-tag-name / unknown-tag / "all" cases.
  • Differential test against the previous implementation: ~50,000 randomized comparisons (string tags, comma-separated tags, duplicate tags, missing tags, buildawesomeExcludeFromCollections as true/string/array, lookups interleaved with add()) produced identical results.

@j9t
j9t requested a review from zachleat as a code owner July 29, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant