Skip to content

[Tree widget]: use IdSet instead of InVirtualSet on next#1629

Draft
JonasDov wants to merge 7 commits intotree-widget/nextfrom
JonasD/ids-set
Draft

[Tree widget]: use IdSet instead of InVirtualSet on next#1629
JonasDov wants to merge 7 commits intotree-widget/nextfrom
JonasD/ids-set

Conversation

@JonasDov
Copy link
Contributor

closes #1590

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tree-Widget Next benchmark

Benchmark suite Current: a4e06f4 Previous: 9e3cf65 Deviation Status
categories tree creates initial filtered view for 50k items 1713.47 ms 1491.45 ms 14.89% 🚨
categories tree creates initial filtered view for 50k items (P95 of main thread blocks) 79 ms 78 ms 1.28% 〰️
categories tree changing definition container visibility changes visibility for 50k subCategories 5400.93 ms 5447.92 ms -0.86% 〰️
categories tree changing definition container visibility changes visibility for 50k subCategories (P95 of main thread blocks) 61 ms 66 ms -7.58% 〰️
categories tree changing definition container visibility changes visibility for 50k categories 9521.5 ms 10935.91 ms -12.93%
categories tree changing definition container visibility changes visibility for 50k categories (P95 of main thread blocks) 357 ms 571 ms -37.48%
classifications tree loads initial view for iModel with 50k classifications 35.82 ms 37.72 ms -5.04% 〰️
classifications tree loads initial view for iModel with 50k classifications (P95 of main thread blocks) 0 ms 0 ms 0% 🟰
classifications tree loads first branch for iModel with 50k classifications 850.2 ms 899.54 ms -5.49% 〰️
classifications tree loads first branch for iModel with 50k classifications (P95 of main thread blocks) 80 ms 79 ms 1.27% 〰️
models tree creates initial filtered view for 50k target items 824.66 ms 889.52 ms -7.29% 〰️
models tree creates initial filtered view for 50k target items (P95 of main thread blocks) 63 ms 82 ms -23.17% 〰️
models tree validates categories visibility for imodel with 50k categories 4663.16 ms 4572.18 ms 1.99% 〰️
models tree validates categories visibility for imodel with 50k categories (P95 of main thread blocks) 326 ms 175 ms 86.29% 🚨
models tree changing model visibility changes visibility for 50k elements 1985.63 ms 1994.85 ms -0.46% 〰️
models tree changing model visibility changes visibility for 50k elements (P95 of main thread blocks) 0 ms 0 ms 0% 🟰
models tree changing category visibility changes visibility for 50k elements 1976.76 ms 1999.18 ms -1.12% 〰️
models tree changing category visibility changes visibility for 50k elements (P95 of main thread blocks) 0 ms 0 ms 0% 🟰
models tree changing per-model-category override changes visibility for 50k elements 1952.07 ms 1957.44 ms -0.27% 〰️
models tree changing per-model-category override changes visibility for 50k elements (P95 of main thread blocks) 0 ms 0 ms 0% 🟰
models tree changing element visibility changes only parent nodes visibility with 50k elements 2700.41 ms 2768.95 ms -2.48% 〰️
models tree changing element visibility changes only parent nodes visibility with 50k elements (P95 of main thread blocks) 324 ms 312 ms 3.85% 〰️
models tree changing element visibility changes only parent nodes visibility with 50k child elements with different categories 2674.13 ms 2696.04 ms -0.81% 〰️
models tree changing element visibility changes only parent nodes visibility with 50k child elements with different categories (P95 of main thread blocks) 222 ms 384 ms -42.19%

This comment was automatically generated by workflow using github-action-benchmark.

supportsFiltering: this.supportsFiltering(),
})}
FROM ${subjectFilterClauses.from} this
FROM ${subjectFilterClauses.from} this, IdSet(?) idSetTable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I'd rather we use JOIN syntax for this
  2. Can we alias this virtual table with something more specific than idSetTable? Maybe childSubject?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applies to other query modifications in this PR, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates multiple ECSQL queries in the tree-widget (next) implementation from large IN (...) lists / InVirtualSet usage patterns to the IdSet virtual table, aiming to improve query performance and scalability (Issue #1590).

Changes:

  • Replaced many IN (${ids.map(() => "?")...}) patterns with JOIN IdSet(?) ... and consolidated bindings into idset bindings.
  • Added ECSQLOPTIONS ENABLE_EXPERIMENTAL_FEATURES to queries that rely on IdSet.
  • Updated related types/utilities (e.g., removing joinId64Arg, tightening some APIs to accept Id64Array) and bumped dependencies/peer requirements to iTwin 5.7.x.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
packages/itwin/tree-widget/src/tree-widget-react/components/trees/models-tree/ModelsTreeDefinition.ts Uses IdSet joins for subject/model/category/element filtering and updates query bindings accordingly.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/imodel-content-tree/IModelContentTreeDefinition.ts Switches subject/model/category filtering to IdSet joins with idset bindings.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/external-sources-tree/ExternalSourcesTreeDefinition.ts Replaces IN (...) filters with IdSet joins and standardizes class name constant usage.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/common/internal/caches/ElementChildrenCache.ts Uses IdSet for recursive children lookup and supplies idset bindings.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/common/internal/caches/AlwaysAndNeverDrawnElementInfoCache.ts Replaces InVirtualSet filtering with IdSet join for element info expansion.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/common/internal/Utils.ts Removes joinId64Arg helper now that queries use IdSet/bindings.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/classifications-tree/internal/ClassificationsTreeIdsCache.ts Switches filtered element lookups to IdSet and narrows inputs to Id64Array.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/classifications-tree/ClassificationsTreeDefinition.ts Migrates multiple classification-related queries/CTEs to IdSet + idset bindings.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/categories-tree/internal/UseSearchPaths.ts Adjusts call site to pass arrays to updated cache API.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/categories-tree/internal/CategoriesTreeIdsCache.ts Uses IdSet for filtered element model lookup and updates API types to Id64Array.
packages/itwin/tree-widget/src/tree-widget-react/components/trees/categories-tree/CategoriesTreeDefinition.ts Migrates several queries to IdSet joins and updates bindings (some membership checks moved to InVirtualSet).
packages/itwin/tree-widget/pnpm-lock.yaml Updates iTwin dependency versions to 5.7.2 and related transitive updates.
packages/itwin/tree-widget/package.json Raises peer/dev dependency ranges to iTwin 5.7.x to support IdSet.
packages/itwin/tree-widget/api/tree-widget-react.api.md API extraction output updated (parameter name normalization).
cspell.json Adds ECSQLOPTIONS (and reorders/adds terms) for spellcheck.
Files not reviewed (1)
  • packages/itwin/tree-widget/pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tree widget: Replace usages of InVirtualSet with IdSet virtual table

3 participants