Native Bulk Element Deletion API

[RohitPtnkr1996]

imodel-native: iTwin/imodel-native#1344

I have added performance tests comparing deleteElement (old) vs deleteElements (new) and deleteDefinitionElements (old) vs deleteElements (new).

The new APIs were expected to do additional validations like checking for intra and inter set FK violations, auto-handling parent child deletions within the same call, expanding sub-models if a modeling element is being deleted, making the deletions element-order-agnostic (all of which are not handled by the old API calls).

Performance stats from the new full-stack-tests perftest added:

1. deleteElement (loop) vs deleteElements (bulk)

Element Count	deleteElement	deleteElements	Improvement %
5	7.39 ms	3.95 ms	46.5%
25	11.67 ms	7.61 ms	34.8%
50	18.12 ms	12.45 ms	31.3%
250	266.90 ms	57.92 ms	78.3%
1,000	1.00 secs	245.92 ms	75.6%
10,000	12.51 secs	2.90 secs	76.8%
100,000	3.56 mins	34.84 secs	83.7%
1 M	19.04 mins	6.06 mins	68.2%
1.7 M	32.77 mins	10.73 mins	67.3%

2. deleteDefinitionElements vs deleteElements

Element Count	deleteDefinitionElements	deleteElements	Improvement %
5	4.38 ms	12.81 ms	-192.3%
10	5.90 ms	15.18 ms	-157.2%
50	18.75 ms	25.99 ms	-38.6%
100	35.83 ms	41.46 ms	-15.7%
1,000	357.16 ms	262.02 ms	26.6%
10,000	6.16 secs	3.24 secs	47.4%
100,000	1.47 mins	35.73 secs	59.7%
1 M	17.57 mins	7.16 mins	59.2%

When definition elements exist in the delete set, we run checks to ensure there is no element which is being used, along with validations for FK violations, parent-child and sub-models cascade delete support.
This gives us a better deletion confidence and a reduction in API call failures, but at the expense of a performance hit.
Adding to that are the domain handler callbacks that need to be called before and after deleting every element, which take up a chunk of the processing time.

Hence, the validations add significant overhead to the running time.
A more realistic performance check will be to run the new Api against the internal tool that the user has written to do the validations and compare the performances.

@khanaffan, will you please take a look at the tests and the native API ?

[rschili]

I approve, but those performance numbers make me wonder if this is really good. The gain is not big, and the slowdown with <50 deletes is significant. Maybe the typescript deleteElements method could check the size of the provided ID set and then call one or the other internally?

[RohitPtnkr1996]

I approve, but those performance numbers make me wonder if this is really good. The gain is not big, and the slowdown with <50 deletes is significant. Maybe the typescript deleteElements method could check the size of the provided ID set and then call one or the other internally?

Yeah, the performance numbers are a little unfair to be honest.
The old api simply deletes the elements one at a time without any validations on the elements being deleted.
The users have written their own tool in typescript that does all the validation before calling deleteElement.

My plan was to ask the users to try out this beta which does everything on the native side instead of their TS tool and share their findings on the performance.

[hl662]

Is there a new entry on NextVersion.md that perhaps mentions this is experimental and encourage users to try using it instead of our existing deleteElement API?

[hl662]

Reviewed the deleteElements bulk deletion API across four phases (API surface, core logic, tests, documentation). Overall this is a solid addition — strong test coverage for cascading, constraint checking, and edge cases. A few items below, roughly ordered by importance.

File-level notes (no specific line):

• All tests go through EditTxn.deleteElements — the public IModelDb.Elements.deleteElements wrapper (with its own input validation, cache eviction, and return-set construction) isn't exercised directly. A small integration test calling iModelDb.elements.deleteElements(...) would cover that path.
• The ModelSelectorRefersToModels link-table cleanup is well covered — any appetite for a similar test with ElementRefersToElements or ElementDrivesElement? The learning doc mentions those are cleaned up too, but there's no test verifying it.

[hl662]

awesome, thanks :)