Skip to content

[df] Preserve schema of empty output TTree in Snapshot #18197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

vepadulano
Copy link
Member

Previously, the behaviour of Snapshot in case no entries pass the selections in the event loop was two-fold:

  • In single thread, Snapshot created an output file with an empty TTree inside, without the user-requested branches.

  • In multi-threaded mode, Snapshot created an output file, without any TTree inside.

This commit aligns the behaviour of the two execution modes. Furthermore, it ensures the user-requested dataset schema is preserved even when the output TTree is empty. i.e., all the user-requested branches will be present in the output TTree.

FYI @TomasDado

@vepadulano vepadulano self-assigned this Mar 30, 2025
Copy link

github-actions bot commented Mar 30, 2025

Test Results

    18 files      18 suites   4d 1h 11m 35s ⏱️
 2 739 tests  2 739 ✅ 0 💤 0 ❌
47 629 runs  47 629 ✅ 0 💤 0 ❌

Results for commit f5d9d72.

♻️ This comment has been updated with latest results.

@vepadulano vepadulano force-pushed the rdf-snapshot-empty-tree-preserve-schema branch from d175480 to a7a87ec Compare March 30, 2025 16:53
@vepadulano vepadulano requested a review from pcanal March 30, 2025 16:55
@TomasDado
Copy link
Contributor

Thanks! I talked to Vincenzo privately, but I will also post the test results there. There seems to be a crash when JITting the types of the branches in Snapshot.

ROOT Version: 6.35.01
Built for linuxx8664gcc on Mar 31 2025, 07:52:29
From remotes/vepadulano/rdf-snapshot-empty-tree-preserve-schema@a7a87eced7
/home/tomas/ROOT_DEBUG/root_install/include/ROOT/RDF/ActionHelpers.hxx:1636:8: fatal error: instantiating fold expression with 385 arguments exceeded expression nesting limit of 256
       ...);
       ^~~~
/home/tomas/ROOT_DEBUG/root_install/include/ROOT/RDF/ActionHelpers.hxx:1673:10: note: in instantiation of function template specialization 'ROOT::Internal::RDF::SnapshotTTreeHelper<float, float, float, float, float, float, float, float, float, float, int, float, float, float, float, int, float, float, float, float, int, float, float, float, float, int, float, float, float, float, int, float, float, float, float, int, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, int, float, float, float, float, int, float, float, float, float, int, float, float, float, float, int, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, unsigned long long, unsigned int, unsigned int, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float, float>::SetEmptyBranches<0UL, 1UL, 2UL, 3UL, 4UL, 5UL, 6UL, 7UL, 8UL, 9UL, 10UL, 11UL, 12UL, 13UL, 14UL, 15UL, 16UL, 17UL, 18UL, 19UL, 20UL, 21UL, 22UL, 23UL, 24UL, 25UL, 26UL, 27UL, 28UL, 29UL, 30UL, 31UL, 32UL, 33UL, 34UL, 35UL, 36UL, 37UL, 38UL, 39UL, 40UL, 41UL, 42UL, 43UL, 44UL, 45UL, 46UL, 47UL, 48UL, 49UL, 50UL, 51UL, 52UL, 53UL, 54UL, 55UL, 56UL, 57UL, 58UL, 59UL, 60UL, 61UL, 62UL, 63UL, 64UL, 65UL, 66UL, 67UL, 68UL, 69UL, 70UL, 71UL, 72UL, 73UL, 74UL, 75UL, 76UL, 77UL, 78UL, 79UL, 80UL, 81UL, 82UL, 83UL, 84UL, 85UL, 86UL, 87UL, 88UL, 89UL, 90UL, 91UL, 92UL, 93UL, 94UL, 95UL, 96UL, 97UL, 98UL, 99UL, 100UL, 101UL, 102UL, 103UL, 104UL, 105UL, 106UL, 107UL, 108UL, 109UL, 110UL, 111UL, 112UL, 113UL, 114UL, 115UL, 116UL, 117UL, 118UL, 119UL, 120UL, 121UL, 122UL, 123UL, 124UL, 125UL, 126UL, 127UL, 128UL, 129UL, 130UL, 131UL, 132UL, 133UL, 134UL, 135UL, 136UL, 137UL, 138UL, 139UL, 140UL, 141UL, 142UL, 143UL, 144UL, 145UL, 146UL, 147UL, 148UL, 149UL, 150UL, 151UL, 152UL, 153UL, 154UL, 155UL, 156UL, 157UL, 158UL, 159UL, 160UL, 161UL, 162UL, 163UL, 164UL, 165UL, 166UL, 167UL, 168UL, 169UL, 170UL, 171UL, 172UL, 173UL, 174UL, 175UL, 176UL, 177UL, 178UL, 179UL, 180UL, 181UL, 182UL, 183UL, 184UL, 185UL, 186UL, 187UL, 188UL, 189UL, 190UL, 191UL, 192UL, 193UL, 194UL, 195UL, 196UL, 197UL, 198UL, 199UL, 200UL, 201UL, 202UL, 203UL, 204UL, 205UL, 206UL, 207UL, 208UL, 209UL, 210UL, 211UL, 212UL, 213UL, 214UL, 215UL, 216UL, 217UL, 218UL, 219UL, 220UL, 221UL, 222UL, 223UL, 224UL, 225UL, 226UL, 227UL, 228UL, 229UL, 230UL, 231UL, 232UL, 233UL, 234UL, 235UL, 236UL, 237UL, 238UL, 239UL, 240UL, 241UL, 242UL, 243UL, 244UL, 245UL, 246UL, 247UL, 248UL, 249UL, 250UL, 251UL, 252UL, 253UL, 254UL, 255UL, 256UL, 257UL, 258UL, 259UL, 260UL, 261UL, 262UL, 263UL, 264UL, 265UL, 266UL, 267UL, 268UL, 269UL, 270UL, 271UL, 272UL, 273UL, 274UL, 275UL, 276UL, 277UL, 278UL, 279UL, 280UL, 281UL, 282UL, 283UL, 284UL, 285UL, 286UL, 287UL, 288UL, 289UL, 290UL, 291UL, 292UL, 293UL, 294UL, 295UL, 296UL, 297UL, 298UL, 299UL, 300UL, 301UL, 302UL, 303UL, 304UL, 305UL, 306UL, 307UL, 308UL, 309UL, 310UL, 311UL, 312UL, 313UL, 314UL, 315UL, 316UL, 317UL, 318UL, 319UL, 320UL, 321UL, 322UL, 323UL, 324UL, 325UL, 326UL, 327UL, 328UL, 329UL, 330UL, 331UL, 332UL, 333UL, 334UL, 335UL, 336UL, 337UL, 338UL, 339UL, 340UL, 341UL, 342UL, 343UL, 344UL, 345UL, 346UL, 347UL, 348UL, 349UL, 350UL, 351UL, 352UL, 353UL, 354UL, 355UL, 356UL, 357UL, 358UL, 359UL, 360UL, 361UL, 362UL, 363UL, 364UL, 365UL, 366UL, 367UL, 368UL, 369UL, 370UL, 371UL, 372UL, 373UL, 374UL, 375UL, 376UL, 377UL, 378UL, 379UL, 380UL, 381UL, 382UL, 383UL, 384UL>' requested here
         SetEmptyBranches(fInputTree, *fOutputTree, ind_t{});
...

@vepadulano vepadulano force-pushed the rdf-snapshot-empty-tree-preserve-schema branch 2 times, most recently from fa439d2 to 551ecba Compare April 9, 2025 21:04
@TomasDado
Copy link
Contributor

Thanks for the updates! With the latest changes my code finishes properly and I also see the branches in the output of a Snapshot that has no events passing in a multithreaded mode.

@vepadulano vepadulano force-pushed the rdf-snapshot-empty-tree-preserve-schema branch from 551ecba to 0a1d7c7 Compare April 16, 2025 07:35
Previously, the behaviour of Snapshot in case no entries pass the selections in
the event loop was two-fold:

* In single thread, Snapshot created an output file with an empty TTree inside,
  without the user-requested branches.

* In multi-threaded mode, Snapshot created an output file, without any TTree
  inside.

This commit aligns the behaviour of the two execution modes. Furthermore, it
ensures the user-requested dataset schema is preserved even when the output
TTree is empty. i.e., all the user-requested branches will be present in the
output TTree.
@vepadulano vepadulano force-pushed the rdf-snapshot-empty-tree-preserve-schema branch from 0a1d7c7 to f5d9d72 Compare April 16, 2025 09:01
@dpiparo dpiparo merged commit 037534b into root-project:master Apr 16, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants