`rootls` improvements

[Axel-Naumann]

Explain what you would like to see improved

We have a proposal for an improved rootls.

After discussion we decided to implement missing features in rootls - for instance to keep it in Python (platform-independent terminal interaction) and to continue to make the code available to the other command line utilities.

In our usage of rootls, what's missing? Let's collect this in comments below.

Optional: share how it could be improved

See comments below

To Reproduce

See comments below

Setup

ROOT master, please - there are some +/- recent changes.

[Axel-Naumann]

Speed

It's actually surprisingly slow, when listing a couple of histograms. time rootls -l -t ~/build/root/master/rwd17/tutorials/hsimple.root takes 0.7s on a fairly fast machine.

[Axel-Naumann]

Details

It'd be nice to show

• number of entries, number of entries outside range, mean, stddev for histograms; maybe sumw2 to signal whether that's calculated or not
• TFile's compression algo and level
• branches' data type, compression ratio, size in bytes; compression level and algo if different from TFile's
• number of TTree entries

@guitargeek what's missing for RooFit classes?

[mxxo]

RNTuple "tree" output

e.g. current output just lists them as opaque objects:

$ rootls -t test80X_NANO_rntuple.root 
ROOT::Experimental::RNTuple  May 27 19:09 2021 Events;1           "object title" 
ROOT::Experimental::RNTuple  May 27 19:09 2021 LuminosityBlocks;1 "object title" 
ROOT::Experimental::RNTuple  May 27 19:09 2021 MetaData;1         "object title" 
ROOT::Experimental::RNTuple  May 27 19:09 2021 ParameterSets;1    "object title" 
ROOT::Experimental::RNTuple  May 27 19:09 2021 Runs;1             "object title" 
TObjString                   May 27 19:09 2021 tag;1              "Collectable string class" 

[eguiraud]

• rootls does not print enough information about a TTree, rootls -t prints too much: all cluster ranges in the file, which might be a lot
• rootls -t f.root prints information about each namecycle of a tree -- that's almost never what the user wants
• the output format of rootls -t is difficult to parse by eye. compare:

~ rootls -t $ROOTSYS/tutorials/dataframe/df017_vecOpsHEP.root
TTree  Mar 27 15:37 2018 myDataset;1 "myDataset"
  nPart  "nPart/I"      89
  px     "px[nPart]/D"  2318
  py     "py[nPart]/D"  2318
  E      "E[nPart]/D"   2317
  Cluster INCLUSIVE ranges:
   - # 0: [0, 2]
  The total number of clusters is 1

with e.g.

> rootbr $ROOTSYS/tutorials/dataframe/df017_vecOpsHEP.root
TTree myDataset [3]
├── Int_t nPart
├── Double_t px: px[nPart]
├── Double_t py: py[nPart]
└── Double_t E: E[nPart]

(gets worse with multiple trees or namecycles of a tree:

/tmp rootls -t f.root
TTree  Jun 07 21:29 2021 t;2 "t" [current cycle]
  x  "x/I"  138
  Cluster INCLUSIVE ranges:
   - # 0: [0, 1]
  The total number of clusters is 1
TTree  Jun 07 21:29 2021 t;1 "t" [backup cycle]
  x  "x/I"  69
  Cluster INCLUSIVE ranges:
   - # 0: [0, 0]
  The total number of clusters is 1

)

[vepadulano]

To give a bit more context, rootls was even more confusing before addressing #7053 since it was printing multiple namecycles of a objects without stating they were multiple namecycles

Example with ROOT 6.22.08

[~]: rootls -l myfile.root 
TProfile  Jun 12 10:00 2021 hprof   "Profile of pz versus px"
TProfile  Jun 12 10:00 2021 hprof   "Profile of pz versus px"
TH1F      Jun 12 10:00 2021 hpx     "This is the px distribution"
TH1F      Jun 12 10:00 2021 hpx     "This is the px distribution"
TH2F      Jun 12 10:00 2021 hpxpy   "py vs px"
TH2F      Jun 12 10:00 2021 hpxpy   "py vs px"
TNtuple   Jun 12 10:00 2021 ntuple  "Demo ntuple"
TNtuple   Jun 12 10:00 2021 ntuple  "Demo ntuple"

Example with ROOT master@fa3677a8e579e14feb72efaef381ac00514dff77

[~]: rootls -l myfile.root 
TProfile  Jun 12 10:00 2021 hprof;2  "Profile of pz versus px" [current cycle]
TProfile  Jun 12 10:00 2021 hprof;1  "Profile of pz versus px" [backup cycle]
TH1F      Jun 12 10:00 2021 hpx;2    "This is the px distribution" [current cycle]
TH1F      Jun 12 10:00 2021 hpx;1    "This is the px distribution" [backup cycle]
TH2F      Jun 12 10:00 2021 hpxpy;2  "py vs px" [current cycle]
TH2F      Jun 12 10:00 2021 hpxpy;1  "py vs px" [backup cycle]
TNtuple   Jun 12 10:00 2021 ntuple;2 "Demo ntuple" [current cycle]
TNtuple   Jun 12 10:00 2021 ntuple;1 "Demo ntuple" [backup cycle]

I have always found that showing namecycles makes the output much more difficult to understand in general (even with TFile::ls at the prompt). At the same time namecycles are a thing and some users might want to see them, so I'd be inclined to add a specific extra option for that such as --namecycles or -n.

Or one could see it from the other side and say that the current -l option is already the correct one to show multiple namecycles since "longListing" should include every extra detail possible about the objects in the file (including namecycles).

If we go for this route though I would still say that the -t option is a bit confusing since it is effectively a -l option but just for trees.

vpadulan@fedorathinkpad-T550 [~]: rootls -h
[...]
  -l, --longListing  Use a long listing format.
  -t, --treeListing  Print tree recursively and use a long listing format.

[eguiraud]

rootls -t also has a problem with long lines. this is what happens with a file with certain branches that have long names: