[Proposal] Metadata file for experiments?

[YanzhaoW]

Hi,

I would like to ask some opinions whether it's a good idea to have a metadata (JSON) file for experiments (in root folder or in config folder). It could be like:

[
  {
    "exp": "s509",
    "detectors": 
    {
      "neuland": 
      {
        "onspill_tpat": 1,
        "offspill_tpat": 14,
        "NumOfPlanes": 26,
      }
    }
  },
  {
    "exp": "s522",
    "detectors": 
    {
      "neuland": 
      {
        "onspill_tpat": 1,
        "offspill_tpat": 11,
        "NumOfPlanes": 26,
      }
    }
  },
  {
    "exp": "s118",
    "detectors": 
    {
      "neuland": 
      {
        "onspill_tpat": 1,
        "offspill_tpat": 14,
        "NumOfPlanes": 26,
      }
    }
  },
]

The JSON file can then be read by CMake. When configure the cmake, experiment ID can be set with:

cmake .. -DR3BEXP="s118"

and the corresponding C++ global constants can be set accordingly during the compile time. We could also set the default value of R3BEXP to be the latest experiment if not specified.

These are my initial thought. Any suggestions and comments are welcomed.

[YanzhaoW]

Maybe set the expID at compile time is a bad idea. Doing so will bind all binary files of R3BRoot to a specific experiment. It is very bad both for executables and ROOT macros if user isn't aware of which experiment R3BRoot is compiled with. Yeah, so better read the metadata file during the run time.

[jose-luis-rs]

This information should be loaded from a parameter container, something like R3BTpatMappingPar. But the number of planes for Neuland is already in the R3BNeulandMappingPar container. You could also include the parameters "offspill_tpat" and "onspill_tpat" in the latter.

[YanzhaoW]

Ok, but do you think it's a good idea to store it in JSON file (in R3BParam repo), which is human readable. The ROOT parameter file could then be generated from this file?

[jose-luis-rs]

Yes if we can manage it with FairRoot, I am not sure. Here you can see the current format that we are using to store the positions of each detector:
https://github.com/R3BRootGroup/R3BParams_s091_s118/blob/dev/parameters/CalibPar_522.par
This is also human readable. Do you thing that JSON format is better?

[YanzhaoW]

Yes, JSON file is pretty much the standard file format for storing metadata. nlohmann/json has been already included in FairSoft. The problem for text based file(CalibPar_522.par) is human readable but not very readable for computers. For example, if someone needs to plot some data in CalibPar using python or whatever, he needs to write his own parser. But with JSON, since most of programming languages already implemented JSON parsers, all he needs is just use a already existing library.

But we could allow both.

[jose-luis-rs]

No problem, but I prefer to use both; otherwise we will have to rewrite all the existing parameter files.

[YanzhaoW]

I have one question though. Why do we create R3BParams for each experiment instead of just one repository?

[hapol]

Hi @YanzhaoW,
setup and parameters are experiment-dependent, as well as geometries and experiment-related macros, the typical contents of the parameter containers. Repositories can be created from a template and are self-consistent for those who want to analyse one experiment, as it is usually the case for our students.

[YanzhaoW]

Hi, @hapol

I see. But Jose Luis also mentioned we also have some metadata files in text format which specifies some geometrical and triggering information. To transform those metadata files to the parameter file in .root, I guess we are using some programs (or ROOT Macro)? If so, do we also copy those programs into each R3BParam repositories?

[hapol]

Yes, some files could (maybe) be the same on several repositories, maybe some are identical in all repositories... But this should be an exception. Even geometry parameter files could/should be experiment related. A "nominal" representation of the detectors it is included in R3BRoot, but parameters are in general particular of each experiment.
Moving a set of parameters from a text representation to a root representation can be done via macros, using the first and second inputs (setFirstInput, setSecondInput) and the outputs (setOutput) of FairRuntimeDb. Examples of these macros are included in all detector directories of the macros repository, if I remember correctly.
And also in the macros folder of the repositories, of course (I forgot to add and edit).

[jose-luis-rs]

We read the file CalibPar_522.par directly with FairRuntimeDb and modify it by hand during the beam time to calibrate our detectors (especially for particle identification from FRS). Then we can create this file for different data runs (different values for the parameters) and read it with our macros depending on the RunID.
For NeuLAND we work directly with Root files because the calibration of the TDCs requires a lot of information and it makes no sense to produce ASCII files.

[YanzhaoW]

I see. If I understand you correctly, the generation scripts of root file parameters from text files should be independent of each experiment. If so, it's also ok to put the generation scripts in R3BRoot (macros folder)?

[YanzhaoW]

For NeuLAND we work directly with Root files because the calibration of the TDCs requires a lot of information and it makes no sense to produce ASCII files.

If I'm not wrong, neuland TDC calibration only needs few values: number of planes, off spill tpat, coarse time clock frequency (so far always same), fine time channel numbers (so far also same). The trigger mapping can be inferred through the calibration process. So in reality just two numbers are needed. In this case, ASCII files would be a better option in my opinion.

[jose-luis-rs]

The fine time calibration is an array with 1000 values, one value per bin:
https://indico.gsi.de/event/5496/contributions/25455/attachments/18598/23317/FPGA_TDC.pdf
For the calibration of some detectors, such as plastic scintillators at FRS where we only have 3 TDC signals, we store the parameters directly in ASCII files. But for other detectors like fibres, ToFD and NeuLAND the default is a Root file because there are many TDC channels.

[YanzhaoW]

I'm not sure about this. From the NeuLAND calibration process, the relation between the fine time value (in ns) and fine time channels (0~4086?) is determined during the calibration process which only need the input of lmd files. I'm not sure these parameters worth being saved in the repository as they are calculated anyway when we do calibration.