Boosting 4C Input and User Experience using YAML

[gilrrei]

Here are the slides of the 4C input presentation presented by @lauraengelhardt and @sebproell Boosting_4C_Input_and_User_Experience_using_YAML.pdf

If you have questions, ideas, or comments, please add them here in this discussion.

[mayrmt]

One more rather central question came to my mind: how can I generate a valid yaml-based input file from scratch?

[mayrmt]

I do not know any people, who write their input files by hand. So, IDE support appears to be nice, but does not help me to create input files. Furthermore, I will not write the ASCII mesh information in the input file, but have to rely on a tool to do that for me. So far, this is pre_exodus.

Let me ask more precisely: how can I setup a new model incl. the mesh?

[amgebauer]

With https://github.com/amgebauer/lnmmeshio you can generate the mesh from many mesh formats. Probably you can also use cubitpy and meshpy (I'm not an expert in these). Then, you setup materials, boundary conditions and parameters according to our documentation and include the generated mesh in our legacy format with the INCLUDE section. I don't see a problem.

pre_exudus is a workflow that has been unchanged decades ago and is not proof for the future. It anyways does not really help you in generating an input file from scratch.

[mayrmt]

pre_exodus is used to generate input files from scratch. It works very well. Furthermore, it is the only way of creating input files, that is documented within 4C.

[sebproell]

Short answer: right now, you can use pre_exodus to generate a .dat and convert that to .yaml.

That's probably not what you meant though. pre_exodus takes three things: .exo will of course stay that way, .bc which is a custom format to match up some conditions with data stored in .exo, and .head which is copy-pasted at the top of the input file. It would be trivial to teach pre_exodus to take the header as .yaml and paste that into the final input file and write the combined info from .bc and .exo in our legacy dat-style string into yaml. Editing the header file (in yaml format) would support all the goodness we showed in the TGM. If there is a demand for this workflow, I am happy to implement it. I was just under the impression that pre_exodus sees little use. Long-term we will remove pre_exodus for #60. As I already said in #263, I am open to kicking the mesh info and pre_exodus out first and then remove all .dat stuff.

[mayrmt]

@sebproell, yes, generating dat and then converting to yaml work, but I do not consider this a great way to create input files.

I do not know, if there is demand (beyond myself). When I create input files (and I admit that this does not happen very often), I use pre_exodus. I don't know, how most other 4C users create their input files.

I agree that pre_exodus could be fed with yaml instead of .head. Aren't boundary conditions also part of the yaml input though?

Although I personally am less enthusiastic about yaml than others, I can see its benefits and do not object to move towards it. My main concern right now is, that there's a documented way of doing things and then there's the way, that is currently be pushed to become the new standard. We have documentation and testing for creating dat files using pre_exodus. We do not have documentation on how to create input files using yaml (expect for creating dat and converting to yaml).

[ischeider]

Trying the new yaml features, I struggled with two "features" of the code where I misinterpreted the error:

1. It would be good if it was not necessary to enter on output file when running 4C --to-yaml <.dat-file>. I expected that the program could generate the same filename with a .yaml extension. Unfortunately, the software did not test the existence of a second parameter, but simply gives Signal: Segmentation fault (11)
2. When having a typo in the program call, so that the input file does not exist, I got the error message Error message: The input file '16ele_2coupl.yaml' must contain a map as root node. (My problem was that the file name was 16ele_2coup.yaml). Of course it would be good if the program told me that the input file does not exist.

I can modify the code for these two deficiencies (or what I see as a deficiency), or we can collect a few more and create a pull request then.

[amgebauer]

Thanks @ischeider for reporting these bugs. Feel free to create a PR fixing these bugs 👍

[sebproell]

@ischeider Second one is definitely a bug. The first one should give a proper error if the second arg is missing. (Our command line args handling is very messy and I just hacked this new option in there.) Personally, I would not want to assume an output file name and always pass it in, but I am open to using the same location as the original file and just change the extension. However, I am not sure if we should silently overwrite an existing file for this case.

Just my thoughts. Would be great if you could open a PR with your envisioned behavior 👍

[georghammerl]

Dear interested developers in 4C input,

I would like to setup a meeting to discuss unclear/open points and to decide on how to progress with yaml input and dat deprecation.
Please fill the poll:
https://terminplaner6.dfn.de/p/a1cf306fa2786b6f102688885403c21b-1146534
such that I can setup a meeting. Everyone is welcome to this meeting. Thank you in advance.

[gilrrei]

@georghammerl Could you elaborate a bit more on the actual agenda?

unclear/open points

Sounds like a Q&A. This is unspecific and hard to answer on the spot. Personally, I think that the various discussions #490 and #263, as well as issues #317, #514, #529, and #74 (probably more of them) or under the label team: input, are more suited to answer specific technical questions. This allows other users to see the answers as well, ask follow-up questions, introduce new ideas, and avoid political discussions.

how to progress with yaml input and dat deprecation

How is this different than #263? I don't fully see what specifically is to be discussed in the meeting, i.e., what are the decisions wanted at the end of the meeting?

As an input enthusiast, I would like to participate, but without a clear agenda my enthusiasm transforms into abstract philosophical input discussions taking up valuable meeting time 😅

[georghammerl]

@gilrrei Thank you for your comment. You summarize very nicely all the effort that has been made in writing.

Now it's a matter of consolidating all the content to come to a common agreement. I prefer to have the whole community in the same boat :-)
Secondly, it's about prioritizing the aspects involved and creating a timeline for how we can proceed with as little friction as possible.
If we find solutions very quickly in a meeting like this, I'm the last person to prolong it and let you get back to spending your time on other valuable tasks.

[sebproell]

For a productive conversation, the people behind most of the recent changes (@lauraengelhardt, @gilrrei, @sebproell) summarized a potential roadmap here: #263 (comment)

We would ask potential participants of a meeting to critically examine this proposal. If there are doubts or questions, please raise them in that discussion. If we cannot resolve the discussion, we can then have a meeting with a clear list of items to be discussed. This also gives us the opportunity to prepare potential solutions for issues raised.

[georghammerl]

The poll for a meeting started in this discussion, so for completeness, here also information on the scheduled meeting (cf. #263 (comment)):

In order to discuss especially Phase 2 (details, time line) of the Recap (#263 (comment)) and clarify potential misunderstandings, here is an invitation to a Zoom meeting on April 1st, 15:30 - 16:30:

https://tum-conf.zoom-x.de/j/69826674614?pwd=T2gYCfeW0RBHH1Avsli4mcLs6JXLOo.1

Please feel free to join. Meeting notes will be provided afterwards.

[gilrrei]

As already mentioned #263:

We've created a project with repo FourCIPP (coauthored by chatgpt):

The FourCIPP repository will hold a Python parser for 4C input data, designed to facilitate the transition to YAML for all 4C Python projects. This tool serves as the backend for 4C input Python projects, providing a streamlined approach to data handling.
It utilizes the 4C_metadata.yaml file to ensure that projects remain up-to-date and consistent with the latest features. FourCIPP aims to enhance the efficiency of 4C data processing while promoting the use of YAML in project workflows.

We'll add features step by step, but the goal is, of course, to handle everything 4C input. Once we have a good working version we'll provide more information, in the meantime, everyone who's interested in this tool can participate, from ideas to coding, contributions are welcome :)