feature request: accept YAML as an input format

[a-frantz]

The WDL SPEC requires engines support JSON as a WDL input format, but YAML is (subjectively) a nicer format and I'd like us to support it.

run and validate-inputs would need to be updated to handle YAML in addition to JSON. Being able to output YAML from inputs (#66 ) should also be supported.

[Gyan-max]

Hi @a-frantz,

I’m interested in working on this issue. I plan to update the run and validate-inputs functions to support YAML alongside JSON. Additionally, I’ll ensure that YAML output is supported as discussed in #66.

Would you prefer a specific YAML parsing library for this implementation, or should I proceed with a commonly used one like serde_yaml? Let me know if you have any guidelines or preferences!

Looking forward to your feedback.

Thanks!

— Gyan-max

[a-frantz]

For starters, I would focus on run and validate-inputs, as the inputs command is still in development (#66). Hold off on doing anything with the inputs command till that merges.

serde seems fine to me!

[Gyan-max]

Hi @a-frantz,

I've implemented the YAML input support feature requested in issue #68 and submitted a pull request for review.

In my implementation, I've:

1. Added support for YAML input files to both the validate-inputs and run commands

2. Created a new utils module with functions to:

   • Detect YAML files based on their extension (.yaml or .yml)
   • Convert YAML content to JSON format
   • Create temporary JSON files from YAML inputs

3. Updated the command documentation to indicate that both JSON and YAML are now supported

4. Added appropriate logging to track the YAML to JSON conversion process

The implementation converts YAML to JSON internally, which allows the existing code that expects JSON to work without modification. I've tested this with both JSON and YAML input files and confirmed that everything works as expected.

I've also created a simple WDL workflow and corresponding JSON/YAML input files for testing purposes.

The PR is available here: #69

Please let me know if you'd like me to make any changes or improvements to the implementation.

Thank you for the opportunity to contribute to this project!

Best regards,
Gyan-max

[a-frantz]

Oops, I was confused here! We haven't ported the wdl run command to sprocket yet! That was probably confusing for you 😅

Please delete all mentions of the run command! This PR can just implement it for validate-inputs. Sorry about the mix up.

[Gyan-max]

@a-frantz,

Thanks for the clarification! I've updated my implementation to focus only on adding YAML support for the validate-inputs command.

I've removed all mentions and implementation of the run command from my PR. The changes now include:

1. Added support for YAML input files to the validate-inputs command

2. Created a utils module with functions to:

   • Detect YAML files based on their extension (.yaml or .yml)
   • Convert YAML content to JSON format
   • Create temporary JSON files from YAML inputs

3. Updated the command documentation to indicate that both JSON and YAML are now supported

4. Added appropriate logging to track the YAML to JSON conversion process

The implementation converts YAML to JSON internally, which allows the existing code that expects JSON to work without modification. I've tested this thoroughly with both JSON and YAML input files.