[bug]: Pipeline cannot be run when package properties script does not exist

[joelostblom]

This sample script...

import seedcase_sprout as sp
from scripts.package_properties import package_properties

def main():
    # Create the properties script in default location.
    sp.create_properties_script()
    # Write properties from properties script to `datapackage.json`.
    sp.write_properties(properties=package_properties)

if __name__ == "__main__":
    main()

...does not work in a fresh repo since package_properties is trying to be imported before the file is created via create_properties_script().

One way to solve it would be a try/except clause in the import but it could easily get a bit convoluted for the users editing this file manually. The necessity to solve this would depend on how common this workflow is. Do we assume that everyone using the package first runs a main.py file that only contains create_properties_script() or do we want fully developed pipelines to be portable into new projects?

[joelostblom]

One possible solution to this could be to move the import logic to inside the write_properties.py module. Then we could use a try/except clause in there that the user doesn't see and that generates an informative error message if the create_properties script has not been run yet. Currently the error is:

ModuleNotFoundError: No module named 'scripts.package_properties'

Which I don't think is very helpful. If we move the logic, we could catch the error and customize the message along the lines of:

FileNotFoundError: The file 'scripts/package_properties.py' does not exist. Either create it, or pass a PackageProperties object directly.

I think this makes sense since we already use properties=package_properties in all the examples I have seen, so making it the default behavior for write_properties when no argument is passed (with the option to pass a PackageProperties directly object to override it)

[lwjohnst86]

Hmm. Or, what about we create the script in the template and remove that function from here? (actually, that might not work... 🤔 )

[lwjohnst86]

This would be nice to have as part of a CLI 😛 just for this part though.

[joelostblom]

My initial thought is that it is convenient for someone to be able to generate the file from scratch even without the template. If the idea is that main.py. is a standalone pipeline that can be re-run in a new project, then I think it needs to be self-contained and create all the necessary files. And as you pointed out, I'm unsure what would happen to the autogenerated metadata if we would move this file to the template.