repo-wide WDL conventions

[mlin]

Agree upon, and document conventions:

• consistent scheme for controlling vg docker image version throughout workflows
• tasks/workflows factorization, and import directories
• camel-casing vs underscore names (uppercase or lowercase)
• Travis testing

[edawson]

I've been playing with imports and organization over in https://github.com/edawson/dawdl. I like the separation of tasks, workflows, and dockerfiles (when they're needed). Version tagging in Docker is something I'm still trying to figure out; in the past I have stuck to "latest". Maybe we use a coded date / time for builds somehow?

Camel casing seems the right answer (as I've bumped into weird WDL issues with underscores). I have gravitated towards upper camel-case and adding the "Task" suffix to tasks.

Imports on FireCloud (and Cromwell, the Broad's WDL runner) fail unless the tasks being imported are pushed to the FireCloud repos. I need to write some wrapper scripts for pushing / making public / updating task import versions.

[mlin]

👍 to keeping dockerfiles next to WDLs when possible. I have a long term idea that the dockerfile should actually be embedded in the WDL (i.e. a 'docker_image' concept alongside 'workflow' and 'task'), but need to finish miniwdl first :)

Re docker image versioning, in vg_docker we tag each image build based on the git tag/revision. So it's traceable to the exact source code version used to build it, and also assuredly sticks around when it's no longer the latest version.

I can get on board with camel casing even though it's not my first choice. From a miniwdl perspective, I'd be curious if you can elaborate on the underscore and import issues you mention.

I'd like to aim towards getting all the WDL in this repo automatically tested on small examples inside Travis CI (or some other future public CI), using Cromwell and/or eventually miniwdl. This forces us to embed all knowledge/assets for how to invoke the workflow in the repo, where otherwise it's too easy to forget something or leave it implicit. What I want to do is get as much of what's here now under such testing as practical, and move everything else into some kind of "incubator" or "needs_testing" folder.

[cmarkello]

@mlin
I like your WDL "incubator" idea. I've been hesitant on throwing the tasks that I've built in my vg_multi_map_call.wdl workflow over to the vg_wdl/wdl repo directory since I'm not sure what the best conventions for task building are without input from you guys.