bug(API): starting DAG fails if DAG-level the directory specified in `workingDir` does not exist

[yottahmd]

[ghansham]

We should not allow such dags to even get parsed. We should give a good bold error message saying "working directory does not exist. Fix the workingDir field". We can also put a check on permissions of this workingDir, if it is readable at the least.

[yottahmd]

That sounds good for a static working directory case. What do you think about variable case? For instance:

params: DIR=
workingDir: ${DIR}
steps:
  - do something in $DIR

[ghansham]

See point no. 3 mentioned here (#1252):

#1252 (comment)

Time to start bringing directory and file parameter validation in the json yaml schema 😀

Lets put our brains together to handle such cases

[yottahmd]

I totally agree on adding validation for directories. Perhaps it would be desirable to run it when the DAG executes to keep variable support. Thanks for sharing the previous discussion link!

[ghansham]

I feel if we introduce schema logic for dag params, it will be user responsibility to get the schema right.

It would be better, if we validate workinDir (and more such dag entries which can be added gradually on need basis) is variable based (I think it is doable) and then do a validation on the computed value.

If we introduce initHandler, we can compute such variable values in the initHandler and if the workingDir is a determined dynamically based on parameters, we can compute the exact value of workingDir in the initHandler and fail it giving an error message working directory value as determined dymamically is xxxx which either does not exist (or is not readable) if it doesn't pass the validation criteria. And display it on the UI side.

That means a default initHandler will always exist and if user specifies one in th dag, it should be called after calling the default one.

I think this approach may be quite a bit better than json schema based approach as dagu can take control the business (sanctity of parameters being passed).
WDYT?

If you want, we can help you in this.