Error behaviour for when a parallel faas-cli build fails

[alexellis]

This issue discusses error behaviour for when a parallel faas-cli build fails.

I was building the samples in the main faas repo using the YAML file, one of them failed and printed red text but it was really hard to see and I could have missed it.

I thought we had already done work to break out upon an error and give a non-zero exit code.

How do we want this to work?

Scenario 1 - sequential

You're building 5 functions sequentially from a YAML file - number 3 fails, should we break out and stop?

Scenario 2 - parallel

You're building the same stack with --parallel=4 and output is intermingled. Do we break out all builds as soon as one of them produces an error?

Solutions

• Break out of either flow upon first error
• Push on and summarise a list of functions built in an ASCII table:

function	time taken	build
hubstats	1m 10s	error
markdown	59s	OK

Other considerations

Headless use of faas-cli build in CI

@developius @itscaro @rgee0

[tommysolsen]

Current behavior is a little all over the place. For example missing language fields in the yaml will not cause a halt, but if docker returns any non-zero codes, the software throws a fatal exception and immediately exits. Since this is done through a fmt.Fatal call it will halt even parallel builds.

Breaking on error

An argument in favor of breaking on error is that the relevant error messages are most likely close(st) to the bottom of the screen. Making it potentially easier to find the information relevant to the build error.
One negative case that could occur with this solution is if you have errors in multiple functions and you are building in parallel, you might end up halting on different errors, since the order of which go routine run first is unpredictable. This is especially true once the layers in the Dockerfile before the error has been cached. This could be pretty confusing for users.

Continue on error

A big advantage of letting each build run would be that we could successfully cache everything that is OK, reducing the time the next build command would take after you have fixed the errors that failed in the previous attempt. The clutter would be worse, especially on parallel builds. I've made a build that continues on error(but returns non-zero code if errors occured), that produce a list like this after all functions have been attempted built.

Build summary
            Name | Build  | Error
-----------------+--------+--------------------------------------------------------------------------
dockerfile_error | FAILED | Execution failed: [docker build --no-cache -t fails .]
      wrong_lang | FAILED | Language template: COBOL not supported. Build a custom Dockerfile instead
     all_missing | FAILED | Please provide a valid --lang or 'Dockerfile' for your function
       testbuild | OK     | None 

Build exited with 3 errors

[alexellis]

I take what you're saying and appreciate the analysis. I would like us to resolve the current false positive behaviour I observed first and work from there with a redesign on the flow.

[tommysolsen]

As far as I can gather, you got an actual error, not a false positive. Current behavior just bury the error, because the other workers still continue to run and print to screen. I've sent a PR that will break out and exit with the correct code.

[alexellis]

@Razesdark thank you for working on this, it still needs some sort of solution but I think the new solution is too complex and could be hard to maintain. Let's go back to the drawing board or see if we can simplify the solution.