Travis Conditional allow_failure MASTERS on release/v0.x

[dehann]

Rather than hard yes or no on DEV MASTERS when running Travis for release/v0.x branches, perhaps there is a way to do a conditional allow_failure setup where stable releases are still tested against upsteam masters, but a failure is allowed.

Suggestions / Ideas

• Best guess:
  • https://docs.travis-ci.com/user/conditional-builds-stages-jobs#conditionally-allowing-jobs-to-fail
  • DF +1
• Another idea is to have to - jobs which are nearly identical for DEV MASTERS case, the one includes an allowable failure flag, the other doesn't. Only one of them runs in the case of release/v0.x and the usual (does not allow failures) in all other cases.

Related

• https://docs.travis-ci.com/user/conditional-builds-stages-jobs
• drop dev masters test on release branch RoME.jl#312

[dehann]

FYI JuliaRegistries/TagBot#163

[Affie]

I'm not completely following the issue.

I had similar conditional failures before and I added it as a comment for future reference (so I don't have to figure it out again):

IncrementalInference.jl/.travis.yml

Lines 30 to 32 in 91d97be

	# Keep this to allow failures again when DFG minor revision is updated
	# - name: "MASTER AND TAGS"
	# if: NOT branch =~ ^release.*$

The problem with this was it's only needed if the minor version changes while in the 0.x versions. So ideally it should look at the tags also. Perhaps we can find a way to bring in patch/minor. It seems to be a common problem for a lot of the larger split ecosystems, such as Images.jl

Better names for what we have could be This and Masters and This and Tags
Currently:

• This and Masters
  • Not tested on release branches
• This and Tags
  • No failures allowed making the test fail if there is a minor release but can be manually checked on This and Masters

So perhaps the solution is "One Shall Pass"

Are you suggesting:

• This and Masters
  • Allow failures on release branches
• This and Tags

I'm against running so many tests if it's not needed (takes too long).