Breaking API check never fails

[jakemac53]

I am trying to add the breaking API check here dart-lang/build#3760.

While in the comment it does notice the breaking change, and the job also sees the breaking change, it still passes, and also in the comment it is listed with a green checkmark.

I should be forced to do a breaking release?

[mosuem]

It seems like you found the fix already, adding breaking to the fail_on list in the settings.

[jakemac53]

It seems like you found the fix already, adding breaking to the fail_on list in the settings.

I did this but it didn't actually make the job fail still

[mosuem]

It does pass now, as the version is correct. Can you point to the invocation which incorrectly passed?

[jakemac53]

https://github.com/dart-lang/build/actions/runs/11150849769/job/30992945328?pr=3760 is the job which passes, but should be failing. It should be requiring a breaking release (5.0.0) but I only did a feature release.

[mosuem]

Why should it be failing? The comment says it needs a minor version bump, which was made. What am I missing?

[jakemac53]

Oh interesting, it was definitely a breaking change.

I widened the return type of a function, so users of that function would be broken.

So, I guess the issue is actually that widening return types should be considered breaking and cause a major version.

[mosuem]

@devmil - Widening the return type seems to be a minor breaking change at the moment - should it be a major breaking change instead?

[devmil]

@devmil - Widening the return type seems to be a minor breaking change at the moment - should it be a major breaking change instead?

I agree. I created an issue to look into this

[devmil]

@devmil - Widening the return type seems to be a minor breaking change at the moment - should it be a major breaking change instead?

Widening the type already should lead to a breaking change and looking into the CI run (https://github.com/dart-lang/build/actions/runs/11150849769/job/30992945328?pr=3760) it looks like dart_apitool detected that change as breaking?

[mosuem]

Hm. But the version part of the report seems to indicate that the version should only be bumped from 4.1.0 to 4.2.0, or am I overlooking something?