Skip to content

AfterScenario Hook runs out of order vs Step hooks when there has been an error #650

Open
@Johnlon

Description

@Johnlon

👓 What did you see?

We have a test in suite_context_test.go tht makes an assertion on what I consider the wrong behaviour.

		name: "fail_then_undefined_fails_scenario", afterStepCnt: 2, beforeStepCnt: 2,
		body: `
				When step fails
				Then step is undefined`,
		log: `
				>>>> Before suite
				>> Before scenario "test"
				Before step "step fails"
				After step "step fails", error: oops, status: failed
				<< After scenario "test", error: oops
				Before step "step is undefined"
				After step "step is undefined", error: step is undefined, status: undefined
				<<<< After suite`,
	},

Above we can see that the 'AfterScenario' hook is expected called before the scenario is actually over and the remaining steps have their BeforeStep/AfterStep hooks called.

✅ What did you expect to see?

I am expecting the hooks to properly nest.
We should not call the After Scenario until all scenarios have been processed.

I expect this...

				>>>> Before suite
				>> Before scenario "test"
				Before step "step fails"
				After step "step fails", error: oops, status: failed
				Before step "step is undefined"
				After step "step is undefined", error: step is undefined, status: undefined
				<< After scenario "test", error: oops
				<<<< After suite`,

📦 Which tool/library version are you using?

v0.14.1 and also main

🔬 How could we reproduce it?

Just inspect the test cases in TestTestSuite_Run in suite_contexr_test.go

I believe this logic I plain wrong.

📚 Any additional context?

Seems same since at least v0.13

The formatters like events/cucumber look ok - but the hook order is bogus.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions