Skip to content

Cleanup in appropriate order when running parallel execution#38247

Open
dsa0x wants to merge 3 commits intomainfrom
sams/tf-test-parallel-deps-cleanup
Open

Cleanup in appropriate order when running parallel execution#38247
dsa0x wants to merge 3 commits intomainfrom
sams/tf-test-parallel-deps-cleanup

Conversation

@dsa0x
Copy link
Member

@dsa0x dsa0x commented Mar 7, 2026

This PR started with me tracking a bug where in parallel mode, we still had some dependency ordering based on the sequence of runs.

For example,

test {
  parallel = true
}

run "test" {
  state_key = "main"
  variables {
    id = "test"
    unused = "unused"
  }
}

run "test_two" {
  state_key = "state2"
  variables {
    // This dependency is a later run, but that should be fine because we are in parallel mode.
    id = run.test_three.id

    // The output state data for this dependency will also be left behind, but the actual
    // resource will have been destroyed by the cleanup step of test_three.
    unused = run.test.unused
  }
}

run "test_three" {
  state_key = "state3"
  variables {
    id = "test_three"
    unused = run.test.unused
  }
}

run "test_four" {
  state_key = "main"
  variables {
    id = "test"
    unused = "unused"
  }
}

In the test above, run.test_three is destroyed before run.test_two, but that is wrong. test_two depends on test_three, so test_two should be destroyed first, i.e reversing the order of creation.

Having fixed that in f6f234f, I realized that the original implementation was unnecessarily complex. Instead, what I have done now is only connect references between the last runs of each state key. These last runs represent the last applied state.

In the above, the state_key main will contain the state applied after evaluating run.test_four. Therefore, any reference to run.test (which also used the state_key main) do not need to be taken into account when connecting cleanup nodes, as run.test_four is the "owner" of the cleanup process.

Target Release

1.15.x

Rollback Plan

  • If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

CHANGELOG entry

  • This change is user-facing and I added a changelog entry.
  • This change is not user-facing.

@dsa0x dsa0x marked this pull request as ready for review March 9, 2026 08:04
@dsa0x dsa0x requested a review from a team as a code owner March 9, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant