Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve testing of diagnostic comparer #36455

Merged
merged 5 commits into from
Feb 11, 2025
Merged

Conversation

SarahFrench
Copy link
Member

@SarahFrench SarahFrench commented Feb 6, 2025

This PR is part of recognising that tests compare diagnostics in different ways in different places. It's unclear if this is due to a need or just due to varying approaches being used in different parts of the code base.

This PR updates the tests for the 'comparer' in ./internal/tfdiags/compare.go to make it clearer about what to expect from the comparer:

  • Some data like sources is ignored when comparing diagnostics
  • Different concrete value types under the tfdiags.Diagnostic interface
    • Note: Often when developers compare diagnostics they convert the variables to be an RPC-friendly diagnostic struct type, as comparison through cmp raises diffs if the concrete type is different.

I'm opening a related PR that explores refactoring existing tests to use a new set of test helpers in the tfdiags package: #36456 . Those test helpers will allow users to convert diagnostics to RPC-friendly diagnostic struct types and compare them using the comparers in this PR.

Target Release

N/A

CHANGELOG entry

  • This change is user-facing and I added a changelog entry.
  • This change is not user-facing.

@SarahFrench SarahFrench added the no-changelog-needed Add this to your PR if the change does not require a changelog entry label Feb 6, 2025
// Example usage:
//
// cmp.Diff(diag1, diag2, tfdiags.DiagnosticComparerWithSource)
var DiagnosticComparerWithSource cmp.Option = cmp.Comparer(diagnosticComparerWithSource)
Copy link
Member Author

@SarahFrench SarahFrench Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for reviewer - do you know of any tests that would need/want to assert about the source data in a diagnostic? If we can't find a use for this then it should be removed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to know too. Otherwise, I wonder if we should wait for such use case before building such a comparer

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty much anything in the configs and stackconfig packages should care about the source data, as it is important in those cases that the errors point to the part of the configuration that is invalid. An example of this would be the tests here: https://github.com/hashicorp/terraform/blob/main/internal/configs/config_test.go#L542-L582. But, there should be lots of examples in those packages.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the tests in configs use hcl.Diagnostic/hcl.Diagnostics types and in stackconfig the diags comparison uses sourceless diags in assertions. So overall it looks like this comparer isn't needed. I'll slim down the PR to just the edits to the existing comparer and then open up the stacked PR for review too.

@SarahFrench SarahFrench changed the title Improve diagnostic comparison: simple and 'fuller' comparison options Improve diagnostic comparison: simple and 'with sources' comparison options Feb 6, 2025
@SarahFrench SarahFrench marked this pull request as ready for review February 6, 2025 21:48
@SarahFrench SarahFrench requested a review from a team as a code owner February 6, 2025 21:48
@SarahFrench SarahFrench changed the title Improve diagnostic comparison: simple and 'with sources' comparison options Improve testing of diagnostic comparer Feb 11, 2025
@SarahFrench
Copy link
Member Author

Thanks!

@SarahFrench SarahFrench merged commit dcce20f into main Feb 11, 2025
8 checks passed
@SarahFrench SarahFrench deleted the sarah/diags-comparisons branch February 11, 2025 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-changelog-needed Add this to your PR if the change does not require a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants