Skip to content

Improve experience for comparing floating point/decimal numbers #74

@chadlwilson

Description

@chadlwilson

Context / Goal

As noted in #39 it generally doesn't make sense to be using columns with floating point types as an input to tests, since their exact values are likely to be different across databases due to rounding/precision differences, and even the way they are then converted to Java types.

We should probably either

  • completely remove support for these values
  • or warn the user if they are trying to compare column values of these types
  • or allow the user to configure some approach of precision

Expected Outcome

  • Decide and design an approach to deal with decimal types
    • If we can figure out a way for precision to be configured we would need to change the HashingStrategy to take this precision and coerce the value to a long before adding to the hash; with the value rounded to this precision.
    • It probably only makes sense to express this precision in terms of significant figures

Out of Scope

Additional context / implementation notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions