Skip to content

[FEATURE]: Add Performance benchmark #56

Open
@mwojtyczka

Description

@mwojtyczka

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

There are no performance benchmarks available for profiler and quality checks.
DQX is using native spark apis for quality checking so the impact is minimal. Nevertheless, it would be useful to have performance benchmarks.

Proposed Solution

Create performance benchmarks on how the performance is impacted by running profiler and applying quality rules on bigger data sets.
This should be written as performance tests that can be run weekly or monthly.

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions