Skip to content

Create repo to end-to-end testing hooks #823

Open
@MaxymVlasov

Description

@MaxymVlasov

About hook examples and how to test hooks.
I don't think that unit tests (hello python) or fuzzing tests (hello OSSF, and maybe, python too) will help with that much. We will probably need to add some e2e tests by creating a separate repo with a bunch of code samples and a bunch of isolated hooks runs.

The good news is that we can do it right now (and I'll probably do it anyway, glad you asked, as I tried to test changes somehow locally with the hope that I cover everything - that's far from ideal)
Bad news - could be hard to automate it right now, but at least it will be a good starting point.

Originally posted by @MaxymVlasov in #822 (comment)

Acceptance criteria:

  • there test cases for passing hooks
  • There test cases for failing hook and we need to compare end result with how it should look (as we know it should look)
    git diff between "good" and resulted fixed "bad" examples?
  • each test is just run in .pre-commit-config.yaml which limited to 1 directory by files:
  • That repo should contain instruction how to run it locally for testing proposes (using our docker image as it has all tools in place?)
    OR be integrated inside this repo:
    git workflow with git clone, change rev: in .pre-commit-config.yaml to current commit, run as action, here simpler example - MaxymVlasov/dive-action/.github/workflows/test.yml. Probabbly make sense to provide --config and maintain separate config for each test case right into dir wit test

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions