Skip to content

feat: Add support for multiple invariants in CLI commands #1662

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

jesicaMao
Copy link
Contributor

  • Tests added for any new code
  • Documentation added for any new functionality
  • Entries added to the respective CHANGELOG.md for any new functionality
  • Feature table on README.md updated for any listed functionality

This PR implements the feature requested in issue #1573 to support checking multiple invariants and reporting which specific ones were violated.
Changes include:
Added support for the --invariants CLI option to accept a list of invariants
Combined multiple invariants with logical AND for evaluation
Implemented detailed reporting of which specific invariants were violated
Updated documentation with examples showing how to use the feature
Added support for both simulator and verification modes

Fixes #1573

@jesicaMao
Copy link
Contributor Author

@bugarela Could you kindly check #1662 . Please let me know if I need to change anything.

Copy link
Collaborator

@bugarela bugarela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, some preliminary comments. I'm getting some compilation issues, also reported by the CI. Was this working for you?

Comment on lines 325 to 326
const guessedMainModule = guessMainModule(prev)
const mainName = prev.args.main || guessedMainModule
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is not necessary, as guessMainModule already returns prev.args.main if provided.

Suggested change
const guessedMainModule = guessMainModule(prev)
const mainName = prev.args.main || guessedMainModule
const mainName = guessMainModule(prev)

Comment on lines 521 to 522
const guessedMainModule = guessMainModule(prev)
const mainName = prev.args.main || guessedMainModule
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const guessedMainModule = guessMainModule(prev)
const mainName = prev.args.main || guessedMainModule
const mainName = guessMainModule(prev)

@jesicaMao jesicaMao force-pushed the feat/multiple-invariants-reporting branch from a7d8ca7 to 8e0c3de Compare May 21, 2025 17:33
@jesicaMao
Copy link
Contributor Author

@bugarela I have made the changes. Could you kindly check and let me know if everything is alright?

@bugarela
Copy link
Collaborator

Hi, sorry for the delay. Seems like there are still compilation issues. Are you able to successfully run npm run compile locally?

@jesicaMao
Copy link
Contributor Author

Hi @bugarela. I did run npm run compile locally after making the changes. Let me check again and I will let you know as soon as I can. Thank You.

@jesicaMao
Copy link
Contributor Author

@bugarela please check if everything is alright now.

@bugarela
Copy link
Collaborator

Hi @jesicaMao! Awesome, it seems to work now 🎉 thanks for seeing this through.
The failing tests are a simple thing: you are adding one more "artificial" definition, and the test checks the definition count, so we just need to update the expectation.

I can take this on from here and get it merged today/tomorrow if that's ok for you. Again, thank you!!

@bugarela
Copy link
Collaborator

Ok, I'm testing things locally and turns out this is never loading the final state of the trace to evaluate the invariant there. It is evaluating the invariant on a completely new state (new evaluator), so it doesn't really work for invariants that reference the state (which are all real invariants in practice):

$ quint run ../examples/language-features/counters.qnt --invariants "n >= 0" "n < 10"
An example execution:

[State 0] { n: 1 }

[State 1] { n: 2 }

[State 2] { n: 3 }

[State 3] { n: 6 }

[State 4] { n: 12 }

[violation] Found an issue (9ms).
Violated invariants:
Use --verbosity=3 to show executions.
Use --seed=0x116c719f7aa6a5 to reproduce.
error: Invariant violated

(this was supposed to print "n < 10")

I have a pretty good idea how to fix this, but it is not something very trivial. I'll find sometime between this and next week to try it. I think the changes you made to the UI are super helpful already and a great head start, but this evaluation part is tricky.

@jesicaMao
Copy link
Contributor Author

@bugarela Thanks a lot for your valuable feedback. I am overwhelmed to know my changes were useful. Please let me if I need to change anything else.

@bugarela
Copy link
Collaborator

I fixed it and slightly changed the formatting:
image

It also works on quint verify :D
image

@bugarela bugarela force-pushed the feat/multiple-invariants-reporting branch from bee7430 to 10af83f Compare May 30, 2025 20:35
@bugarela bugarela enabled auto-merge May 30, 2025 20:39
@bugarela bugarela disabled auto-merge May 30, 2025 20:39
@bugarela bugarela merged commit e7b857f into informalsystems:main May 30, 2025
28 checks passed
@bugarela bugarela mentioned this pull request Jun 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable checking a set of invariants and see which ones broke
2 participants