Skip to content

Refactor cabal-install solver config log output #10854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

erikd
Copy link
Member

@erikd erikd commented Mar 26, 2025

Includes:

This is the PR #9541 rebased and fixed to build.


Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

@erikd erikd requested review from mpickering and grayjay March 26, 2025 01:21
@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch 5 times, most recently from e602461 to 8c1868b Compare March 26, 2025 01:53
@ulysses4ever
Copy link
Collaborator

Any chance you could add examples of what the new output looks like? Say, in the PR description.

@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch 5 times, most recently from c8f419c to 5a2528d Compare March 26, 2025 04:06
@grayjay
Copy link
Collaborator

grayjay commented Mar 27, 2025

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

@erikd
Copy link
Member Author

erikd commented Mar 27, 2025

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

According to this comment this seems like a precursor to #9159 .

@grayjay
Copy link
Collaborator

grayjay commented Mar 31, 2025

According to this comment this seems like a precursor to #9159 .

The original change in #9159 was split into a refactoring change and a fix for #4251. Now the refactoring change is in #9159, and the fix for #4251 is in #9541. #9541 contains #9159, because the fix depends on the refactoring.

#9560 has also been merged since #9541 was written and helps address #4251. Do you know how this fix compares now?

@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from 5a2528d to 78733cd Compare April 1, 2025 06:37
@erikd
Copy link
Member Author

erikd commented Apr 1, 2025

Current version of this PR aims to minimize the differences in the cabal-install:unit-test output.

Still need to provide a information about how this version improves the solver output compared to the current output.

@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from 78733cd to b7b0c64 Compare April 2, 2025 00:27
@erikd
Copy link
Member Author

erikd commented Apr 2, 2025

The changes between the output on master and the output in this PR is mostly incredibly minor (as shown by the tiny patch to the tests).

This is the only difference I could find in the cabal-install:unit-tests output:
master

      minimize conflict set with --minimize-conflict-set:                                                                              FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:274:
        Unexpected error:
        Could not resolve dependencies:
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: D (dependency of A)
        [__1] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__1] fail (backjumping, conflict set: A, D)
        After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: A (5), D (4)
        Use -p '/minimize conflict set with --minimize-conflict-set/' to rerun this test only.
      show original conflict set with --no-minimize-conflict-set:                                                                      FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:274:
        Unexpected error:
        Could not resolve dependencies:
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: A (7), B (2), C (2), D (2)
        Try running with --minimize-conflict-set to improve the error message.
        Use -p '/show original conflict set with --no-minimize-conflict-set/' to rerun this test only.

New:

      minimize conflict set with --minimize-conflict-set:                                                                              FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:268:
        Unexpected solver log:
        targets: A
        constraints: 
          any.base installed (non-reinstallable package)
          any.ghc-bignum installed (non-reinstallable package)
          any.ghc-internal installed (non-reinstallable package)
          any.ghc-prim installed (non-reinstallable package)
          any.ghc installed (non-reinstallable package)
          any.integer-gmp installed (non-reinstallable package)
          any.integer-simple installed (non-reinstallable package)
          any.template-haskell installed (non-reinstallable package)
        preferences: 
        strategy: PreferLatestForSelected
        reorder goals: False
        count conflicts: True
        fine grained conflicts: True
        minimize conflict set: True
        independent goals: False
        avoid reinstalls: False
        shadow packages: False
        strong flags: False
        allow boot library installs: False
        only constrained packages: OnlyConstrainedNone
        max backjumps: infinite
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        [__0] trying: A-2.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] next goal: C (dependency of A)
        [__2] rejecting: C-1.0.0 (conflict: A => C==2.0.0)
        [__2] fail (backjumping, conflict set: A, C)
        [__0] trying: A-1.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] trying: C-1.0.0 (dependency of A)
        [__3] next goal: D (dependency of A)
        [__3] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__3] fail (backjumping, conflict set: A, D)
        [__0] fail (backjumping, conflict set: A, B, C, D)
        Found no solution after exhaustively searching the dependency tree. Rerunning the dependency solver to minimize the conflict set ({A, B, C, D}).
        Trying to remove variable "A" from the conflict set.
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        [__0] trying: A-2.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] next goal: C (dependency of A)
        [__2] rejecting: C-1.0.0 (conflict: A => C==2.0.0)
        [__2] fail (backjumping, conflict set: A, C)
        [__0] trying: A-1.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] trying: C-1.0.0 (dependency of A)
        [__3] next goal: D (dependency of A)
        [__3] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__3] fail (backjumping, conflict set: A, D)
        [__0] fail (backjumping, conflict set: A, B, C, D)
        Failed to remove "A" from the conflict set. Continuing with {A, B, C, D}.
        Trying to remove variable "B" from the conflict set.
        [__0] tryingE: A-3.0.0 (user goal)
        [__1] tryingE: C-1.0.0 (dependency of A)
        [__2] next goal: D (dependency of A)
        [__2] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__2] fail (backjumping, conflict set: A, D)
        [__0] skipping: A; 1.0.0, 2.0.0 (has the same characteristics that caused the previous version to fail: depends on 'D' but excludes version 1.0.0)
        [__0] fail (backjumping, conflict set: A, D)
        Successfully removed "B" from the conflict set. Continuing with {A, D}.
        Trying to remove variable "D" from the conflict set.
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: D (dependency of A)
        [__1] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__1] fail (backjumping, conflict set: A, D)
        [__0] skipping: A; 1.0.0, 2.0.0 (has the same characteristics that caused the previous version to fail: depends on 'D' but excludes version 1.0.0)
        [__0] fail (backjumping, conflict set: A, D)
        Failed to remove "D" from the conflict set. Continuing with {A, D}.

I suppose the main benefit of this PR is that in the file cabal-install-solver/src/Distribution/Solver/Modular/Message.hs detection of errors is separated from reporting of errors.

@erikd
Copy link
Member Author

erikd commented Apr 2, 2025

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

I have had a look at #9159 (against master from 18 months ago) but its rather difficult what really changes between them. I am studying #9159 more closely to figure out what it actually does.

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

@grayjay
Copy link
Collaborator

grayjay commented Apr 3, 2025

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

Isn't this PR an updated version of #9541? Do you mean #9560?

@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from b7b0c64 to 6fbd8d7 Compare April 6, 2025 21:38
@erikd
Copy link
Member Author

erikd commented Apr 6, 2025

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

Isn't this PR an updated version of #9541? Do you mean #9560?

Yes, I got confused. This is an updated version of #9541. Ad for #9560, that has been merged but does not seem to be related this PR. I do think that #9159 is related.

So the correct comment is, "I am not able find any significant differences between the behavior of the code in this PR and in #9159 ".

@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from 6fbd8d7 to dce06cb Compare April 8, 2025 01:52
@grayjay
Copy link
Collaborator

grayjay commented Apr 8, 2025

Here is the current state of each of the PRs, as I understand it:

#9159: It contains two commits (b20ea53 and f10dbcf) that refactor the code in preparation for the improvement to error messages in #9541.
#9541: It contains the two commits from #9159, as well as a third commit (e4775b4) to improve error messages by removing duplication of package names.
#9560: It removes duplication of package names in error messages, without significant refactoring.

Since #9560 was already merged, it seems like the main difference between this PR and master is the refactoring. Are you interested in merging this just for the refactoring? Are you planning to make additional changes to the solver log that depend on it?

@erikd
Copy link
Member Author

erikd commented Apr 8, 2025

My hope is to get this refactor merged (after the fix suggested by @mpickering ). Then I intend to work on improving error messages as per commit e4775b4 .

@grayjay
Copy link
Collaborator

grayjay commented Apr 9, 2025

I meant that this PR already contains the contents of e4775b4 (removing duplicate package names, similar to #9560), so I was wondering if you were planning to make more changes beyond e4775b4 that depend on the refactoring.

@erikd
Copy link
Member Author

erikd commented Apr 16, 2025

@grayjay Let me look at #9159 and try to address those comments.

@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from 4644150 to 6c1b10c Compare May 20, 2025 21:55
@erikd
Copy link
Member Author

erikd commented May 20, 2025

@grayjay Sorry for the delay. Here is the updated PR with the change requests make in #9159.

  • LogSucessMsg -> LogSuccess
  • LogFailureMsg -> LogFailure
  • All constructors of Entry data type ( file cabal-install-solver/src/Distribution/Solver/Types/Progress.hs) renamed from Log* to Entry* .
  • EntryMsg data type renamed to EntryAtLevel.
  • Make SummarizedMsg type opaque (constructors not exported).
  • Undo some un-needed formatting changes for imports

@erikd erikd requested a review from Bodigrim May 22, 2025 00:57
@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from 6c1b10c to a023ab3 Compare May 22, 2025 23:16
Copy link
Collaborator

@grayjay grayjay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erikd No problem! Thanks for making the renaming changes. I tried to copy the remaining comments from #9159, though I may have applied some incorrectly since they were hard to view in the Github UI. I also added a few new comments.

Comment on lines 320 to 321
retryMap :: (t -> step) -> RetryLog t fail done -> RetryLog step fail done
retryMap f l = fromProgress $ (\p -> foldProgress (\x xs -> Step (f x) xs) Fail Done p) $ toProgress l
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From #9159: I think that it would be better to avoid creating a helper function that hides calls to toProgress and fromProgress, because it may be very expensive to convert between the linked list and difference list. Ideally, toProgress would only be called once at the end of dependency solving.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a couple of runs at this and was not able to write retryMap without using toProgress and fromProgess.

I am not saying its not possible, just that I am not smart enough to do it. It might even be easy, but I do not see it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that it's possible to convert the RetryLogss efficiently, but I think that it would be easier to remove the calls to retryMap, as I described in my previous comment.

yvan-sraka and others added 2 commits May 27, 2025 15:58
Includes:

* Apply some of @grayjay and @mpickering comments
* Fix haskell#4251

Co-Authored-By: Erik de Castro Lopo <[email protected]>
These fixes are require due to improvements in solver error reporting.
@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch 2 times, most recently from d1d76cf to 621ff80 Compare June 4, 2025 03:57
erikd added 2 commits June 4, 2025 14:12
Extensive effort was put in to simplify the solver. In the end
only a pretty minor simplification was possible.
@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from 621ff80 to fbdfc74 Compare June 4, 2025 04:12
@erikd
Copy link
Member Author

erikd commented Jun 4, 2025

One of the CI failures is with ghc-9.6.7 and cabal-install-3.12.1.0 . When I run the validation tests locally with those two versions everything passes.

@ulysses4ever
Copy link
Collaborator

Try rebasing on top of current master (if not already).

Copy link
Collaborator

@grayjay grayjay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I responded to some of your latest comments and changes, but I haven't done another full review yet.

Comment on lines 320 to 321
retryMap :: (t -> step) -> RetryLog t fail done -> RetryLog step fail done
retryMap f l = fromProgress $ (\p -> foldProgress (\x xs -> Step (f x) xs) Fail Done p) $ toProgress l
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that it's possible to convert the RetryLogss efficiently, but I think that it would be easier to remove the calls to retryMap, as I described in my previous comment.

@erikd
Copy link
Member Author

erikd commented Jun 4, 2025

Try rebasing on top of current master (if not already).

It says:

Current branch erikd/cosmetic-changes-2 is up to date.

@ulysses4ever
Copy link
Collaborator

Let me try to restart the failing jobs then. It's the same test in all configuration, so this test may be up for the flaky marker if it fails like that ...

@erikd
Copy link
Member Author

erikd commented Jun 5, 2025

I have left the new fixes for PR comments discrete for easier review. When this is approved for merge I will squash them down as appropriate.

@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from f0fb511 to 7f2599e Compare June 5, 2025 01:40
@erikd erikd force-pushed the erikd/cosmetic-changes-2 branch from 7f2599e to b77ca13 Compare June 5, 2025 02:07
@erikd
Copy link
Member Author

erikd commented Jun 5, 2025

One of the "Validate" failures is with ghc-9.6.7 and cabal-install-3.12.1.0. With those versions it passes locally.

The "Bootstrap" failure is a failure in Python code and my PR does not touch any Python code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Solver "rejecting" message is too verbose
6 participants