Refactor cabal-install solver config log output #10854

erikd · 2025-03-26T01:21:00Z

Includes:

Apply some of @grayjay and @mpickering comments
Fix Solver "rejecting" message is too verbose #4251

This is the PR #9541 rebased and fixed to build.

Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

Patches conform to the coding conventions.
Any changes that could be relevant to users have been recorded in the changelog.
- Is the change significant? If so, remember to add significance: significant in the changelog file.
The documentation has been updated, if necessary.
Manual QA notes have been included.
Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

ulysses4ever · 2025-03-26T02:04:04Z

Any chance you could add examples of what the new output looks like? Say, in the PR description.

grayjay · 2025-03-27T01:01:05Z

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs

erikd · 2025-03-27T23:04:33Z

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

According to this comment this seems like a precursor to #9159 .

grayjay · 2025-03-31T08:01:24Z

According to this comment this seems like a precursor to #9159 .

The original change in #9159 was split into a refactoring change and a fix for #4251. Now the refactoring change is in #9159, and the fix for #4251 is in #9541. #9541 contains #9159, because the fix depends on the refactoring.

#9560 has also been merged since #9541 was written and helps address #4251. Do you know how this fix compares now?

erikd · 2025-04-01T06:39:23Z

Current version of this PR aims to minimize the differences in the cabal-install:unit-test output.

Still need to provide a information about how this version improves the solver output compared to the current output.

erikd · 2025-04-02T02:39:13Z

The changes between the output on master and the output in this PR is mostly incredibly minor (as shown by the tiny patch to the tests).

This is the only difference I could find in the cabal-install:unit-tests output:
master

      minimize conflict set with --minimize-conflict-set:                                                                              FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:274:
        Unexpected error:
        Could not resolve dependencies:
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: D (dependency of A)
        [__1] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__1] fail (backjumping, conflict set: A, D)
        After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: A (5), D (4)
        Use -p '/minimize conflict set with --minimize-conflict-set/' to rerun this test only.
      show original conflict set with --no-minimize-conflict-set:                                                                      FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:274:
        Unexpected error:
        Could not resolve dependencies:
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: A (7), B (2), C (2), D (2)
        Try running with --minimize-conflict-set to improve the error message.
        Use -p '/show original conflict set with --no-minimize-conflict-set/' to rerun this test only.

New:

      minimize conflict set with --minimize-conflict-set:                                                                              FAIL
        tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs:268:
        Unexpected solver log:
        targets: A
        constraints: 
          any.base installed (non-reinstallable package)
          any.ghc-bignum installed (non-reinstallable package)
          any.ghc-internal installed (non-reinstallable package)
          any.ghc-prim installed (non-reinstallable package)
          any.ghc installed (non-reinstallable package)
          any.integer-gmp installed (non-reinstallable package)
          any.integer-simple installed (non-reinstallable package)
          any.template-haskell installed (non-reinstallable package)
        preferences: 
        strategy: PreferLatestForSelected
        reorder goals: False
        count conflicts: True
        fine grained conflicts: True
        minimize conflict set: True
        independent goals: False
        avoid reinstalls: False
        shadow packages: False
        strong flags: False
        allow boot library installs: False
        only constrained packages: OnlyConstrainedNone
        max backjumps: infinite
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        [__0] trying: A-2.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] next goal: C (dependency of A)
        [__2] rejecting: C-1.0.0 (conflict: A => C==2.0.0)
        [__2] fail (backjumping, conflict set: A, C)
        [__0] trying: A-1.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] trying: C-1.0.0 (dependency of A)
        [__3] next goal: D (dependency of A)
        [__3] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__3] fail (backjumping, conflict set: A, D)
        [__0] fail (backjumping, conflict set: A, B, C, D)
        Found no solution after exhaustively searching the dependency tree. Rerunning the dependency solver to minimize the conflict set ({A, B, C, D}).
        Trying to remove variable "A" from the conflict set.
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: B (dependency of A)
        [__1] rejecting: B-1.0.0 (conflict: A => B==2.0.0)
        [__1] fail (backjumping, conflict set: A, B)
        [__0] trying: A-2.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] next goal: C (dependency of A)
        [__2] rejecting: C-1.0.0 (conflict: A => C==2.0.0)
        [__2] fail (backjumping, conflict set: A, C)
        [__0] trying: A-1.0.0
        [__1] trying: B-1.0.0 (dependency of A)
        [__2] trying: C-1.0.0 (dependency of A)
        [__3] next goal: D (dependency of A)
        [__3] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__3] fail (backjumping, conflict set: A, D)
        [__0] fail (backjumping, conflict set: A, B, C, D)
        Failed to remove "A" from the conflict set. Continuing with {A, B, C, D}.
        Trying to remove variable "B" from the conflict set.
        [__0] tryingE: A-3.0.0 (user goal)
        [__1] tryingE: C-1.0.0 (dependency of A)
        [__2] next goal: D (dependency of A)
        [__2] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__2] fail (backjumping, conflict set: A, D)
        [__0] skipping: A; 1.0.0, 2.0.0 (has the same characteristics that caused the previous version to fail: depends on 'D' but excludes version 1.0.0)
        [__0] fail (backjumping, conflict set: A, D)
        Successfully removed "B" from the conflict set. Continuing with {A, D}.
        Trying to remove variable "D" from the conflict set.
        [__0] trying: A-3.0.0 (user goal)
        [__1] next goal: D (dependency of A)
        [__1] rejecting: D-1.0.0 (conflict: A => D==2.0.0)
        [__1] fail (backjumping, conflict set: A, D)
        [__0] skipping: A; 1.0.0, 2.0.0 (has the same characteristics that caused the previous version to fail: depends on 'D' but excludes version 1.0.0)
        [__0] fail (backjumping, conflict set: A, D)
        Failed to remove "D" from the conflict set. Continuing with {A, D}.

I suppose the main benefit of this PR is that in the file cabal-install-solver/src/Distribution/Solver/Modular/Message.hs detection of errors is separated from reporting of errors.

erikd · 2025-04-02T03:35:47Z

I haven't had a chance to look at the code yet, but, if I remember correctly, #9541 was a followup to #9159. Would it make sense to first update and merge #9159?

I have had a look at #9159 (against master from 18 months ago) but its rather difficult what really changes between them. I am studying #9159 more closely to figure out what it actually does.

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

grayjay · 2025-04-03T07:17:23Z

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

Isn't this PR an updated version of #9541? Do you mean #9560?

erikd · 2025-04-06T22:19:23Z

Update: I am not able find any significant differences between the behavior of the code in this PR and in #9541 .

Isn't this PR an updated version of #9541? Do you mean #9560?

Yes, I got confused. This is an updated version of #9541. Ad for #9560, that has been merged but does not seem to be related this PR. I do think that #9159 is related.

So the correct comment is, "I am not able find any significant differences between the behavior of the code in this PR and in #9159 ".

grayjay · 2025-04-08T20:16:39Z

Here is the current state of each of the PRs, as I understand it:

#9159: It contains two commits (b20ea53 and f10dbcf) that refactor the code in preparation for the improvement to error messages in #9541.
#9541: It contains the two commits from #9159, as well as a third commit (e4775b4) to improve error messages by removing duplication of package names.
#9560: It removes duplication of package names in error messages, without significant refactoring.

Since #9560 was already merged, it seems like the main difference between this PR and master is the refactoring. Are you interested in merging this just for the refactoring? Are you planning to make additional changes to the solver log that depend on it?

erikd · 2025-04-08T21:28:38Z

My hope is to get this refactor merged (after the fix suggested by @mpickering ). Then I intend to work on improving error messages as per commit e4775b4 .

grayjay · 2025-04-09T00:30:57Z

I meant that this PR already contains the contents of e4775b4 (removing duplicate package names, similar to #9560), so I was wondering if you were planning to make more changes beyond e4775b4 that depend on the refactoring.

erikd · 2025-04-16T08:42:19Z

@grayjay Let me look at #9159 and try to address those comments.

erikd · 2025-05-20T21:56:01Z

@grayjay Sorry for the delay. Here is the updated PR with the change requests make in #9159.

LogSucessMsg -> LogSuccess
LogFailureMsg -> LogFailure
All constructors of Entry data type ( file cabal-install-solver/src/Distribution/Solver/Types/Progress.hs) renamed from Log* to Entry* .
EntryMsg data type renamed to EntryAtLevel.
Make SummarizedMsg type opaque (constructors not exported).
Undo some un-needed formatting changes for imports

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs

cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs

grayjay

@erikd No problem! Thanks for making the renaming changes. I tried to copy the remaining comments from #9159, though I may have applied some incorrectly since they were hard to view in the Github UI. I also added a few new comments.

grayjay · 2025-05-22T17:47:52Z

cabal-install-solver/src/Distribution/Solver/Modular.hs

+    retryMap :: (t -> step) -> RetryLog t fail done -> RetryLog step fail done
+    retryMap f l = fromProgress $ (\p -> foldProgress (\x xs -> Step (f x) xs) Fail Done p) $ toProgress l


From #9159: I think that it would be better to avoid creating a helper function that hides calls to toProgress and fromProgress, because it may be very expensive to convert between the linked list and difference list. Ideally, toProgress would only be called once at the end of dependency solving.

Had a couple of runs at this and was not able to write retryMap without using toProgress and fromProgess.

I am not saying its not possible, just that I am not smart enough to do it. It might even be easy, but I do not see it.

I'm not sure that it's possible to convert the RetryLogss efficiently, but I think that it would be easier to remove the calls to retryMap, as I described in my previous comment.

cabal-install-solver/src/Distribution/Solver/Modular.hs

cabal-install-solver/src/Distribution/Solver/Types/Progress.hs

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs

cabal-install/src/Distribution/Client/Dependency.hs

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs

@grayjay

Includes: * Apply some of @grayjay and @mpickering comments * Fix haskell#4251 Co-Authored-By: Erik de Castro Lopo <[email protected]>

These fixes are require due to improvements in solver error reporting.

Extensive effort was put in to simplify the solver. In the end only a pretty minor simplification was possible.

erikd · 2025-06-04T06:17:52Z

One of the CI failures is with ghc-9.6.7 and cabal-install-3.12.1.0 . When I run the validation tests locally with those two versions everything passes.

ulysses4ever · 2025-06-04T07:30:59Z

Try rebasing on top of current master (if not already).

grayjay

I responded to some of your latest comments and changes, but I haven't done another full review yet.

cabal-install-solver/src/Distribution/Solver/Types/Progress.hs

cabal-install-solver/src/Distribution/Solver/Modular.hs

grayjay · 2025-06-04T06:47:27Z

cabal-install-solver/src/Distribution/Solver/Modular.hs

+    retryMap :: (t -> step) -> RetryLog t fail done -> RetryLog step fail done
+    retryMap f l = fromProgress $ (\p -> foldProgress (\x xs -> Step (f x) xs) Fail Done p) $ toProgress l


I'm not sure that it's possible to convert the RetryLogss efficiently, but I think that it would be easier to remove the calls to retryMap, as I described in my previous comment.

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs

erikd · 2025-06-04T07:56:44Z

Try rebasing on top of current master (if not already).

It says:

Current branch erikd/cosmetic-changes-2 is up to date.

ulysses4ever · 2025-06-04T15:06:42Z

Let me try to restart the failing jobs then. It's the same test in all configuration, so this test may be up for the flaky marker if it fails like that ...

erikd · 2025-06-05T01:27:03Z

I have left the new fixes for PR comments discrete for easier review. When this is approved for merge I will squash them down as appropriate.

erikd · 2025-06-05T03:42:24Z

One of the "Validate" failures is with ghc-9.6.7 and cabal-install-3.12.1.0. With those versions it passes locally.

The "Bootstrap" failure is a failure in Python code and my PR does not touch any Python code.

I have already tried rebasing against master.

erikd requested review from mpickering and grayjay March 26, 2025 01:21

erikd mentioned this pull request Mar 26, 2025

Cosmetic changes to cabal-install summarized solver output #9541

Closed

5 tasks

erikd force-pushed the erikd/cosmetic-changes-2 branch 5 times, most recently from e602461 to 8c1868b Compare March 26, 2025 01:53

erikd force-pushed the erikd/cosmetic-changes-2 branch 5 times, most recently from c8f419c to 5a2528d Compare March 26, 2025 04:06

Bodigrim reviewed Mar 27, 2025

View reviewed changes

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs Outdated Show resolved Hide resolved

erikd force-pushed the erikd/cosmetic-changes-2 branch from 5a2528d to 78733cd Compare April 1, 2025 06:37

erikd force-pushed the erikd/cosmetic-changes-2 branch from 78733cd to b7b0c64 Compare April 2, 2025 00:27

erikd force-pushed the erikd/cosmetic-changes-2 branch from b7b0c64 to 6fbd8d7 Compare April 6, 2025 21:38

erikd force-pushed the erikd/cosmetic-changes-2 branch from 6fbd8d7 to dce06cb Compare April 8, 2025 01:52

grayjay added the cabal-install: solver label May 8, 2025

erikd force-pushed the erikd/cosmetic-changes-2 branch from 4644150 to 6c1b10c Compare May 20, 2025 21:55

erikd requested a review from Bodigrim May 22, 2025 00:57

Bodigrim reviewed May 22, 2025

View reviewed changes

erikd force-pushed the erikd/cosmetic-changes-2 branch from 6c1b10c to a023ab3 Compare May 22, 2025 23:16

Bodigrim reviewed May 22, 2025

View reviewed changes

cabal-install-solver/src/Distribution/Solver/Modular/Message.hs Outdated Show resolved Hide resolved

grayjay reviewed May 23, 2025

View reviewed changes

Bodigrim approved these changes May 26, 2025

View reviewed changes

yvan-sraka and others added 2 commits May 27, 2025 15:58

Refactor cabal-install solver config log output

552c039

Includes: * Apply some of @grayjay and @mpickering comments * Fix haskell#4251 Co-Authored-By: Erik de Castro Lopo <[email protected]>

tests: Fixes to expected message strings

ac35301

These fixes are require due to improvements in solver error reporting.

erikd force-pushed the erikd/cosmetic-changes-2 branch 2 times, most recently from d1d76cf to 621ff80 Compare June 4, 2025 03:57

erikd added 2 commits June 4, 2025 14:12

Address PR comments

ba648e4

Modular-solver: Simplify

fbdfc74

Extensive effort was put in to simplify the solver. In the end only a pretty minor simplification was possible.

erikd force-pushed the erikd/cosmetic-changes-2 branch from 621ff80 to fbdfc74 Compare June 4, 2025 04:12

grayjay reviewed Jun 4, 2025

View reviewed changes

erikd added 2 commits June 5, 2025 11:25

Use showOption instead of showOptions for singleton lists

860ff78

Rename SolverState back to Message

9cb5a04

erikd force-pushed the erikd/cosmetic-changes-2 branch from f0fb511 to 7f2599e Compare June 5, 2025 01:40

erikd added 2 commits June 5, 2025 12:07

Move Message type back to ../Solver/Modular/Message.hs

b5bd72a

Remove all calls to retryMap and remove it

b77ca13

erikd force-pushed the erikd/cosmetic-changes-2 branch from 7f2599e to b77ca13 Compare June 5, 2025 02:07

		retryMap :: (t -> step) -> RetryLog t fail done -> RetryLog step fail done
		retryMap f l = fromProgress $ (\p -> foldProgress (\x xs -> Step (f x) xs) Fail Done p) $ toProgress l

Refactor cabal-install solver config log output #10854

Are you sure you want to change the base?

Refactor cabal-install solver config log output #10854

Conversation

erikd commented Mar 26, 2025 • edited by ulysses4ever Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ulysses4ever commented Mar 26, 2025

Uh oh!

grayjay commented Mar 27, 2025

Uh oh!

Uh oh!

erikd commented Mar 27, 2025

Uh oh!

grayjay commented Mar 31, 2025

Uh oh!

erikd commented Apr 1, 2025

Uh oh!

erikd commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erikd commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grayjay commented Apr 3, 2025

Uh oh!

erikd commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grayjay commented Apr 8, 2025

Uh oh!

erikd commented Apr 8, 2025

Uh oh!

grayjay commented Apr 9, 2025

Uh oh!

erikd commented Apr 16, 2025

Uh oh!

erikd commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

grayjay left a comment

Choose a reason for hiding this comment

Uh oh!

grayjay May 22, 2025

Choose a reason for hiding this comment

Uh oh!

erikd Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

grayjay Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erikd commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ulysses4ever commented Jun 4, 2025

Uh oh!

grayjay left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

grayjay Jun 4, 2025

erikd commented Mar 26, 2025 •

edited by ulysses4ever

Loading

erikd commented Apr 2, 2025 •

edited

Loading

erikd commented Apr 2, 2025 •

edited

Loading

erikd commented Apr 6, 2025 •

edited

Loading

erikd commented May 20, 2025 •

edited

Loading

erikd commented Jun 4, 2025 •

edited

Loading

erikd commented Jun 5, 2025 •

edited

Loading

erikd commented Jun 5, 2025 •

edited

Loading