[bootstrapper] Improve handling of invalid hash

[jamescrowley]

Description

I receive a

The file 'C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache\Paket\5.92.3\paket.exe' already exists. 

error when calling "paket.exe restore" having bumped the version to 5.92.3.

Have never seen this before. It goes away on a brand new build agent (they're automated), but comes back again.

Repro steps

Not yet able to reproduce. Known aspects:

• Occurs when calling paket.exe restore
• Most branches on build server running on 5.85.1 with paket.exe in magic mode, with version passed as argument.
• New branch on build server running 5.92.3 with paket.exe in magic mode, with version passed in paket.dependencies.
• Build servers rebuilt from scratch each day, and happened two days running so far.
• Works successfully on first run of the day.
• Subsequent builds of branch fail on that build server. Not sure if there is interaction with other builds going on here.
• Happened once on local machine but haven't been able to reproduce since after clearing cache folder.

Expected behavior

paket.exe restores packages as in previous versions.

Actual behavior

The file 'C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache\Paket\5.92.3\paket.exe' already exists.

Packages are then not restored.

Known workarounds

Kill the build server and force a fresh instance.

[matthid]

system32??? So you are running this stuff on a system account?
What I would expect is that the bootstrapper is cached and then all are using the same exe in the Cache.

In your specific scenario it seems the machines are cleared from time to time?
We really need to repro this or at least track it down. Can you enable logs for the bootstrapper by setting the verbose flag or the PAKET_BOOTSTRAPPER_TRACE environment variable to true (ie PAKET_BOOTSTRAPPER_TRACE=true) and report back the results?

[miloszes]

Hi,

We've runned with the same issue. The problem arises on the continues integration machines (TeamCity).
We're running paket (paket.boostrap latest version (checked an half hour ago) in the magic mode) with the following arguments:

.paket\paket.exe restore --verbose
The environment variable has been also set with command (before paket execution :) )
setx PAKET_BOOTSTRAPPER_TRACE true

The only message We've received was:

The file 'C:\Users\XXXX\AppData\Local\NuGet\Cache\Paket\5.92.4\paket.exe' already exists. (Github - cached (temporarily ignore updates))

We're using the user dedicated for TeamCity builder.

The problem also occurs with the install command.

When you remove manually the file C:\Users\XXXX\AppData\Local\NuGet\Cache\Paket\5.92.4\paket.exe the build process runs correctly once. After the next paket restoration the same problem exists, which is not easilly seen, because bootstrap finishes its work with 0 code and the solution is compiled with old references :/.

Best Regards,
MiloszeS

[matthid]

I don't even know what we changed there recently, could that be a side effect from signing the .exe?

[miloszes]

I don't know whether it helps. I've switched to the paket.bootstrap (execution of paket.bootstrapper.exe, then paket.exe). When I executed it manually I've received the following messages:

Checking Paket version (downloading latest stable)...
Version 5.92.4 found in cache but it's hash isn't valid.
Starting download from https://github.com/fsprojects/Paket/releases/download/5.92.4/paket.exe
Hash of downloaded paket.exe is invalid, retrying once
Hash of downloaded paket.exe still invalid (Using the file anyway)
The file 'C:\Users\XXXXX\AppData\Local\NuGet\Cache\Paket\5.92.4\paket.exe' already exists. (Github - cached)

But the paket.exe has been downloaded into the same folder as paket.boostrap

It looks like a problem with hash validation and exception handling...

[miloszes]

@matthid it looks like the side offect of signing.

[matthid]

@forki?

[matthid]

see #2729 wtf

[forki]

Ok can you please push an unsigned version for now as quick fix? Will need to investigate next week. /cc @agross Am 06.09.2017 15:05 schrieb "Matthias Dittrich" <notifications@github.com>:

…

see #2729 <#2729> wtf — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2725 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNKrrUYHBxE1Y9SrE8-YLWaVOBKnJks5sfpiAgaJpZM4PNBe7> .

[agross]

We've signed before, also with timestamping. Not sure why a new certificate should change semantics.

[matthid]

probably a bug in the build process. we calculate the hash before signing the exe?
also a bug in the bootstrapper because it doesn't report the actual problem. it should either always warn and work or never work and fail with the correct message imho

[matthid]

I'll leave this open for the bootstrapper problem, as #2729 better describes the invalid hash...

[matthid]

So the immediate problem has been solved #2729 and the hashes match again, please retry.
I'll leave this open as the bootstrapper still needs to be fixed in case this happens again.
A PR is appreciated.