"File already exists" error when installing multiple copies of the same package external to the packages folder, ignores --force

[galaxystar]

This issue is somewhat related to: #1472

When Paket installs to a path outside of the ./packages directory, I receive the error below, when multiple packages reference the same package (and all of which trying to install to the path outside of the ./packages folder)

I've tried using paket install -f and paket install --hard and paket instal -f --hard but they all fail when trying to extract the file.

Regardless of the use case that caused this It would be nice if the force flag would overwrite files which already exist, as I would expect FORCE to do.

Paket failed with:
    Error during extraction of /<path>/packages/<package group>/PackageName/PackageName.1.0.8.nupkg.
Message: Could not create file "/<path outside of packages foder>/<some file>". File already exists.

Stack Trace:

StackTrace:
    at Paket.NuGetV2+ExtractPackage@356-7.Invoke (System.String message) <0x1108c52a0 + 0x0002f> in <filename unknown>:0 
  at Microsoft.FSharp.Core.PrintfImpl+StringPrintfEnv`1[TResult].Finalize () <0x10abe5c10 + 0x00039> in <filename unknown>:0 
  at Microsoft.FSharp.Core.PrintfImpl+Final4@259[TState,TResidue,TResult,A,B,C,D].Invoke (Microsoft.FSharp.Core.FSharpFunc`2 env, A a, B b, C c, D d) <0x1108c50d0 + 0x001bb> in <filename unknown>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3307-3[T2,T3,T4,T5,TResult,T1].Invoke (T2 u, T3 v, T4 w, T5 x) <0x1108c5080 + 0x00047> in <filename unknown>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3301-2[T2,T3,T4,TResult,T1].Invoke (T2 u, T3 v, T4 w) <0x1108c5030 + 0x0003d> in <filename unknown>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3266-1[T2,T3,TResult,T1].Invoke (T2 u, T3 v) <0x10f502eb0 + 0x00033> in <filename unknown>:0 
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3253[T2,TResult,T1].Invoke (T2 u) <0x10abe58a0 + 0x00029> in <filename unknown>:0 
  at Paket.NuGetV2+ExtractPackage@356-11.Invoke (System.String arg40) <0x1108c5000 + 0x00021> in <filename unknown>:0 
  at Microsoft.FSharp.Core.FSharpFunc`2[T,TResult].InvokeFast[V] (Microsoft.FSharp.Core.FSharpFunc`2 func, Microsoft.FSharp.Core.T arg1, Microsoft.FSharp.Core.TResult arg2) <0x10f3d6b80 + 0x000ad> in <filename unknown>:0 
  at Microsoft.FSharp.Core.FSharpFunc`2[T,TResult].InvokeFast[V,W] (Microsoft.FSharp.Core.FSharpFunc`2 func, Microsoft.FSharp.Core.T arg1, Microsoft.FSharp.Core.TResult arg2, Microsoft.FSharp.Core.V arg3) <0x1108c4d40 + 0x0016f> in <filename unknown>:0 
  at Microsoft.FSharp.Core.FSharpFunc`2[T,TResult].InvokeFast[V,W,X] (Microsoft.FSharp.Core.FSharpFunc`2 func, Microsoft.FSharp.Core.T arg1, Microsoft.FSharp.Core.TResult arg2, Microsoft.FSharp.Core.V arg3, Microsoft.FSharp.Core.W arg4) <0x1108c48f0 + 0x0023b> in <filename unknown>:0 
  at Paket.NuGetV2+ExtractPackage@349-6.Invoke (System.Exception _arg1) <0x1108c3e10 + 0x00274> in <filename unknown>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@893[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) <0x1108c3c40 + 0x0003b> in <filename unknown>:0 
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@851[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1 args) <0x10f5b77b0 + 0x0015f> in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () in <filename unknown>:line 0
   at Microsoft.FSharp.Control.AsyncBuilderImpl.commit[a] (Microsoft.FSharp.Control.Result`1 res) in <filename unknown>:line 0
   at Microsoft.FSharp.Control.CancellationTokenOps.RunSynchronously[a] (CancellationToken token, Microsoft.FSharp.Control.FSharpAsync`1 computation, Microsoft.FSharp.Core.FSharpOption`1 timeout) in <filename unknown>:line 0
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T] (Microsoft.FSharp.Control.FSharpAsync`1 computation, Microsoft.FSharp.Core.FSharpOption`1 timeout, Microsoft.FSharp.Core.FSharpOption`1 cancellationToken) in <filename unknown>:line 0
   at Paket.InstallProcess+CreateModel@151-1.Invoke (KeyValuePair`2 kv') in <filename unknown>:line 0
   at Microsoft.FSharp.Collections.IEnumerator+map@111[b,a].DoMoveNext (b& ) in <filename unknown>:line 0
   at Microsoft.FSharp.Collections.IEnumerator+MapEnumerator`1[T].System-Collections-IEnumerator-MoveNext () in <filename unknown>:line 0
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeOuter@713[T,TResult] (Microsoft.FSharp.Core.CompilerServices.ConcatEnumerator`2 x, Microsoft.FSharp.Core.Unit unitVar0) in <filename unknown>:line 0
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@706[T,TResult] (Microsoft.FSharp.Core.CompilerServices.ConcatEnumerator`2 x, Microsoft.FSharp.Core.Unit unitVar0) in <filename unknown>:line 0
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeOuter@713[T,TResult] (Microsoft.FSharp.Core.CompilerServices.ConcatEnumerator`2 x, Microsoft.FSharp.Core.Unit unitVar0) in <filename unknown>:line 0
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@706[T,TResult] (Microsoft.FSharp.Core.CompilerServices.ConcatEnumerator`2 x, Microsoft.FSharp.Core.Unit unitVar0) in <filename unknown>:line 0
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeOuter@713[T,TResult] (Microsoft.FSharp.Core.CompilerServices.ConcatEnumerator`2 x, Microsoft.FSharp.Core.Unit unitVar0) in <filename unknown>:line 0
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.takeInner@706[T,TResult] (Microsoft.FSharp.Core.CompilerServices.ConcatEnumerator`2 x, Microsoft.FSharp.Core.Unit unitVar0) in <filename unknown>:line 0
   at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers+ConcatEnumerator`2[T,TResult].System-Collections-IEnumerator-MoveNext () in <filename unknown>:line 0
   at System.Collections.Generic.List`1[T]..ctor (IEnumerable`1 collection) in <filename unknown>:line 0
   at Microsoft.FSharp.Collections.SeqModule.ToArray[T] (IEnumerable`1 source) in <filename unknown>:line 0
   at Paket.InstallProcess.CreateModel (System.String root, Boolean force, Paket.DependenciesFile dependenciesFile, Paket.LockFile lockFile, Microsoft.FSharp.Collections.FSharpSet`1 packages) in <filename unknown>:line 0
   at Paket.InstallProcess.InstallIntoProjects (Paket.InstallerOptions options, Boolean forceTouch, Paket.DependenciesFile dependenciesFile, Paket.LockFile lockFile, Microsoft.FSharp.Collections.FSharpList`1 projectsAndReferences) in <filename unknown>:line 0
   at Paket.UpdateProcess.SmartInstall (Paket.DependenciesFile dependenciesFile, Paket.UpdateMode updateMode, Paket.UpdaterOptions options) in <filename unknown>:line 0
   at <StartupCode$Paket-Core>.$PublicAPI+Install@164.Invoke (Microsoft.FSharp.Core.Unit unitVar0) in <filename unknown>:line 0
   at Paket.Utils.RunInLockedAccessMode[a] (System.String rootFolder, Microsoft.FSharp.Core.FSharpFunc`2 action) in <filename unknown>:line 0

[galaxystar]

I've narrowed this issue down to ExtractPackage in NuGetV2.fs, on line 351 where Paket uses ZipFile.ExtractToDirectory. We could use ZipArchive instead (which is used elsewhere in the project). If the Force flag is set then use code similar to what is provided here:
http://stackoverflow.com/questions/14795197/forcefully-replacing-existing-files-during-extracting-file-using-system-io-compr

[forki]

do you have a repro sample for me?

[galaxystar]

A simple repro should be as follows:

Create a package that contains 1 dependency packet. The dependent package should have at least one file, and map its content to be deployed outside of the package folder. Use paket install to install the package. Then use paket install --force to try and install the package again, observe the following message:

Paket failed with:
    Error during extraction of /<path>/packages/<package group>/PackageName/PackageName.1.0.8.nupkg.
Message: Could not create file "/<path outside of packages foder>/<some file>". File already exists.

Package A depends on Package B

Package A paket.dependency file:

...
    nuget PackageB
...

Package B paket.template file:

...
files
    bin/ ==> ../../<some_path_outside_packages_folder>/
...

[forki]

and map its content to be deployed outside of the package folder

Is this something that nuget supports? That sounds pretty dangerous to me

[galaxystar]

Paket supports it, and I rely on it. It's a bit of a round about way to get what I want. I'm using it to deploy a bootstrapper script into a folder that can then later deploy (copy) the package groups where I want them.

My ideal solution would be to be able to Install packages and then be able to Deploy package groups where I want them, in 2 separate steps. It would be nice to be able to map the group's deploy location in the template file. At which point I could deploy to paths outside of the packages folder.

p.s. I know this is not exactly how Paket was intended to be used, but my toolset (Unity3d) does not support the "proper way" to use Paket. And no, I don't want to use Paket.Unity3D.

[forki]

Paket supports it, and I rely on it.

It's more an accidental feature. I don't think it's "supported"

[galaxystar]

We fixed a bug (#1472) a few weeks back with globbing when packaging which allowed me to continue to use it . Sounds "supported" :-).

But the point of the matter, this repro case may not be "supported", but the request is still sound. When using the "--force" flag, it should override any existing files, which it is not doing.

[forki]

just tried to reproduce. see 65bb4a8

the message says: "Can't extract since it would create a file outside of target folder"

so what am I doing differently?

[galaxystar]

Perhaps Paket 3 handles it differently. My test case is using Paket 2.51.4.0 (sorry for not mentioning it sooner).

[forki]

nope the test is against paket 2

[galaxystar]

The screenshot you posted says "Paket Version 3.0.0.0"

[forki]

Yes, but the test case is showing same error in both branches.
On Mar 9, 2016 20:25, "galaxystar" notifications@github.com wrote:

The screenshot you posted says "Paket Version 3.0.0.0"

—
Reply to this email directly or view it on GitHub
#1504 (comment).

[galaxystar]

I'm not sure, I just installed Paket 2.51.11.0 and I see an identical error to what I posed above.

Paket failed with:
    Error during extraction of /Users/galaxystar/Git/Fyber/packages/bootstrap/PackageManagerBootStrap/PackageManagerBootStrap.1.0.7.nupkg.
Message: Could not create file "/Users/galaxystar/Git/Fyber/Assets/Plugins/Packages/bootstrap/PackageManagerBootStrap/Editor/PackageManager.cs". File already exists.
 In rare cases a firewall might have blocked the download. Please look into the file and see if it contains text with further information.