You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our organization requires us to use Docker Agents in a Jenkins pipeline. This means that pnpm runs within a Docker container. The Workspace is mounted to said Docker container as a volume. We then run pnpm install --frozen-lockfile within that container. The dependencies are written to the mounted volume, thus making them available in next steps in which we mount the same directory to a different Docker container.
The issue is that it seems like I can't get the pnpm store to work. I would suspect that hard links just don't work as they're supposed to with Docker volumes. Jenkins creates a separate workspace for every branch that is currently active, meaning we have ~10-20 very similar instances of node_modules folders. Every one of these takes additional storage. The amount of free disk space decreases linearly with the amount of open branches, which leads me to the conclusion that pnpm does not actually hard link.
I have tried creating a store directory on the agent and mounting that to the Docker container as well. pnpm is writing to the store (there are files in the store after running the pipeline), but it does not seem like it is hard linking correctly.
Is there anything I can do or do I just have to accept that I can't use the global store in our situation?
We have repeatedly run out of disk space - I am trying to figure out whether I should simply bite the bullet and increase the disk space enough for us to not have to worry about this again or if there is a better solution to this.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Our organization requires us to use Docker Agents in a Jenkins pipeline. This means that pnpm runs within a Docker container. The Workspace is mounted to said Docker container as a volume. We then run
pnpm install --frozen-lockfile
within that container. The dependencies are written to the mounted volume, thus making them available in next steps in which we mount the same directory to a different Docker container.The issue is that it seems like I can't get the pnpm store to work. I would suspect that hard links just don't work as they're supposed to with Docker volumes. Jenkins creates a separate workspace for every branch that is currently active, meaning we have ~10-20 very similar instances of
node_modules
folders. Every one of these takes additional storage. The amount of free disk space decreases linearly with the amount of open branches, which leads me to the conclusion that pnpm does not actually hard link.I found this, which supports my suspicion: https://stackoverflow.com/questions/55380443/how-to-create-hard-link-to-file-in-a-docker-volume
I have tried creating a store directory on the agent and mounting that to the Docker container as well. pnpm is writing to the store (there are files in the store after running the pipeline), but it does not seem like it is hard linking correctly.
Is there anything I can do or do I just have to accept that I can't use the global store in our situation?
We have repeatedly run out of disk space - I am trying to figure out whether I should simply bite the bullet and increase the disk space enough for us to not have to worry about this again or if there is a better solution to this.
Beta Was this translation helpful? Give feedback.
All reactions