Skip to content

Respect .dockerignore for jobs that use docker #1879

Description

@jrd

For now, if a job is using a docker image, the root source directory is copied under a specific mount point in a specific container. That mountpoint (ciProjectDir) is bound to a volume to save files for later.

That works great, but not when using LFS.

I have some huge files in LFS (that’s the point) and those files are:

  • listed in .dockerignore
  • not checked out in CI (GIT_LFS_SKIP_SMUDGE=1)

But for obvious reason, I have them checked out locally.

gitlab-ci-local uses the command docker cp to sync my local folder to ciProjectDir in the temporary container. That takes LOTS of time because docker first create a tar file of the current directory then send it back to the daemon, then the daemon read the tar and extract it to the final location.

When multiple jobs are run, it takes ages to run.

Unfortunately docker cp has no meaning to pass exclude files.
I was thinking of a more convoluted mean:

  • bind mount the source to a specific folder in the container in read-only (that is quick)
  • use other way to sync that folder to ciProjectDir (rsync ?) by converting .dockerignore file to a format valid for the tool (rsync or another)

Any better option is also valid…

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions