Monorepo / Tarball Generation Performance

[DGAISmith]

Hi there, we're having some issues implementing TestContainers in our monorepo. I have a (hacky) workaround in mind, but I just wanted to know if there was an easier way with the configuration options already available. I've tried all sorts of combinations with limited success.

We have an Nx monorepo which contains a mix of react, dotnet, and python apps / libraries. The basic structure, and where I've placed the various docker files, is like this -

monorepo-root
├──.github
├──apps
│  └──app1
│      └──app1.app
│         └──app1.app.csproj
│         └──Dockerfile
│  └──app2
│      └──app2.app
│         └──app2.app.csproj
│         └──Dockerfile
├──dist
│  └──app1
│  └──app2
│  └──library1
│  └──library2
├──libs
│  └──library1
│     └──library1.csproj
│  └──library2
│     └──library2.csproj
├──node_modules
├──.dockerignore

The dotnet apps are function apps, and we are building them into an image prior to starting up a container for them. Because the Dockerfiles in each app need to reference the monorepo-root/libs folder, we are forced to use the following directory targeting:

        return new ImageFromDockerfileBuilder()
           .WithName(containerBuilderArgs.GenerateResourceName("image"))
           .WithDockerfileDirectory(CommonDirectoryPath.GetGitDirectory(), string.Empty)
           .WithDockerfile(dockerFileRelativePath)
           .WithLogger(containerBuilderArgs.Logger)
           .WithBuildArgument("RESOURCE_REAPER_SESSION_ID", ResourceReaper.DefaultSessionId.ToString("D"));

So WithDockerfileDirectory will resolve to the monorepo-root, and dockerFileRelativePath resolves to "apps/app1/app1.app/Dockerfile" (or whatever app we're running tests for).

This works, but local performance is awful (over 30 mins) when building the tarball. In the above structure, I've left out tests, tools, pipelines, many other files. When working locally, developers will build many apps over time and the /dist folder (contains equivalents of obj/bin folders) and node_modules folders will become very large. Looking at my local state, I currently have 268775 files in 42258 folders, total size is over 4Gb. I have entries in .dockerignore to exclude dist, node-modules and others but it doesn't help the performance. I assume this is because when generating the tarball, TestContainers.net is doing this:

      using (var tarOutputFileStream = new FileStream(dockerfileArchiveFilePath, FileMode.Create, FileAccess.Write))
      {
        using (var tarOutputStream = new TarOutputStream(tarOutputFileStream, Encoding.Default))
        {
          tarOutputStream.IsStreamOwner = false;

          foreach (var absoluteFilePath in GetFiles(dockerfileDirectoryPath))
          {
            // SharpZipLib drops the root path: https://github.com/icsharpcode/SharpZipLib/pull/582.
            var relativeFilePath = absoluteFilePath.Substring(dockerfileDirectoryPath.TrimEnd(Path.AltDirectorySeparatorChar).Length + 1);

            if (dockerIgnoreFile.Denies(relativeFilePath))
            {
              continue;
            }

I guess it's having to check each file, and then seeing if dockerignore is excluding it.

My question then - is there a way to target the field of view of the tarball generation to avoid the need to scan everything from the WithDockerfileDirectory down, just to exclude 95% of the files? I can't move the dockerfileDirectory up to the app1 level because I also need the libs. But needing the libs means I bring the entire monorepo into scope.

If not, my hacky workaround is to copy the app in question and the entire libs folder to a temporary staging area and use that to build the image.

Many thanks for bearing with me, and thanks in advance for any help!

[HofmeisterAn]

No, unfortunately there isn't. But it's something that could be added. I generally don't recommend using the image builder approach because it can't take advantage of BuildKit. BuildKit isn't available through the Docker Engine API.

[DGAISmith]

Many thanks, we went ahead and added a step to isolate the app-under-test via file copy and it's sorted things out. I'll have to have a look at BuildKit, see if there are any nice solutions by going down that route.