Run DockerComposeContainer on GitLab CI

[arvkonstantin]

I've made a lot of attempts..

I have two runners:

concurrent = 1
check_interval = 0
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "Docker in docker"
  url = "******"
  id = 26
  token = "your_token"
  token_obtained_at = 2023-05-29T13:04:02Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.docker]
    tls_verify = false
    image = "docker:19.03-dind"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    pull_policy = "if-not-present"
    volumes = ["/certs/client", "/cache"]
    shm_size = 0
    environment = [
        "DOCKER_HOST=tcp://localhost:2375",
    ]

[[runners]]
  name = "MACHINE_NAME"
  url = "******"
  token = "GENERATED_GITLAB_RUNNER_TOKEN"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0

And I have a test that runs compose:

  override def startContainers(): Containers = {
    val dockerComposeContainer = DockerComposeContainer(
        DockerComposeContainer.ComposeFile(Left(new File("src/test/resources/docker/docker-compose.yml"))),
        exposedServices = Seq(
*********
        )
      )
    dockerComposeContainer.start()
    dockerComposeContainer
  }

Locally - everything works for me, but something is constantly wrong in CI.
I have tried several ci configurations:

test:
  stage: test
  tags:
    - dind
  variables:
    # Instruct Testcontainers to use the daemon of DinD, use port 2375 for non-tls connections.
    DOCKER_HOST: "tcp://docker:2375"
    # Instruct Docker not to start over TLS.
    DOCKER_TLS_CERTDIR: ""
    # Improve performance with overlayfs.
    DOCKER_DRIVER: overlay2
  image: ******/artifacts/build-tools/scala-sbt-docker-image:latest
  services:
    - name: docker:dind
      # explicitly disable tls to avoid docker startup interruption
      command: ["--tls=false"]
  cache:
    paths:
      - target/
  script:
    - sbt clean coverage test coverageReport
    - mkdir -p public
    - mkdir -p public/coverage
    - cp -rv target/${SCALA}/scoverage-report public/coverage
  artifacts:
    expire_in: 1 days
    name: coveragereport
    paths:
      - public/coverage

and this(with second runner):

test:
  stage: test
  tags:
    - test
  image: ******/artifacts/build-tools/scala-sbt-docker-image:latest
  cache:
    paths:
      - target/
  script:
    - sbt clean coverage test coverageReport
    - mkdir -p public
    - mkdir -p public/coverage
    - cp -rv target/${SCALA}/scoverage-report public/coverage
  artifacts:
    expire_in: 1 days
    name: coveragereport
    paths:
      - public/coverage

The most common mistakes:

1. the application cannot connect to the service from compose by port
2. can't wait for the service to start (waits for the port to be listened to)