devenv container should run in regular user

[pbek]

I saw https://devenv.sh/integrations/devenv-container/#__tabbed_1_2 to run a script in a (private) GitLab CI, but is it also possible to run processes, for example with devenv test? Locally on NixOS is worked great.

When I run devenv test I get:

Log

$ devenv test
Overriding .devenv to .devenv.fy4qO7
• Building tests ...
Using Cachix: devenv
Trusting devenv.cachix.org on first use with the public key devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=
✔ Building tests in 52.9s
• Building processes ...
✔ Building processes in 11.3s
• Starting processes ...
• Building shell ...
✔ Building shell in 3.16s
PID is 1302
Stop:      $ devenv processes stop
✔ Starting processes in 3.78s
• Running tests ...
• Building shell ...
💡 The dotenv file '.env.dev.local' was not found.
Running tasks     devenv:enterShell
Running           devenv:enterShell 0ms
1 Running                           60.05µs
Succeeded         devenv:enterShell 18ms
1 Succeeded                         18.85ms
🔨 Middleware API shell
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql	] Installing MariaDB/MySQL system tables in '/builds/dbp/middleware/api/.devenv.fy4qO7/state/mysql' ...
[mysql	] 2025-04-15  8:04:12 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 95
[mysql	] 2025-04-15  8:04:12 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
✔ Building shell in 1.80s
[mysql	] OK
[mysql	] 
[mysql	] To start mariadbd at boot time you have to copy
[mysql	] support-files/mariadb.service to the right place for your system
[mysql	] 
[mysql	] 
[mysql	] PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
[mysql	] To do so, start the server, then issue the following command:
[mysql	] 
[mysql	] '/nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/bin/mariadb-secure-installation'
[mysql	] 
[mysql	] which will also give you the option of removing the test
[mysql	] databases and anonymous user created by default.  This is
[mysql	] strongly recommended for production servers.
[mysql	] 
[mysql	] See the MariaDB Knowledgebase at https://mariadb.com/kb
[mysql	] 
[mysql	] You can start the MariaDB daemon with:
[mysql	] cd '/nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11' ; /nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/bin/mariadbd-safe --datadir='/builds/dbp/middleware/api/.devenv.fy4qO7/state/mysql'
[mysql	] 
[mysql	] You can test the MariaDB daemon with mysql-test-run.pl
[mysql	] cd '/nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/OFF' ; perl mariadb-test-run.pl
[mysql	] 
[mysql	] Please report any problems at https://mariadb.org/jira
[mysql	] 
[mysql	] The latest information about MariaDB is available at https://mariadb.org/.
[mysql	] 
[mysql	] Consider joining MariaDB's strong and vibrant community:
[mysql	] https://mariadb.org/get-involved/
[mysql	] 
[mysql	] /nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/bin/mysqld: Please consult the Knowledge Base to find out how to run mysqld as root!
[mysql	] 2025-04-15  8:04:14 0 [ERROR] Aborting
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
• Executing in shell ...
💡 The dotenv file '.env.dev.local' was not found.
Running tasks     devenv:enterShell
Running           devenv:enterShell 0ms
1 Running                           102.90µs
Succeeded         devenv:enterShell 5ms
1 Succeeded                         5.88ms
🔨 Middleware API shell
• Setting up shell environment ...
💡 The dotenv file '.env.dev.local' was not found.
Running tasks     devenv:enterShell
Running           devenv:enterShell 0ms
1 Running                           51.54µs
Succeeded         devenv:enterShell 5ms
1 Succeeded                         6.59ms
🔨 Middleware API shell
• Testing ...
Running tasks     devenv:enterTest
Running           devenv:enterTest  0ms
1 Running                           118.57µs
Not implemented   devenv:enterTest  
1 Skipped                           146.16µs
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
✔ Executing in shell in 15.1s
✔ Running tests in 17.4s
Stopping process with PID 1302
Tests failed :(
Stopping processes...
Project Completed. Press Ctrl+C to quit
Processes stopped.
Error:   × Tests failed

with

devenv.nix

{
  pkgs,
  lib,
  config,
  inputs,
  ...
}:

{
  # https://devenv.sh/supported-languages/php/
  languages.php = {
    enable = true;
    version = "8.3";
    ini = ''
      memory_limit = 256M
    '';
  };

  # https://devenv.sh/basics/
  env = {
    CORE_DATABASE_HOST = "localhost";
    FORMALIZE_DATABASE_HOST = "localhost";
    BLOB_CONNECTOR_LOCAL_DATABASE_HOST = "localhost";
    MONO_DATABASE_HOST = "localhost";
    DISPATCH_DATABASE_HOST = "localhost";
    AUTHORIZATION_DATABASE_HOST = "localhost";

    CORE_DATABASE_PASSWORD = "secret";
    FORMALIZE_DATABASE_PASSWORD = "secret";
    BLOB_CONNECTOR_LOCAL_DATABASE_PASSWORD = "secret";
    MONO_DATABASE_PASSWORD = "secret";
    DISPATCH_DATABASE_PASSWORD = "secret";
    AUTHORIZATION_DATABASE_PASSWORD = "secret";
  };

  services.mysql = {
    enable = true;
    initialDatabases = [
      { name = "db1"; }
      { name = "db2"; }
    ];
    ensureUsers = [
      {
        name = "db";
        password = "secret";
        ensurePermissions = {
          "db1.*" = "ALL PRIVILEGES";
          "db2.*" = "ALL PRIVILEGES";
        };
      }
    ];
  };

  # https://devenv.sh/integrations/dotenv/
  dotenv.enable = true;
  dotenv.filename = [
    ".env"
    ".env.dev"
    ".env.dev.local"
  ];

  # https://devenv.sh/packages/
  #  packages = with pkgs; [
  #    # For treefmt
  #    just
  #    nodePackages.prettier
  #    nixfmt-rfc-style
  #    shfmt
  #    statix
  #    taplo
  #    php83Packages.php-cs-fixer
  #  ];

  # https://devenv.sh/scripts/
  scripts = {
    db-clean.exec = ''
      echo "📥 Cleaning DB state"
      rm .devenv/state/mysql/ -Rf
    '';

    db-migrate.exec = ''
      echo "📥 Running DB migrations"
      php bin/console dbp:relay:core:migrate --no-interaction
    '';
  };

  enterShell = ''
    echo "🔨 Middleware API shell"
  '';

  # https://devenv.sh/tests/
  enterTest = ''
    wait_for_port 3306
    echo "🔨 Doing composer install"
    composer install
    db-clean
    db-migrate
  '';

  # https://devenv.sh/git-hooks/
  # git-hooks.hooks.treefmt.enable = true;

  # See full reference at https://devenv.sh/reference/options/
}

and

devenv.yaml

inputs:
  nixpkgs:
    url: github:NixOS/nixpkgs/nixos-unstable

Has anyone experience with that? I tried playing around with https://docs.gitlab.com/ci/services/ to run it, but didn't get far yet.

[domenkozar]

[mysql ] /nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/bin/mysqld: Please consult the Knowledge Base to find out how to run mysqld as root!

What container image are you using? devenv one? Looks like we need to set it up to run as regular user.

[pbek]

Yes, ghcr.io/cachix/devenv/devenv:latest as in the devenv doc.

[pbek]

Just doing main...pbek:devenv:feature/container-regular-user didn't do the trick for me yet...

I still get:

Expand

Overriding .devenv to .devenv.4zxGhe
• Building tests ...
Using Cachix: devenv
Trusting devenv.cachix.org on first use with the public key devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=
✔ Building tests in 55.0s
• Building processes ...
✔ Building processes in 11.4s
• Starting processes ...
• Building shell ...
✔ Building shell in 3.10s
PID is 1298
Stop:      $ devenv processes stop
✔ Starting processes in 3.84s
• Running tests ...
• Building shell ...
💡 The dotenv file '.env.dev.local' was not found.
✔ Building shell in 33.0ms
• Executing in shell ...
Running tasks     devenv:enterShell
Running           devenv:enterShell 0ms
1 Running                           243.45µs
Succeeded         devenv:enterShell 6ms
1 Succeeded                         7.40ms
✨ devenv 1.5.2 is newer than devenv input (1.5.0) in devenv.lock. Run 'devenv update' to sync.
🔨 Middleware API shell
💡 The dotenv file '.env.dev.local' was not found.
Running tasks     devenv:enterShell
Running           devenv:enterShell 0ms
1 Running                           65.30µs
Succeeded         devenv:enterShell 5ms
1 Succeeded                         5.98ms
✨ devenv 1.5.2 is newer than devenv input (1.5.0) in devenv.lock. Run 'devenv update' to sync.
🔨 Middleware API shell
• Setting up shell environment ...
💡 The dotenv file '.env.dev.local' was not found.
Running tasks     devenv:enterShell
Running           devenv:enterShell 0ms
1 Running                           123.26µs
Succeeded         devenv:enterShell 5ms
1 Succeeded                         5.94ms
✨ devenv 1.5.2 is newer than devenv input (1.5.0) in devenv.lock. Run 'devenv update' to sync.
🔨 Middleware API shell
• Testing ...
Running tasks     devenv:enterTest
Running           devenv:enterTest  0ms
1 Running                           64.04µs
Not implemented   devenv:enterTest  
1 Skipped                           77.39µs
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql	] Installing MariaDB/MySQL system tables in '/builds/dbp/middleware/api/.devenv.4zxGhe/state/mysql' ...
[mysql	] 2025-04-18  6:48:43 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 95
[mysql	] 2025-04-18  6:48:43 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql	] OK
[mysql	] 
[mysql	] To start mariadbd at boot time you have to copy
[mysql	] support-files/mariadb.service to the right place for your system
[mysql	] 
[mysql	] 
[mysql	] PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
[mysql	] To do so, start the server, then issue the following command:
[mysql	] 
[mysql	] '/nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/bin/mariadb-secure-installation'
[mysql	] 
[mysql	] which will also give you the option of removing the test
[mysql	] databases and anonymous user created by default.  This is
[mysql	] strongly recommended for production servers.
[mysql	] 
[mysql	] See the MariaDB Knowledgebase at https://mariadb.com/kb
[mysql	] 
[mysql	] You can start the MariaDB daemon with:
[mysql	] cd '/nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11' ; /nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/bin/mariadbd-safe --datadir='/builds/dbp/middleware/api/.devenv.4zxGhe/state/mysql'
[mysql	] 
[mysql	] You can test the MariaDB daemon with mysql-test-run.pl
[mysql	] cd '/nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/OFF' ; perl mariadb-test-run.pl
[mysql	] 
[mysql	] Please report any problems at https://mariadb.org/jira
[mysql	] 
[mysql	] The latest information about MariaDB is available at https://mariadb.org/.
[mysql	] 
[mysql	] Consider joining MariaDB's strong and vibrant community:
[mysql	] https://mariadb.org/get-involved/
[mysql	] 
[mysql	] /nix/store/7bl2a6sg29z3kdj4s71c4lz2g3ia71np-mariadb-server-10.11.11/bin/mysqld: Please consult the Knowledge Base to find out how to run mysqld as root!
[mysql	] 2025-04-18  6:48:45 0 [ERROR] Aborting
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
[mysql-configure	] Sleeping 1s while we wait for MySQL to come up
✔ Executing in shell in 15.0s
✔ Running tests in 15.1s
Stopping process with PID 1298
Tests failed :(
Stopping processes...
Project Completed. Press Ctrl+C to quit
Processes stopped.
Error:   × Tests failed

[pbek]

Sorry, my bad, the devenv GitHub workflow of my fork built from main branch, not my feature branch. I'll try again. 😉

[pbek]

User nixos doesn't exist, so I added one directly into /etc/passwd (since useradd can't write the file): https://github.com/pbek/devenv/blob/36c766308d7a7c96297b46fbf610d8920079fd23/containers/devenv/Dockerfile#L17-L27

(Is there a better way to do that in the nixos docker image? Normal nixcfg?)

But the GitLab CI job threw an error immediately:

Overriding .devenv to .devenv.DiNErh
• Building tests ...
error:
       error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted
Failed to get cachix caches due to evaluation error
error:
       error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted
✖ Building tests in 62.8ms
Error:   × Command `/nix/store/ni24hs509kjhi1ckm6wxhhj8h333437l-nix-devenv-
  │ 2.24.0pre20250312_f3f44b2/bin/nix --show-trace --extra-experimental-
  │ features nix-command --extra-experimental-features flakes --option warn-
  │ dirty false --keep-going --max-jobs 3 --option eval-cache false build
  │ --no-link --print-out-paths .#devenv.test -vv --log-format internal-json`
  │ failed with with exit code 1
Cleaning up project directory and file based variables

Any advice, @domenkozar?