Skip to content

Podman support in scripts#363

Open
mattw-nws wants to merge 10 commits into
CIROH-UA:mainfrom
mattw-nws:guidesh-podman-support
Open

Podman support in scripts#363
mattw-nws wants to merge 10 commits into
CIROH-UA:mainfrom
mattw-nws:guidesh-podman-support

Conversation

@mattw-nws
Copy link
Copy Markdown

Enable the use of Podman in place of Docker, primarily to support environments where Docker is not permitted.

Some additional docs are probably needed if there's interest in this, but making it available since I needed to make this modification for our environment.

Additions

  • Adds -p option to substitute podman for docker command in scripts

Removals

  • NA

Updates

  • Adjusts language in some places subbing "container" for "docker" (e.g. "... image")

Testing

Method

Ran through workflows with a Podman system... not all combinations probably tested well. This might be a best-effort-to-unsupported feature.

Screenshots / output snippets

$ ./guide.sh -h
Usage: guide.sh [arg ...]
Options:
  -d [path]: Designates the provided path as the data directory to import into the visualizer.
  -h: Displays usage information, then exits.
  -i [image]: Specifies which Docker image of NGIAB to run.
  -p: Use Podman instead of Docker.
  -r: Retains previous console output when launching the script.
  -t [tag]: Specifies which Docker image tag of NGIAB to run.

→ Cleaning up resources...
  ✓ Cleanup completed.

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)

Testing checklist

Target Environment support

  • Windows (wsl)
  • Linux
  • MacOs (apple silicon)

@benlee0423
Copy link
Copy Markdown
Contributor

@mattw-nws
What you proposed here works in your environment?

Comment thread guide.sh Outdated
Comment thread guide.sh
echo -e " ${CHECK_MARK} $DOCKER_CMD detected (version: ${BGreen}$docker_version${Color_Off})"
else
handle_error "Docker not found. This script requires Docker to run the NextGen model."
handle_error "$DOCKER_CMD not found. This script requires Docker or Podman to run the NextGen model."
Copy link
Copy Markdown
Contributor

@benlee0423 benlee0423 Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattw-nws
This check fails in Ubuntu with podman installed.
I ran with .guide.sh -p

ERROR: docker not found. This script requires Docker or Podman to run the NextGen model.
Can you please take a look?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run: ./guide.sh -p

It will work.

@benlee0423
Copy link
Copy Markdown
Contributor

Podman cannot directly download from docker.io, and need to add the following
unqualified-search-registries = ["docker.io"]
in /etc/containers/registries.conf

@arpita0911patel arpita0911patel requested a review from kamalcou May 4, 2026 14:36
@kamalcou
Copy link
Copy Markdown
Collaborator

kamalcou commented May 4, 2026

tested and verified.

@kamalcou kamalcou closed this May 4, 2026
@github-project-automation github-project-automation Bot moved this from Todo to Done in NGIAB Ecosystem May 4, 2026
@kamalcou kamalcou reopened this May 4, 2026
Copy link
Copy Markdown
Collaborator

@kamalcou kamalcou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes work. For podman, unqualified-search-registries = ["docker.io"] need to be added in /etc/containers/registries.conf

@JoshCu
Copy link
Copy Markdown
Member

JoshCu commented May 4, 2026

adding docker:// to the front of the image name means people don't need to add docker.io to their registries manually
e.g.
podman pull docker://awiciroh/ciroh-ngen-image
image

it breaks docker though so we'd need to set it as another variable rather than just adding it to everything
image

@arpita0911patel arpita0911patel requested a review from quinnylee May 12, 2026 13:26
@arpita0911patel arpita0911patel moved this from Done to In progress in NGIAB Ecosystem May 12, 2026
Comment thread guide.sh Outdated
@kamalchowdhury326
Copy link
Copy Markdown

kamalchowdhury326 commented May 13, 2026

@mattw-nws In the guide.sh line 446, docker pull $IMAGE_NAME should be changed to $DOCKER_CMD pull $IMAGE_NAME

Copy link
Copy Markdown
Collaborator

@quinnylee quinnylee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general comment that seems wholly unrelated to this PR and is probably just related to how I installed podman, I am not able to get past t-route when I run NGIAB through podman on my macbook pro (32 cores, 64 GB of RAM). The error message is pretty useless but my guess is that the VM is too small and t-route is too big? I have no issues running t-route/NGIAB on my macbook through docker.

2026-05-14 14:03:26,074 - root - INFO - [compute.py:660 - compute_nhd_routing_v02]: starting Parallel JIT calculation
/ngen/HelloNGEN.sh: line 137:     8 Killed                  $run_command

Comment thread guide.sh Outdated
Comment thread guide.sh Outdated
@quinnylee quinnylee dismissed their stale review May 14, 2026 17:10

resolved through my commits

@quinnylee quinnylee requested a review from benlee0423 May 14, 2026 17:10
@quinnylee quinnylee marked this pull request as ready for review May 14, 2026 17:10
@quinnylee
Copy link
Copy Markdown
Collaborator

quinnylee commented May 14, 2026

Tested on windows with WSL, macOS, ubuntu. The NextGen portion runs correctly on all OS with both docker and podman.

A couple notes

TEEHR

EDIT: TEEHR now works with podman thanks to commit 4821ed9. Keeping this here because I think it's interesting

TEEHR does not work with podman because of this:

Traceback (most recent call last):
  File "/app/teehr_ngen.py", line 184, in <module>
    main()
  File "/app/teehr_ngen.py", line 43, in main
    ev.enable_logging()
  File "/usr/local/lib/python3.11/site-packages/teehr/evaluation/evaluation.py", line 188, in enable_logging
    handler = logging.FileHandler(Path(self.dir_path, 'teehr.log'))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 1181, in __init__
    StreamHandler.__init__(self, self._open())
                                 ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 1213, in _open
    return open_func(self.baseFilename, self.mode,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/app/data/teehr/teehr.log'

TEEHR

I am not really sure how to fix this, but I suspect it has something to do with ownership or security differences. Here is what ownership and security context looks like when the TEEHR image is run through Docker:

root@a39a7cfbe26a:/app/data/teehr# ls -lZ
total 24
-rw-r--r--  1 root root ?   42 May 14 19:29 __init__.py
drwxr-xr-x  7 root root ?  224 May 14 19:29 cache
drwxr-xr-x 12 root root ?  384 May 14 19:29 dataset
-rw-r--r--  1 root root ? 3481 May 14 19:29 ngen_usgs_crosswalk.parquet
-rw-r--r--  1 root root ? 2940 May 14 19:29 nwm_usgs_crosswalk.parquet
-rw-r--r--  1 root root ?  593 May 14 19:29 readme.md
drwxr-xr-x  4 root root ?  128 May 14 19:29 scripts
-rw-r--r--  1 root root ? 7008 May 14 19:29 teehr.log

And here is what it looks like when run through Podman:

root@b645f3df8f2e:/app/data/teehr# ls -lZ
total 24
-rw-r--r--.  1 root root system_u:object_r:nfs_t:s0   42 May 14 19:29 __init__.py
drwxr-xr-x.  7 root root system_u:object_r:nfs_t:s0  224 May 14 19:29 cache
drwxr-xr-x. 12 root root system_u:object_r:nfs_t:s0  384 May 14 19:29 dataset
-rw-r--r--.  1 root root system_u:object_r:nfs_t:s0 3481 May 14 19:29 ngen_usgs_crosswalk.parquet
-rw-r--r--.  1 root root system_u:object_r:nfs_t:s0 2940 May 14 19:29 nwm_usgs_crosswalk.parquet
-rw-r--r--.  1 root root system_u:object_r:nfs_t:s0  593 May 14 19:29 readme.md
drwxr-xr-x.  4 root root system_u:object_r:nfs_t:s0  128 May 14 19:29 scripts
-rw-r--r--.  1 root root system_u:object_r:nfs_t:s0 7008 May 14 19:29 teehr.log

The fix is very simple, we just append :Z to the TEEHR command. See https://developers.redhat.com/articles/2025/04/11/my-advice-selinux-container-labeling#2__run_container_with_a_private_volume_

There are also some issues with running NGIAB with Docker and then with Podman in that order. Docker creates the teehr directory with root permissions, and then if you run the exact same simulation but with Podman, the teehr directory is still owned by root. This causes ownership problems in Podman (it expects the directory to be owned by the user). But this is probably such a rare case that I won't even worry about it

Tethys

  • the Tethys visualizer requires sudo access, which might render it useless for users without sudo access
  • the Tethys port must be >=1024 with podman use, otherwise you get something like this:
Error: rootlessport cannot expose privileged port 80, you can add 'net.ipv4.ip_unprivileged_port_start=80' to /etc/sysctl.conf (currently 1024), or choose a larger port number (>= 1024): listen tcp 0.0.0.0:80: bind: permission denied
  ✗ Failed to start Tethys container.
✗ Failed to start Tethys container. Exiting.

It's probably simple enough to change the default port to 8080 and include a little message for podman users during the port selection step

@arpita0911patel
Copy link
Copy Markdown
Member

arpita0911patel commented May 14, 2026

@samlamont @romer8 - please take alook when you get chance, we are planning to have this feature added before DevCon if possible.

@arpita0911patel
Copy link
Copy Markdown
Member

Thank you Quinn for looking into this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

8 participants