Skip to content

fix(codeserver): fix 502 bad gateway for culling probe#3092

Open
EdisonSu768 wants to merge 1 commit intoopendatahub-io:mainfrom
EdisonSu768:fix-codeserver-culling-502
Open

fix(codeserver): fix 502 bad gateway for culling probe#3092
EdisonSu768 wants to merge 1 commit intoopendatahub-io:mainfrom
EdisonSu768:fix-codeserver-culling-502

Conversation

@EdisonSu768
Copy link
Copy Markdown

@EdisonSu768 EdisonSu768 commented Mar 13, 2026

Fixes #3091

Summary

Fix 502 Bad Gateway error that occurs during Activity Culling probe in Code Server workbench.

Changes

  • Add mod_unixd module to httpd.conf (required for CGI execution on Unix systems)
  • Remove mod_mime module (not needed for this use case)
  • Add DirectoryIndex access.cgi to ensure proper routing to the CGI script

Problem

The culling probe was returning 502 errors because the Apache httpd server wasn't properly configured to execute CGI scripts. The missing mod_unixd module prevented the CGI server from starting correctly.

Solution

This fix ensures the /api/kernels endpoint correctly routes to access.cgi for Activity Culling functionality.

Add mod_unixd, remove mod_mime, and add DirectoryIndex to httpd.conf to ensure the CGI server starts and correctly routes /api/kernels to access.cgi for Activity Culling.
@openshift-ci openshift-ci Bot requested review from daniellutz and jiridanek March 13, 2026 08:48
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 13, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jiridanek for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions github-actions Bot added the review-requested GitHub Bot creates notification on #pr-review-ai-ide-team slack channel label Mar 13, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 13, 2026

Hi @EdisonSu768. Thanks for your PR.

I'm waiting for a opendatahub-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 13, 2026

📝 Walkthrough

Walkthrough

Modified Apache httpd configuration file to load the unixd_module, remove mime_module, and set access.cgi as the directory index for the /opt/app-root directory. Total of 2 lines added and 1 line removed.

Changes

Cohort / File(s) Summary
Apache HTTPD Configuration
codeserver/ubi9-python-3.12/httpd/httpd.conf
Added unixd_module loading, removed mime_module loading, and configured DirectoryIndex to prioritize access.cgi for the /opt/app-root directory.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title uses imperative mood ('fix') and directly describes the main change, but lacks a required ticket reference.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Branch Prefix Policy ✅ Passed The PR title follows conventional commit format without branch prefix, which is compliant with main branch requirements.
Description check ✅ Passed The PR description includes all required template sections with detailed technical information about the problem, solution, and changes made.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot added size/xs and removed size/xs labels Mar 13, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@codeserver/ubi9-python-3.12/httpd/httpd.conf`:
- Around line 6-10: The Apache config is missing the LoadModule mime_module
declaration required for AddHandler cgi-script .cgi to work; add a LoadModule
mime_module modules/mod_mime.so entry alongside the other LoadModule lines (near
the existing mpm_prefork_module, unixd_module, authz_core_module, dir_module,
cgi_module entries) so mod_mime is loaded before the AddHandler registration and
the config can be parsed successfully.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 936cacc7-5b37-4634-84cf-48e3868b1d8c

📥 Commits

Reviewing files that changed from the base of the PR and between 586e188 and bf771e5.

📒 Files selected for processing (1)
  • codeserver/ubi9-python-3.12/httpd/httpd.conf

Comment on lines 6 to 10
LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule mime_module modules/mod_mime.so
LoadModule dir_module modules/mod_dir.so
LoadModule cgi_module modules/mod_cgi.so
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether AddHandler is present without mod_mime being loaded.
rg -nP '^\s*LoadModule\s+mime_module\b|^\s*AddHandler\s+cgi-script\b' \
  codeserver/ubi9-python-3.12/httpd/httpd.conf \
  codeserver/ubi9-python-3.12/httpd/codeserver-cgi.conf

Repository: opendatahub-io/notebooks

Length of output: 234


🏁 Script executed:

head -n 40 codeserver/ubi9-python-3.12/httpd/httpd.conf | cat -n

Repository: opendatahub-io/notebooks

Length of output: 1298


Re-add LoadModule mime_module to prevent Apache config parse failure.

AddHandler cgi-script .cgi at line 34 requires mod_mime to function. This module is not currently loaded in lines 6–11. Apache will fail to parse the config and the CGI handler registration will not occur, blocking the probe endpoint.

Proposed fix
 LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
 LoadModule unixd_module modules/mod_unixd.so
 LoadModule authz_core_module modules/mod_authz_core.so
 LoadModule dir_module modules/mod_dir.so
+LoadModule mime_module modules/mod_mime.so
 LoadModule cgi_module modules/mod_cgi.so
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@codeserver/ubi9-python-3.12/httpd/httpd.conf` around lines 6 - 10, The Apache
config is missing the LoadModule mime_module declaration required for AddHandler
cgi-script .cgi to work; add a LoadModule mime_module modules/mod_mime.so entry
alongside the other LoadModule lines (near the existing mpm_prefork_module,
unixd_module, authz_core_module, dir_module, cgi_module entries) so mod_mime is
loaded before the AddHandler registration and the config can be parsed
successfully.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed. Removing mod_mime causes Apache to fail at config parse time with:

AH00526: Syntax error on line 34 of /etc/httpd/conf/httpd.conf:
Invalid command 'AddHandler', perhaps misspelled or defined by a module not included in the server configuration

This means httpd never starts, and nginx returns 502 for every request to /api/kernels/. mod_mime must be kept (or AddHandler replaced with SetHandler, which is provided by the core and does not require mod_mime).

@jiridanek
Copy link
Copy Markdown
Member

/ok-to-test

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 13, 2026

@EdisonSu768: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/codeserver-ubi9-python-3-12-pr-image-mirror bf771e5 link true /test codeserver-ubi9-python-3-12-pr-image-mirror
ci/prow/images bf771e5 link true /test images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.


# Load modules
LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
LoadModule unixd_module modules/mod_unixd.so
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai isn't this only needed if httpd is started as root? and in our containers we never do that

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jiridanek — That's a great observation. You're partially correct:

  • The mod_unixd module provides the User and Group directives, whose primary purpose is to drop privileges after Apache starts as root (which is needed to bind to privileged ports like 80/443).
  • For those specific directives, the server must be run initially as root — if started as a non-root user, the User/Group drop is simply skipped.

However, mod_unixd goes beyond just User/Group. On Unix-like systems, it is classified as a Base module and also provides lower-level OS environment initialization functions (such as os_init_job_environment) that the mpm_prefork_module calls during startup, regardless of whether httpd is running as root. That is why you typically see mod_unixd listed as a hard dependency in Unix Apache configurations even in containerized, non-root environments.

Notably, mod_fcgid explicitly requires mod_unixd to be loaded before itself in the httpd config — and similar internal linkage exists for prefork-based CGI setups.

So while the User/Group directives are indeed no-ops in an OpenShift container (since httpd starts as a random non-root UID), mod_unixd itself is still a required module for the prefork MPM to initialize correctly on Unix. That said, it would be worth the PR author (@EdisonSu768) confirming whether the actual fix for the 502 was:

  1. Adding mod_unixd (fixing a startup/initialization failure), or
  2. Adding DirectoryIndex access.cgi (fixing the routing of requests to the CGI script).

The DirectoryIndex change looks like the more directly impactful fix for ensuring the culling probe reaches access.cgi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test review-requested GitHub Bot creates notification on #pr-review-ai-ide-team slack channel size/xs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Code Server Culling 502 Bad Gateway: Missing mod_unixd and DirectoryIndex

2 participants