Skip to content

fix: python 3.12 compatibility#380

Open
Copilot wants to merge 7 commits intomasterfrom
copilot/fix-github-actions-tests
Open

fix: python 3.12 compatibility#380
Copilot wants to merge 7 commits intomasterfrom
copilot/fix-github-actions-tests

Conversation

Copy link
Copy Markdown

Copilot AI commented Apr 8, 2026

pkg_resources is no longer available by default in Python 3.12 environments, causing ModuleNotFoundError on import of edx_sga.sga and cascading failures across all 27 tests in CI.

What are the relevant tickets?

N/A

What's this PR do?

  • edx_sga/sga.py: Replace import pkg_resources with from importlib.resources import files (stdlib, Python 3.9+). Update _resource() and load_resource() to use files(__package__).joinpath(path).read_bytes() — using __package__ (edx_sga) rather than __name__ (edx_sga.sga) ensures resources resolve against the package directory correctly, including in zipped distributions.
  • test_requirements.txt: Upgrade edx-lint from 5.3.66.0.0 — old version used pkg_resources internally in its pylint plugin, causing a second failure point.
  • pylintrc: Regenerated via edx_lint write pylintrc for edx-lint 6.0.0 compatibility.
  • edx_sga/test_settings.py: Migrated the deprecated DEFAULT_FILE_STORAGE setting to the STORAGES dictionary format required by Django 4.2+.
  • edx_sga/tests/common.py: Updated test setup/teardown helpers to save and restore STORAGES["default"] instead of the removed DEFAULT_FILE_STORAGE setting.
  • edx_sga/tests/integration_tests.py: Fixed user creation in the staff/non-staff parameterized test helper to create a proper User + UserProfile pair instead of a bare User.objects.create().
  • run_edx_integration_tests.sh: Added fallback logic to copy pyproject.toml from edx-platform when setup.cfg is absent (newer edx-platform releases no longer ship setup.cfg).
# Before
import pkg_resources

def _resource(path):
    data = pkg_resources.resource_string(__name__, path)
    return data.decode("utf8")

# After
from importlib.resources import files

def _resource(path):
    data = files(__package__).joinpath(path).read_bytes()
    return data.decode("utf8")

How should this be manually tested?

Run tox -e py312-django42 and tox -e py312-django52 — both should pass all 64 tests with no pylint errors.

Where should the reviewer start?

edx_sga/sga.py lines 9–18 (import change) and the _resource/load_resource functions near the bottom of the file. Also review edx_sga/test_settings.py and edx_sga/tests/common.py for the STORAGES migration, and run_edx_integration_tests.sh for the setup.cfg/pyproject.toml fallback.

Any background context you want to provide?

pkg_resources is provided by setuptools, which is no longer bundled by default in Python 3.12 venvs. importlib.resources is the stdlib replacement and has been available since Python 3.9. files(__package__) is used instead of files(__name__) because __name__ refers to the module edx_sga.sga rather than the edx_sga package where static resources reside.

The DEFAULT_FILE_STORAGE setting was deprecated in Django 4.2 in favour of the STORAGES dict; test scaffolding has been updated accordingly. The integration test runner now gracefully handles edx-platform repositories that have migrated from setup.cfg to pyproject.toml.

Screenshots (if appropriate)

N/A

What GIF best describes this PR or how it makes you feel?

N/A

Original prompt

Fix failing GitHub Actions tests in mitodl/edx-sga

Tests are currently failing in the GitHub Actions CI workflow for the mitodl/edx-sga repository. Please investigate the failing tests, identify the root causes, and fix them.

Steps to follow

  1. Check the most recent GitHub Actions workflow run(s) on the master branch to identify which tests are failing and what errors are reported.
  2. Review the relevant source files, test files, and configuration files (e.g., tox.ini, setup.py, requirements*.txt) to understand the failure context.
  3. Apply the necessary code fixes to make the tests pass.

Known issues to look for (based on prior analysis)

  • ModuleNotFoundError: No module named 'pkg_resources' on Python 3.12 — caused by import pkg_resources in edx_sga/sga.py. Fix by replacing pkg_resources usage with importlib.resources.files (available in Python 3.9+ stdlib).
  • Cascading AttributeError: module 'edx_sga' has no attribute 'sga' — caused by the above import failure leaving edx_sga partially initialized.

Expected fix for edx_sga/sga.py

  • Remove: import pkg_resources
  • Add: from importlib.resources import files
  • Replace _resource():
def _resource(path):  # pragma: NO COVER
    """
    Handy helper for getting resources from our kit.
    """
    data = files(__name__).joinpath(path).read_bytes()
    return data.decode("utf8")
  • Replace load_resource():
def load_resource(resource_path):  # pragma: NO COVER
    """
    Gets the content of a resource
    """
    resource_content = files(__name__).joinpath(resource_path).read_bytes()
    return str(resource_content.decode("utf8"))

Also check for any other test failures not covered above and fix those as well.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: Please find a solution for this failing job. Use the logs, job definition, and any referenced files where the failure occurred. Keep your response focused on the solution and include code suggestions when appropriate.

Assistant: Let me fetch the job logs to investigate the failure.

Assistant: There are two distinct root causes for the 27 test failures. Here's the breakdown and fixes:


Root Cause 1: ModuleNotFoundError: No module named 'pkg_resources'

pkg_resources was removed from Python 3.12's standard library. It's provided by setuptools, but setuptools is no longer bundled by default in newer Python environments.

Affected tests: test_ctor, test_max_score, test_save_sga, test_upload_allowed, test_clear_student_state, etc.

The issue is in edx_sga/sga.py line 17:

import pkg_resources

...and its usages at lines 1042–1051.

Fix A — Replace pkg_resources with importlib.resources (recommended)

# sga.py — remove: import pkg_resources
from importlib.resources import files

def _resource(path):  # pragma: NO COVER
    """
    Handy helper for getting resources from our kit.
    """
    data = files(__name__).joinpath(path).read_bytes()
    return data.decode("utf8")


def load_resource(resource_path):  # pragma: NO COVER
    """
    Gets the content of a resource
    """
    resource_content = files(__name__).joinpath(resource_path).read_bytes()
    return str(resource_content.decode("utf8"))

Fix B — Alternatively, pin setuptools as a dependency

Add it to setup.py's install_requires (or requirements.txt):

install_requires=[
    "setuptools",
    "XBlock",
    "xblock-utils",
    "web_fragments",
],

Fix A is preferred since pkg_resources is deprecated for resource loading.


Root Cause 2: AttributeError: module 'edx_sga' has no attribute 'sga'

The test's fake_import in setUp intercepts ImportErrors but doesn't handle the case where edx_sga.sga fails to import due to the pkg_resources error — leaving the edx_sga module in a partially-initialized state. The subsequent from edx_sga.sga import StaffGradedAssignmentXBlock in make_xblock then finds edx_sga without a sga attribute.

Affected tests: test_download_annotated, test_student_view, test_staff_download, etc.

This is a cascading failure from Root Cause 1. Fixing pkg_resources (above) will resolve these as well, since edx_sga/sga.py will then import cleanly.


Summary

Fix What it resolves
Replace pkg_resources with importlib.resources in sga.py Fixes ModuleNotFoundError on Python 3.12
(cascading) edx_sga.sga imports cleanly Fixes all AttributeError: module 'edx_sga' has no attribute 'sga' failures

Applying Fix A alone should resolve all 27 failing tests.

User: Create a PR with the Fix A

Assistant: We're waiting for your response to the tool call confirmation dialog to continue.

User: @copilot A...

This pull request was created from Copilot chat.

Copilot AI changed the title [WIP] Fix failing GitHub Actions tests in mitodl/edx-sga Fix Python 3.12 compatibility: replace pkg_resources with importlib.resources Apr 8, 2026
Copilot AI requested a review from asadali145 April 8, 2026 13:31
Copilot AI and others added 4 commits April 9, 2026 16:40
@asadali145 asadali145 force-pushed the copilot/fix-github-actions-tests branch from db5b182 to a3639b8 Compare April 9, 2026 11:40
@asadali145 asadali145 changed the title Fix Python 3.12 compatibility: replace pkg_resources with importlib.resources fix: python 3.12 compatibility Apr 9, 2026
@asadali145 asadali145 marked this pull request as ready for review April 9, 2026 18:03
@asadali145 asadali145 requested review from Copilot April 9, 2026 18:05
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates edx_sga to run cleanly on Python 3.12 by removing reliance on pkg_resources and aligning tooling/config with newer Python/Django environments.

Changes:

  • Replace pkg_resources resource loading in edx_sga/sga.py with stdlib importlib.resources.
  • Upgrade edx-lint and regenerate pylintrc for compatibility.
  • Update test and integration-test scaffolding for newer Django storage configuration and edx-platform config layout.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
edx_sga/sga.py Switch resource loading from pkg_resources to importlib.resources.
test_requirements.txt Bump edx-lint to avoid pkg_resources dependency in lint tooling.
pylintrc Update generated header/hash for edx-lint 6.0.0.
edx_sga/test_settings.py Move from DEFAULT_FILE_STORAGE to Django 4.2+ STORAGES setting.
edx_sga/tests/common.py Update temp storage test mixin to use STORAGES.
edx_sga/tests/integration_tests.py Create a non-staff user with an associated UserProfile for runtime prep.
run_edx_integration_tests.sh Copy setup.cfg or fall back to pyproject.toml from edx-platform for integration runs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

edx_sga/sga.py Outdated
Comment on lines 1042 to 1043
data = files(__name__).joinpath(path).read_bytes()
return data.decode("utf8")
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

importlib.resources.files() expects a package (or package name). In this module, __name__ is edx_sga.sga (not a package), so files(__name__) can raise “is not a package” at runtime. Use files(__package__) or files('edx_sga') (or pass the actual edx_sga package module) so resources resolve correctly.

Copilot uses AI. Check for mistakes.
edx_sga/sga.py Outdated
Comment on lines 1050 to 1051
resource_content = files(__name__).joinpath(resource_path).read_bytes()
return str(resource_content.decode("utf8"))
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as _resource(): files(__name__) is using the module name (edx_sga.sga) rather than the package, which can break resource loading. Switch to files(__package__) / files('edx_sga') for consistent behavior across environments (including zipped distributions).

Copilot uses AI. Check for mistakes.
Comment on lines +21 to +28
# Copy setup.cfg if it exists, otherwise try pyproject.toml
if [ -f /openedx/edx-platform/setup.cfg ]; then
cp /openedx/edx-platform/setup.cfg .
elif [ -f /openedx/edx-platform/pyproject.toml ]; then
cp /openedx/edx-platform/pyproject.toml .
else
echo "Neither setup.cfg nor pyproject.toml found, skipping."
fi
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description lists changes to edx_sga/sga.py, test_requirements.txt, and pylintrc, but this PR also updates integration test script + multiple test/settings files (e.g., run_edx_integration_tests.sh, edx_sga/test_settings.py, edx_sga/tests/common.py, edx_sga/tests/integration_tests.py). Please update the description to reflect these additional behavioral changes so reviewers know what to focus on.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply changes based on this feedback

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description has been updated to document all changed files:

  • edx_sga/test_settings.py: migrated DEFAULT_FILE_STORAGESTORAGES dict (Django 4.2+ deprecation)
  • edx_sga/tests/common.py: updated test setup/teardown to save and restore STORAGES["default"] instead of the removed DEFAULT_FILE_STORAGE setting
  • edx_sga/tests/integration_tests.py: fixed user creation in staff/non-staff test helper to create a proper User+UserProfile pair
  • run_edx_integration_tests.sh: added fallback to copy pyproject.toml when setup.cfg is absent from edx-platform

@asadali145
Copy link
Copy Markdown

@arslanashraf7 This should be ready for a quick look. I have incorporated the AI review and also tested it.

@arslanashraf7 arslanashraf7 self-assigned this Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants