Skip to content

Enhancement: Support offline installation #31

@vectro

Description

@vectro

Summary

The current shfmt-py package requires an active internet connection during installation. The setup.py script is designed to download the appropriate pre-compiled shfmt binary from the official GitHub releases for the target platform at the moment a user runs pip install.
This behavior makes it impossible to install shfmt-py in secure, airgapped, or firewalled environments. It also introduces an install-time dependency on the availability of GitHub, which can lead to non-reproducible or failed builds in CI/CD pipelines.

Expected Behavior

The shfmt-py package should be installable from a local artifact (a source distribution or a wheel) without requiring any network access during the installation process itself.

Proposed Solution

The packaging process should be updated to separate the fetching of binaries from the installation logic. I think there are many different ways to accomplish this but here are a couple that I see:

Option 1: Bundle Binaries in Distribution Artifacts

One way to achieve this could be to adjust the packaging strategy to bundle the shfmt binaries within the distribution files uploaded to PyPI. Modern packaging workflows often use tools like cibuildwheel in a CI/CD pipeline to automate this.

A possible implementation might involve these concepts:

  • A Pre-Build Step in CI: A custom command could be created (e.g., in setup.py) that downloads all necessary shfmt binaries. This command would only be run by the CI/CD pipeline at release time, inside a temporary environment. The binaries would not be committed to the Git repository.

  • Automated Wheel Building: After the pre-build step, a tool like cibuildwheel could then build the platform-specific wheels. It would be configured to find the correct, pre-downloaded binary and bundle it. A benefit of this tool is that it also handles complexities like creating manylinux wheels for broad Linux compatibility.

  • Automated Publishing: Finally, the CI pipeline could publish the newly created, self-contained wheels (and sdist) directly to PyPI.

This approach would automate the release process while keeping the Git repository itself free of binary files.

Option 2: Platform-Specific Packages

Create separate PyPI packages for each platform (e.g., shfmt-py-linux-x64, shfmt-py-darwin-arm64). The main shfmt-py package would become a meta-package that depends on the appropriate platform-specific package. This approach keeps individual package sizes small but increases maintenance complexity.

End-User Impact

When a user runs pip install shfmt-py:

  • pip will download the most appropriate file from PyPI.
  • If a pre-built wheel exists for their platform, it will be used. The shfmt binary is already inside, and the installation will complete offline.
  • If they build from source, pip will use the sdist. Since all binaries are included in the sdist, the installation can proceed without any further network calls.

Benefits of this Approach

  • Enables Airgapped Installation: Unlocks usage in secure and offline environments.
  • Improves Build Reliability: Removes the external network dependency at install time, making user builds deterministic.
  • Follows Best Practices: Aligns with the standard methodology for distributing Python wrappers around third-party executables.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions