Skip to content

Use ppbt toolchain when building. #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
99d4788
Use ppbt toolchain when building.
dwoz Sep 20, 2024
88d89d3
Add ppbt to build dependencies
dwoz Sep 20, 2024
f7cf802
Fix ppbt import and extract
dwoz Sep 20, 2024
c2c9178
Fix unit tests
dwoz Sep 20, 2024
59ccb9e
Add no-pretty argument for building
dwoz Sep 20, 2024
e5ac585
Add ppbt to test and docs deps
dwoz Sep 20, 2024
e7d9c56
Add ppbt to test requirements
dwoz Sep 20, 2024
aa000fb
Only require ppbt on linux
dwoz Sep 20, 2024
d10879d
ppbt is in requirements file
dwoz Sep 22, 2024
d0f17c8
Fix unit test on non linux platforms
dwoz Oct 21, 2024
f28bd3b
fix tests/test_verify_build.py::test_pip_install_salt_git
dwoz Oct 21, 2024
e59a615
Fix command order
dwoz Feb 28, 2025
8d18afe
Move cargo home directory to relenv data dir
dwoz Feb 28, 2025
1a4826d
Make ppbt optional
dwoz Mar 1, 2025
5b64e06
Skip ppbt when not on linux
dwoz Mar 1, 2025
ff385c3
Do not raise missing ppbt unless on linux
dwoz Mar 1, 2025
2b84c5f
Clean up and simplify
dwoz Mar 1, 2025
9bdc3ff
Wrap ppbt import in exception handler
dwoz Mar 1, 2025
5a6e6b2
Fix function call
dwoz Mar 1, 2025
0f5d226
Log detected version
dwoz Mar 1, 2025
163c9ca
Fix build unit test
dwoz Mar 1, 2025
544d411
Cleaner syntax
dwoz Mar 1, 2025
0a948a8
run ppbt extract to test why cryptography failes
dwoz Mar 1, 2025
2e59570
Do not hide errors
dwoz Mar 2, 2025
4bce4ef
Toolchains only exist on linux
dwoz Mar 2, 2025
eb26c02
Fix pre-commit wart
dwoz Mar 2, 2025
e5ac18a
Only linux has toolchains
dwoz Mar 2, 2025
47d06ab
Install ppbt for fips tests
dwoz Mar 3, 2025
b2b4262
Add python3-devel to system dependencies for fips tests
dwoz Mar 3, 2025
be7cd60
Add glibc-devel to fips system deps
dwoz Mar 3, 2025
b937bb6
debug pip
dwoz Mar 4, 2025
c0be225
fix order
dwoz Mar 4, 2025
c2b852d
Fix fips test failures
dwoz Mar 6, 2025
4be59ac
Add pyversions
dwoz May 9, 2025
ae27829
Detect python versions
dwoz May 10, 2025
61e8760
Add our own version class
dwoz May 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-cross-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
sudo apt-get update
sudo apt-get install -y build-essential bison python3-all patchelf swig python3-virtualenv
virtualenv venv
venv/bin/pip3 install nox
env/bin/python3 -m pip install nox ppbt

- name: Python Version
run: |
Expand Down
21 changes: 5 additions & 16 deletions .github/workflows/build-native-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,27 +50,16 @@ jobs:
sudo apt-get update
sudo apt-get install -y build-essential bison python3-all patchelf swig cmake libldap2-dev libsasl2-dev ldap-utils libssl-dev pkg-config libvirt-dev default-libmysqlclient-dev python3-virtualenv
virtualenv venv
venv/bin/pip3 install nox
venv/bin/python3 -m pip install nox ppbt

- name: Python Version
run: |
venv/bin/python3 --version
venv/bin/python3 -c 'import os; print(os.name)'

- name: Download Toolchain Artifact
uses: actions/download-artifact@v4
with:
name: toolchain-${{ matrix.host }}-${{ matrix.target }}-linux-gnu.tar.xz
path: .

- name: Extract Toolchain Artifact
run: |
mkdir -p toolchain
tar -C toolchain -xvf toolchain-${{ matrix.host }}-${{ matrix.target }}-linux-gnu.tar.xz

- name: Build
run: |
venv/bin/python3 -m relenv build --arch=${{ matrix.target }} --python=${{ matrix.version }}
venv/bin/python3 -m relenv build --no-pretty --arch=${{ matrix.target }} --python=${{ matrix.version }}

- name: Verify Build
if: ${{ matrix.host == matrix.target }}
Expand All @@ -87,7 +76,7 @@ jobs:

- name: "Upload artifact: build/${{ matrix.version }}-${{ matrix.target }}-linux-gnu.tar.xz"
uses: actions/upload-artifact@v4
if: ${{ matrix.host == matrix.target && always()}}
if: ${{ matrix.host == matrix.target && always() }}
with:
name: ${{ matrix.version }}-${{ matrix.target }}-linux-gnu.tar.xz
path: build/${{ matrix.version }}-${{ matrix.target }}-linux-gnu.tar.xz
Expand Down Expand Up @@ -130,7 +119,7 @@ jobs:

- name: Build
run: |
python3 -m relenv build --python=${{ matrix.version }}
python3 -m relenv build --no-pretty --python=${{ matrix.version }}

- name: Verify Build
run: |
Expand Down Expand Up @@ -188,7 +177,7 @@ jobs:

- name: Build
run: |
python3 -m relenv build --python=${{ matrix.version }}
python3 -m relenv build --no-pretty --python=${{ matrix.version }}

- name: Verify Build
run: |
Expand Down
42 changes: 15 additions & 27 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,6 @@ jobs:
deleted:
- deleted:
- '**'
toolchain:
- added|modified:
- '.github/workflows/*toolchain*.yml'
- 'relenv/toolchain.py'
- 'relenv/_toolchain/**'

pre-commit:
name: Pre-Commit
uses: ./.github/workflows/pre-commit-action.yml
Expand All @@ -79,20 +73,11 @@ jobs:
kind: "${{ inputs.kind }}"
cmd: "${{ inputs.package_command }}"

toolchain:
name: Toolchain Builds
uses: ./.github/workflows/toolchain-action.yml
needs:
- get-changed-files
with:
changed-files: ${{ needs.get-changed-files.outputs.changed-files }}

build-native:
name: Python Native Builds
uses: ./.github/workflows/build-native-action.yml
needs:
- get-changed-files
- toolchain
with:
changed-files: ${{ needs.get-changed-files.outputs.changed-files }}

Expand All @@ -103,17 +88,6 @@ jobs:
- get-changed-files
uses: ./.github/workflows/test-fips-action.yml

#build-cross:
# name: Python Cross Builds
# uses: ./.github/workflows/build-action.yml
# needs:
# - get-changed-files
# - toolchain
# - build-native
# with:
# changed-files: ${{ needs.get-changed-files.outputs.changed-files }}
# kind: "cross"

test:
name: Test
needs:
Expand Down Expand Up @@ -142,6 +116,18 @@ jobs:
# - test-fips
# secrets:
# PYPI_API_TOKEN: "${{ secrets.PYPI_API_TOKEN }}"
#
upload-build-artifacts:
name: Deploy Prebuilt Python Builds
if: ${{ inputs.kind == 'release' && success() }}
uses: ./.github/workflows/deploy-build-action.yml
needs:
- build-python-package
- build-native
- test-fips
#- build-cross
with:
version: "${{ needs.build-python-package.outputs.version }}"

deploy-python-package:
name: "Deploy python package"
Expand All @@ -153,6 +139,7 @@ jobs:
- docs
- build-python-package
- test-fips
- upload-build-artifacts
steps:
- uses: actions/create-github-app-token@v1
id: app-token
Expand Down Expand Up @@ -185,14 +172,15 @@ jobs:
if: always()
needs:
- pre-commit
- toolchain
- build-native
#- build-cross
- test
- docs
- upload-build-artifacts
- deploy-python-package
- create-release
steps:

- name: Get workflow information
id: get-workflow-info
uses: technote-space/workflow-conclusion-action@v3
Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/test-fips-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:

- name: Install System Dependencies
run: |
yum install -y openssl-fips-provider python3 python3-virtualenv git gcc
yum install -y openssl-fips-provider python3 python3-devel python3-virtualenv git gcc binutils glibc-devel linux-api-headers libffi-devel

- uses: actions/checkout@v3

Expand All @@ -41,6 +41,12 @@ jobs:
run: |
virtualenv venv

- name: Pip debug
run: |
./venv/bin/python3 -m pip install --upgrade pip
./venv/bin/python3 -c 'import platform; print(platform.libc_ver())'
./venv/bin/pip3 debug --verbose

- name: Install Virtual Environment Dependencies
run: |
./venv/bin/pip3 install -r requirements/tests.txt
Expand Down
6 changes: 0 additions & 6 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,6 @@ def build(session, arch):
invoke_relenv(session, "build", f"--arch={arch}")


@nox.session
@nox.parametrize("arch", ("x86_64", "aarch64"))
def toolchain(session, arch):
invoke_relenv(session, "toolchain", "build", f"--arch={arch}")


# Convenience sessions
@nox.session
def build_x86_64(session):
Expand Down
8 changes: 4 additions & 4 deletions relenv/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

from argparse import ArgumentParser

from . import build, buildenv, check, create, fetch, toolchain
from . import build, buildenv, check, create, fetch, pyversions, toolchain
from .common import __version__


Expand Down Expand Up @@ -34,6 +34,7 @@ def setup_cli():
fetch,
check,
buildenv,
pyversions,
]
for mod in modules_to_setup:
mod.setup_parser(subparsers)
Expand All @@ -47,10 +48,9 @@ def main():
"""
parser = setup_cli()
args = parser.parse_args()
# args.func(args)
try:
if hasattr(args, "func"):
args.func(args)
except AttributeError:
else:
parser.print_help()
parser.exit(1, "\nNo subcommand given...\n\n")

Expand Down
Loading