Skip to content

Commit 02d8828

Browse files
dguidodguido
committed
test: Migrate from bash scripts to pytest test suite
- Replace bash test scripts with comprehensive pytest test suite - Add test fixtures for proper isolation and cleanup - Update CI workflows to use pytest instead of bash scripts - Consolidate all dev dependencies in pyproject.toml - Remove dangerous rm -rf patterns in favor of safe deletion methods
1 parent 2e989ca commit 02d8828

File tree

11 files changed

+892
-45
lines changed

11 files changed

+892
-45
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 29 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,12 @@ concurrency:
1616

1717
jobs:
1818
tests:
19-
name: CI (Python ${{ matrix.python }} on ${{ matrix.os }}, ${{ matrix.type }} test)
19+
name: CI (Python ${{ matrix.python }} on ${{ matrix.os }})
2020
runs-on: ${{ matrix.os }}
2121
strategy:
2222
matrix:
2323
python: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
2424
os: ["ubuntu-latest", "macos-latest", "windows-2022"]
25-
type: ["solc", "solc_upgrade", "os_specific"]
2625
steps:
2726
- uses: actions/checkout@v4
2827
- uses: actions/setup-python@v5
@@ -32,17 +31,39 @@ jobs:
3231
- name: Create Python virtual environment
3332
run: |
3433
${{ steps.python.outputs.python-path }} -m venv test-venv
35-
- name: Install solc-select
34+
- name: Install solc-select and test dependencies
3635
shell: bash
3736
run: |
3837
source test-venv/${{ (runner.os == 'Windows' && 'Scripts') || 'bin' }}/activate
3938
python -m pip install --upgrade pip
40-
pip3 install .
39+
pip3 install -e ".[dev]"
4140
- name: Run Tests
4241
shell: bash
43-
env:
44-
TEST_TYPE: ${{ (matrix.type != 'os_specific' && matrix.type) || runner.os }}
4542
run: |
4643
source test-venv/${{ (runner.os == 'Windows' && 'Scripts') || 'bin' }}/activate
47-
TEST_TYPE="$(echo "$TEST_TYPE" | tr '[:upper:]' '[:lower:]')"
48-
bash scripts/test_${TEST_TYPE}.sh
44+
# Run pytest with appropriate platform markers
45+
# Skip slow tests (upgrade test) and version_boundaries tests in CI
46+
python -m pytest tests/ -v --tb=short -k "not version_boundaries" -m "not slow"
47+
48+
comprehensive-tests:
49+
name: Comprehensive Tests (Python ${{ matrix.python }} on ${{ matrix.os }})
50+
# Only run on schedule or when labeled 'full-tests'
51+
if: github.event_name == 'schedule' || contains(github.event.pull_request.labels.*.name, 'full-tests')
52+
runs-on: ${{ matrix.os }}
53+
strategy:
54+
matrix:
55+
python: ["3.11"] # Run comprehensive tests on one Python version
56+
os: ["ubuntu-latest", "macos-latest", "windows-2022"]
57+
steps:
58+
- uses: actions/checkout@v4
59+
- uses: actions/setup-python@v5
60+
with:
61+
python-version: ${{ matrix.python }}
62+
- name: Install solc-select and test dependencies
63+
run: |
64+
python -m pip install --upgrade pip
65+
pip3 install -e ".[dev]"
66+
- name: Run Comprehensive Tests
67+
run: |
68+
# Run ALL tests including slow and version_boundaries
69+
python -m pytest tests/ -v --tb=short

‎.github/workflows/lint.yml‎

Lines changed: 2 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -73,46 +73,11 @@ jobs:
7373
with:
7474
python-version: 3.8
7575

76-
- name: Install ruff
77-
run: pip install ruff
76+
- name: Install development dependencies
77+
run: pip install -e ".[dev]"
7878

7979
- name: Run ruff check
8080
run: ruff check .
8181

8282
- name: Run ruff format check
8383
run: ruff format --check .
84-
85-
black:
86-
name: Lint Code Base (black)
87-
runs-on: ubuntu-latest
88-
89-
steps:
90-
- name: Checkout Code
91-
uses: actions/checkout@v4
92-
with:
93-
# super-linter needs the full git history to get the
94-
# list of files that changed across commits
95-
fetch-depth: 0
96-
97-
- name: Set up Python 3.8
98-
uses: actions/setup-python@v5
99-
with:
100-
python-version: 3.8
101-
102-
- name: Install dependencies
103-
run: |
104-
mkdir -p .github/linters
105-
cp pyproject.toml .github/linters
106-
107-
- name: Black
108-
uses: super-linter/super-linter/[email protected]
109-
if: always()
110-
env:
111-
# run linter on everything to catch preexisting problems
112-
VALIDATE_ALL_CODEBASE: true
113-
DEFAULT_BRANCH: master
114-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
115-
# Run only black
116-
VALIDATE_PYTHON_BLACK: true
117-
PYTHON_BLACK_CONFIG_FILE: pyproject.toml
118-
FILTER_REGEX_EXCLUDE: .*tests/.*.(json|zip|sol)

‎ISSUE_DRAFT.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Add Linux ARM64/aarch64 Architecture Support
2+
3+
## Summary
4+
5+
solc-select currently lacks support for Linux ARM64/aarch64 architecture, causing installation and usage failures on ARM-based Linux systems (including Docker containers running on Apple Silicon Macs, AWS Graviton instances, Raspberry Pi, and other ARM64 Linux environments).
6+
7+
## Current Behavior
8+
9+
When running solc-select on a Linux ARM64 system:
10+
1. The `soliditylang_platform()` function incorrectly identifies the system as `linux-amd64` regardless of actual architecture
11+
2. solc-select attempts to download x86_64 binaries from `https://binaries.soliditylang.org/linux-amd64/`
12+
3. The downloaded x86_64 binaries fail to execute on ARM64 systems with "Exec format error"
13+
14+
## Root Cause Analysis
15+
16+
### Architecture Detection Issue
17+
The `soliditylang_platform()` function in `solc_select/solc_select.py:275-284` only checks the operating system, not the architecture:
18+
19+
```python
20+
def soliditylang_platform() -> str:
21+
if sys.platform.startswith("linux"):
22+
platform = LINUX_AMD64 # Always returns AMD64 for any Linux
23+
elif sys.platform == "darwin":
24+
platform = MACOSX_AMD64
25+
elif sys.platform in ["win32", "cygwin"]:
26+
platform = WINDOWS_AMD64
27+
else:
28+
raise argparse.ArgumentTypeError("Unsupported platform")
29+
return platform
30+
```
31+
32+
### Binary Availability Challenge
33+
The Solidity team does not provide official ARM64 Linux binaries:
34+
- Available platforms at binaries.soliditylang.org:
35+
- ✅ linux-amd64
36+
- ✅ macosx-amd64 (Universal binaries since v0.8.24)
37+
- ✅ windows-amd64
38+
- ✅ emscripten-wasm32
39+
- ✅ emscripten-asmjs
40+
- ❌ linux-arm64 (does not exist)
41+
- ❌ linux-aarch64 (does not exist)
42+
43+
## Comparison with macOS Approach
44+
45+
solc-select already handles architecture differences on macOS:
46+
- Detects Apple Silicon vs Intel Macs
47+
- Checks for Rosetta 2 availability for running Intel binaries on ARM
48+
- Identifies Universal binaries (v0.8.24+) that run natively on both architectures
49+
- See `solc_select/utils.py:10-30` for implementation
50+
51+
## Proposed Solution
52+
53+
### Phase 1: Immediate Improvements (Error Handling)
54+
1. **Detect actual architecture** on Linux systems using `platform.machine()`
55+
2. **Provide clear error messages** when running on unsupported architectures
56+
3. **Document workarounds** for ARM64 Linux users
57+
58+
### Phase 2: ARM64 Support Implementation
59+
60+
#### Option A: Use solc-js/WASM binaries (Recommended)
61+
- Leverage the existing emscripten-wasm32 binaries which work on any architecture
62+
- Add a new platform type `LINUX_ARM64_WASM` that uses WASM binaries
63+
- Implement a wrapper to make WASM binaries behave like native ones
64+
- Benefits: Works immediately, maintained by Solidity team, cross-architecture
65+
- Drawbacks: Slightly slower performance, requires Node.js
66+
67+
#### Option B: Build native ARM64 binaries
68+
- Set up CI/CD to compile Solidity for linux-arm64
69+
- Host binaries in Crytic's supplemental repository
70+
- Similar to how older Linux versions (0.4.0-0.4.10) are handled
71+
- Benefits: Native performance
72+
- Drawbacks: Maintenance burden, build infrastructure needed
73+
74+
#### Option C: Use QEMU user-mode emulation
75+
- Document how to set up QEMU with binfmt_misc for transparent x86_64 emulation
76+
- Similar to Rosetta 2 on macOS but for Linux
77+
- Benefits: Works with existing binaries
78+
- Drawbacks: Performance overhead, setup complexity
79+
80+
## Recommended Implementation Plan
81+
82+
1. **Update architecture detection**:
83+
```python
84+
def soliditylang_platform() -> str:
85+
if sys.platform.startswith("linux"):
86+
machine = platform.machine()
87+
if machine in ["x86_64", "AMD64"]:
88+
platform = LINUX_AMD64
89+
elif machine in ["aarch64", "arm64"]:
90+
platform = LINUX_ARM64 # New constant
91+
else:
92+
raise argparse.ArgumentTypeError(f"Unsupported Linux architecture: {machine}")
93+
# ... rest of function
94+
```
95+
96+
2. **Add WASM-based fallback for ARM64**:
97+
- Download emscripten-wasm32 binaries for ARM64 Linux
98+
- Create a Node.js wrapper script that executes the WASM binary
99+
- Install wrapper as the `solc` executable
100+
101+
3. **Add compatibility checks**:
102+
- Check for Node.js availability when on ARM64
103+
- Provide helpful error messages with installation instructions
104+
105+
## Impact
106+
107+
This issue affects:
108+
- Docker containers on Apple Silicon Macs (very common in development)
109+
- AWS Graviton instances (increasingly popular for cost savings)
110+
- Raspberry Pi and other ARM SBCs
111+
- Any CI/CD running on ARM64 infrastructure
112+
- Projects using Manticore, Slither, or other tools depending on solc-select
113+
114+
## Testing
115+
116+
The solution should be tested on:
117+
- [ ] x86_64 Linux (regression testing)
118+
- [ ] ARM64 Linux (Docker on Apple Silicon)
119+
- [ ] ARM64 Linux (native, e.g., Raspberry Pi OS)
120+
- [ ] Various Solidity versions (especially older ones like 0.4.x)
121+
122+
## Related Issues
123+
124+
- Similar architecture detection is needed for Windows ARM64 (emerging platform)
125+
- The approach could be extended to support other architectures (RISC-V, etc.)
126+
127+
## References
128+
129+
- [Solidity Official Binaries Repository](https://binaries.soliditylang.org/)
130+
- [macOS Universal Binary Support (v0.8.24+)](https://github.com/ethereum/solidity/issues/12291#issuecomment-2223328961)
131+
- [solc-js npm package](https://www.npmjs.com/package/solc) - JavaScript/WASM version

‎pyproject.toml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,19 @@ Homepage = "https://github.com/crytic/solc-select"
4040
Repository = "https://github.com/crytic/solc-select.git"
4141
Issues = "https://github.com/crytic/solc-select/issues"
4242

43+
[project.optional-dependencies]
44+
dev = [
45+
# Testing
46+
"pytest>=7.0.0",
47+
"pytest-cov>=4.0.0",
48+
"pytest-xdist>=3.0.0",
49+
# Linting and type checking
50+
"ruff>=0.1.0",
51+
"pylint>=2.15.0",
52+
"mypy>=1.0.0",
53+
"types-setuptools",
54+
]
55+
4356
[project.scripts]
4457
solc-select = "solc_select.__main__:solc_select"
4558
solc = "solc_select.__main__:solc"
@@ -88,3 +101,20 @@ duplicate-code,
88101
import-error,
89102
unsubscriptable-object
90103
"""
104+
105+
[tool.pytest.ini_options]
106+
testpaths = ["tests"]
107+
python_files = "test_*.py"
108+
python_classes = "Test*"
109+
python_functions = "test_*"
110+
addopts = [
111+
"-v",
112+
"--strict-markers",
113+
"--tb=short",
114+
]
115+
markers = [
116+
"slow: marks tests as slow (deselect with '-m \"not slow\"')",
117+
"linux: marks tests to run only on Linux",
118+
"macos: marks tests to run only on macOS",
119+
"windows: marks tests to run only on Windows",
120+
]

‎run_tests.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Simple test runner for solc-select.
4+
5+
This script provides an easy way to run the pytest test suite
6+
with appropriate options.
7+
"""
8+
9+
import subprocess
10+
import sys
11+
12+
13+
def main():
14+
"""Run the test suite."""
15+
# Basic pytest command
16+
cmd = [sys.executable, "-m", "pytest", "tests/", "-v"]
17+
18+
# Add platform-specific marker based on current platform
19+
if sys.platform == "linux":
20+
# Run all non-Windows, non-macOS tests plus Linux tests
21+
cmd.extend(["-m", "not (windows or macos)"])
22+
elif sys.platform == "darwin":
23+
# Run all non-Windows, non-Linux tests plus macOS tests
24+
cmd.extend(["-m", "not (windows or linux)"])
25+
elif sys.platform == "win32":
26+
# Run all non-Linux, non-macOS tests plus Windows tests
27+
cmd.extend(["-m", "not (linux or macos)"])
28+
29+
print(f"Running: {' '.join(cmd)}")
30+
result = subprocess.run(cmd)
31+
return result.returncode
32+
33+
34+
if __name__ == "__main__":
35+
sys.exit(main())

0 commit comments

Comments
 (0)