Skip to content

Commit c3696b0

Browse files
committed
Merge development → 8eb50cf: core, ci, config (+1 more)
📚 Documentation: * docs: update ARM32 piwheels stats for v3.3.5 [skip ci] (ec94365) 📝 Other changes: * Update 1 code files; Update configuration (64dfe73) Modified: • src/omnipkg/core.py (+2/-1 lines) • pyproject.toml (+1/-1 lines) * chore: exclude licenses directory from Dependabot scans (84b20ed) * chore: upgrade dependencies to resolve high-severity CVEs (4c6f821) 📊 7 files changed, 161 insertions(+), 57 deletions(-) 📁 Detailed file changes: Source code: • src/omnipkg/core.py: +2 -1 Documentation: • CHANGELOG.md: +97 • README.md: +1 Configuration: • .github/dependabot.yml: +2 • pyproject.toml: +1 -1 Other: • requirements-trace.txt: +31 -29 • requirements.txt: +27 -26 Commits: 8eb50cf..4e273a3
2 parents 8eb50cf + 4e273a3 commit c3696b0

7 files changed

Lines changed: 161 additions & 57 deletions

File tree

.github/dependabot.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,5 @@ updates:
1212
interval: "monthly"
1313
ignore:
1414
- dependency-name: "*"
15+
exclude-paths:
16+
- "licenses/**"

CHANGELOG.md

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,103 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
---
99

10+
## [3.4.0] — 2026-06-07
11+
12+
The Modular Installation & IPC Engine Update
13+
14+
Welcome to **omnipkg v3.4.0**, a massive architectural milestone! This release brings a completely overhauled installation pipeline, microsecond-level C dispatcher optimizations, true zero-copy GPU IPC fallbacks, and deep ELF-level NumPy ABI parsing.
15+
16+
With over 35 commits and 14,000 lines of code changed, omnipkg is faster, safer, and smarter than ever.
17+
18+
## ✨ Major Architectural Features
19+
20+
We have decoupled the monolithic `_run_pip_install` into a highly optimized `ModularInstaller`.
21+
* **Staircase Dispatch:** Installations now cascade intelligently through `FFI``Daemon IPC``subprocess uv``pip fallback`, maximizing speed while ensuring bulletproof reliability across all environments.
22+
* **Atomic Stash-Swaps (`stable-main`):** Rebuilding bubbles from scratch is a thing of the past. Omnipkg now temporarily stashes the active package, allows `uv` to install the new version directly into the main environment, moves the fresh install into a bubble, and atomically restores the stash.
23+
24+
* **Instant Help:** `8pkg --help` and `8pkg --version` are now intercepted directly by the C dispatcher using `uint64_t` word-load comparisons, rendering in **< 300 microseconds** (0ms startup time).
25+
* **Smart i18n Routing:** Implemented a lightweight `lang_marker_en` file and `OMNIPKG_LANG` environment variable cache. English users hit the ultra-fast C rendering path, while non-English locales seamlessly fall back to fully translated Python `argparse` menus.
26+
* **Persistent Config:** Changing your language via `8pkg config set language` now automatically updates your `.bashrc` / `.zshrc` to ensure your language choice persists across terminal sessions.
27+
28+
* Omnipkg's package metadata builder now natively parses `.so` ELF sections to extract the baked-in `NPY_FEATURE_VERSION` C-API requirement from compiled extensions (like PyTorch and TensorFlow).
29+
* This intelligence is saved to the Knowledge Base (`numpy_abi_version` and `numpy_abi_range`), allowing the daemon to flawlessly select the correct NumPy bubble at runtime and ensuring Content-Addressable Storage (CAS) safely segregates incompatible tensor binaries.
30+
31+
* **Turing/CUDA 13 Fallback:** Added the `TORCH_MP_QUEUE` IPC fallback for environments where raw `cudaIpcOpenMemHandle` is broken (e.g., Turing GPUs on driver 610). Tensors stay on the GPU using `share_memory_()`, keeping execution blazingly fast.
32+
* **TUI Resource Monitor:** `8pkg daemon monitor` now features an interactive TUI to track live RAM/VRAM usage.
33+
* **Configurable Idle Policies:** Use `8pkg daemon idle` to persistently configure exactly how many warm Python workers stay alive in the background, balancing instant execution with idle RAM footprint.
34+
35+
## 🛡️ Security & Dependency Updates
36+
A massive sweep of dependency bumps to resolve several GitHub Dependabot CVE alerts:
37+
* **Bumped `urllib3` to `2.7.0`:** Resolves High-severity decompression bomb (CVE-2026-44432) and sensitive header leaks.
38+
* **Bumped `uv` to `0.11.19`:** Patches Arbitrary File Write/Deletion via entrypoint names and RECORD files.
39+
* **Bumped `idna` to `3.18`:** Resolves Domain Name encoding bypasses.
40+
* **Bumped `aiohttp` to `3.14.0`:** Fixes cross-origin redirect and untrusted deserialization vulnerabilities.
41+
* **Bumped `authlib` to `1.7.2`:** Fixes OIDC open redirects and CSRF-via-cache vulnerabilities.
42+
* **Dependabot config:** Excluded the `licenses/` directory to prevent false-positive syntax crashes during manifest parsing.
43+
44+
## 🐛 Bug Fixes & Quality of Life
45+
* **Ghost Repair Targeting:** The self-healer now targets the main environment directly using a pinned KB snapshot, preventing "ghost" metadata from tricking the system into bubbling a repaired package while leaving the main environment broken.
46+
* **Translation Shadowing:** Replaced all `_` tuple unpacking variables with `unused` globally to prevent shadowing of the `gettext` `_()` translation function.
47+
* **Unicode / Terminal Safety:** Standardized `safe_print` across the entire codebase (Dispatcher, Core, and Tests) to prevent `UnicodeEncodeError` crashes on Windows and strict ASCII terminals.
48+
* **Windows Compatibility:** Guarded POSIX `fcntl` usage behind platform checks, leveraging `msvcrt.locking` on Windows for safe cross-process file locks.
49+
50+
## 🧪 Testing & Diagnostics
51+
* **KBSyncVerifier:** Refactored `test_verify_bubble_deps.py` into a robust, object-oriented symmetry checker that bi-directionally validates the SQLite Knowledge Base against physical disk states.
52+
* **Daemon IPC Showcase:** Added an interactive "Daemon IPC Showcase" to the `8pkg demo` menu (Option 11) to demonstrate zero-copy GPU tensor sharing across Python boundaries.
53+
* Consolidated FFI diagnostic and contract tests into the main `tests/` directory and added the `integration` marker to `pytest.ini`.
54+
55+
---
56+
57+
**📝 Code Changes:**
58+
- UPDATE: build_hooks.py (269 lines changed)
59+
- UPDATE: setup.py (383 lines changed)
60+
- UPDATE: src/omnipkg/cli.py (1068 lines changed)
61+
- UPDATE: src/omnipkg/core.py (2691 lines changed)
62+
- UPDATE: src/omnipkg/dispatcher.c (292 lines changed)
63+
- UPDATE: src/omnipkg/dispatcher.py (341 lines changed)
64+
- UPDATE: src/omnipkg/i18n.py (12 lines changed)
65+
- UPDATE: src/omnipkg/installation/installers.py (805 lines changed)
66+
- UPDATE: src/omnipkg/installation/smart_install.py (2453 lines changed)
67+
- UPDATE: src/omnipkg/integration/reproducible.py (105 lines changed)
68+
- UPDATE: src/omnipkg/isolation/fs_lock_queue.py (2 lines changed)
69+
- UPDATE: src/omnipkg/isolation/fs_watcher.py (14 lines changed)
70+
- UPDATE: src/omnipkg/isolation/resource_monitor.py (432 lines changed)
71+
- UPDATE: src/omnipkg/isolation/worker_daemon.py (948 lines changed)
72+
- UPDATE: src/omnipkg/package_meta_builder.py (333 lines changed)
73+
- UPDATE: src/omnipkg/windows_bootstrap/launcher.py (2 lines changed)
74+
75+
**🧪 Tests:**
76+
- UPDATE: pytest.ini (1 lines)
77+
- UPDATE: src/omnipkg/tests/omnipkg_ipc_showcase.py (1002 lines)
78+
- UPDATE: src/omnipkg/tests/test_concurrent_install.py (4 lines)
79+
- UPDATE: src/omnipkg/tests/test_daemon_tags.py (40 lines)
80+
- UPDATE: src/omnipkg/tests/test_loader_stress_test.py (72 lines)
81+
- UPDATE: src/omnipkg/tests/test_old_rich.py (2 lines)
82+
- UPDATE: src/omnipkg/tests/test_rich_switching.py (6 lines)
83+
- UPDATE: src/omnipkg/tests/test_tensorflow_switching.py (17 lines)
84+
- NEW: tests/diag_uv_ffi.py (425 lines)
85+
- UPDATE: tests/test_dispatcher_contracts.py (2 lines)
86+
- UPDATE: tests/test_ffi_verify.py (22 lines)
87+
- UPDATE: tests/test_flask_port_finder.py (18 lines)
88+
- UPDATE: tests/test_omnipkg_contracts.py (22 lines)
89+
- NEW: tests/test_uv_ffi_contracts.py (793 lines)
90+
- UPDATE: tests/test_verify_bubble_deps.py (296 lines)
91+
92+
**⚙️ Configuration:**
93+
- pyproject.toml (2 lines)
94+
95+
**Additional Changes:**
96+
- Update 1 code files; Update configuration
97+
- Update 1 code files
98+
- fix: fix safe print usage in dispatcher.py
99+
- fix: resolve translation shadowing, standardize safe_print, and refactor tests
100+
- feat(i18n): per-language help routing and fast-path string micro-optimizations
101+
102+
**Updates:**
103+
- Update global reach badge in README
104+
105+
_51 files changed, 10172 insertions(+), 4637 deletions(-)_
106+
10107
## [3.3.5] — 2026-05-28
11108

12109
Core Synchronization & Cross-Platform Performance

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1456,6 +1456,7 @@ pip3 install omnipkg==3.3.5
14561456

14571457

14581458

1459+
14591460

14601461

14611462
<a href="https://www.piwheels.org/project/omnipkg/">

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ backend-path = ["."]
66

77
[project]
88
name = "omnipkg"
9-
version = "3.3.5"
9+
version = "3.4.0"
1010
authors = [
1111
{ name = "1minds3t", email = "1minds3t@proton.me" },
1212
]

requirements-trace.txt

Lines changed: 31 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,17 @@
44
#
55
# pip-compile --output-file=requirements-trace.txt --strip-extras pyproject.toml
66
#
7-
aiohappyeyeballs==2.6.1
7+
aiohappyeyeballs==2.6.2
88
# via aiohttp
9-
aiohttp==3.13.5 ; python_version >= "3.9"
9+
aiohttp==3.14.0 ; python_version >= "3.9"
1010
# via omnipkg (pyproject.toml)
1111
aiosignal==1.4.0
1212
# via aiohttp
1313
annotated-doc==0.0.4
1414
# via typer
15-
attrs==25.4.0
15+
attrs==26.1.0
1616
# via aiohttp
17-
authlib==1.6.9 ; python_version >= "3.9"
17+
authlib==1.7.2 ; python_version >= "3.9"
1818
# via omnipkg (pyproject.toml)
1919
blinker==1.9.0
2020
# via flask
@@ -24,49 +24,50 @@ cachecontrol==0.14.4
2424
# via
2525
# cachecontrol
2626
# pip-audit
27-
certifi==2026.2.25
27+
certifi==2026.5.20
2828
# via requests
2929
cffi==2.0.0
3030
# via cryptography
31-
charset-normalizer==3.4.6
31+
charset-normalizer==3.4.7
3232
# via requests
33-
click==8.3.1
34-
# via
35-
# flask
36-
# typer
37-
cryptography==46.0.7 ; python_version >= "3.8"
33+
click==8.4.1
34+
# via flask
35+
cryptography==48.0.0 ; python_version >= "3.8" and platform_system != "Emscripten"
3836
# via
3937
# authlib
38+
# joserfc
4039
# omnipkg (pyproject.toml)
41-
cyclonedx-python-lib==11.7.0
40+
cyclonedx-python-lib==11.8.0
4241
# via pip-audit
4342
defusedxml==0.7.1
4443
# via py-serializable
45-
filelock==3.25.2 ; python_version >= "3.10"
44+
filelock==3.29.1 ; python_version >= "3.10"
4645
# via
4746
# cachecontrol
4847
# omnipkg (pyproject.toml)
4948
flask==3.1.3 ; python_version >= "3.9"
5049
# via
5150
# flask-cors
5251
# omnipkg (pyproject.toml)
53-
flask-cors==6.0.2 ; python_version >= "3.8"
52+
flask-cors==6.0.3 ; python_version >= "3.8"
5453
# via omnipkg (pyproject.toml)
5554
frozenlist==1.8.0
5655
# via
5756
# aiohttp
5857
# aiosignal
59-
idna==3.11
58+
idna==3.18
6059
# via
6160
# requests
6261
# yarl
6362
itsdangerous==2.2.0
6463
# via flask
6564
jinja2==3.1.6
6665
# via flask
66+
joserfc==1.7.0
67+
# via authlib
6768
license-expression==30.4.4
6869
# via cyclonedx-python-lib
69-
markdown-it-py==4.0.0
70+
markdown-it-py==4.2.0
7071
# via rich
7172
markupsafe==3.0.3
7273
# via
@@ -83,7 +84,7 @@ multidict==6.7.1
8384
# yarl
8485
packageurl-python==0.17.6
8586
# via cyclonedx-python-lib
86-
packaging==26.0 ; python_version >= "3.10"
87+
packaging==26.2 ; python_version >= "3.10"
8788
# via
8889
# omnipkg (pyproject.toml)
8990
# pip-audit
@@ -94,13 +95,13 @@ pip-audit==2.10.0
9495
# via omnipkg (pyproject.toml)
9596
pip-requirements-parser==32.0.1
9697
# via pip-audit
97-
platformdirs==4.9.4
98+
platformdirs==4.10.0
9899
# via pip-audit
99-
propcache==0.4.1
100+
propcache==0.5.2
100101
# via
101102
# aiohttp
102103
# yarl
103-
psutil==7.2.2
104+
psutil==7.2.2 ; platform_system != "Emscripten"
104105
# via omnipkg (pyproject.toml)
105106
py-serializable==2.1.0
106107
# via cyclonedx-python-lib
@@ -112,12 +113,12 @@ pygments==2.20.0 ; python_version >= "3.9"
112113
# rich
113114
pyparsing==3.3.2
114115
# via pip-requirements-parser
115-
requests==2.33.1
116+
requests==2.34.2
116117
# via
117118
# cachecontrol
118119
# omnipkg (pyproject.toml)
119120
# pip-audit
120-
rich==14.3.3
121+
rich==15.0.0
121122
# via
122123
# omnipkg (pyproject.toml)
123124
# pip-audit
@@ -126,30 +127,31 @@ shellingham==1.5.4
126127
# via typer
127128
sortedcontainers==2.4.0
128129
# via cyclonedx-python-lib
129-
tomli==2.4.0
130+
tomli==2.4.1
130131
# via pip-audit
131132
tomli-w==1.2.0
132133
# via pip-audit
133-
typer==0.24.1
134+
typer==0.26.7
134135
# via omnipkg (pyproject.toml)
135136
typing-extensions==4.15.0
136137
# via
138+
# aiohttp
137139
# aiosignal
138140
# cyclonedx-python-lib
139-
urllib3==2.6.3 ; python_version >= "3.10"
141+
urllib3==2.7.0 ; python_version >= "3.10"
140142
# via
141143
# omnipkg (pyproject.toml)
142144
# requests
143-
uv==0.10.11 ; python_version >= "3.8"
145+
uv==0.11.19 ; python_version >= "3.8"
144146
# via omnipkg (pyproject.toml)
145-
uv-ffi==0.10.8.post7 ; python_version >= "3.8" and python_version <= "3.14"
147+
uv-ffi==0.10.8.post15 ; python_version >= "3.8" and python_version <= "3.14" and platform_system != "Emscripten"
146148
# via omnipkg (pyproject.toml)
147-
werkzeug==3.1.6 ; python_version >= "3.9"
149+
werkzeug==3.1.8 ; python_version >= "3.9"
148150
# via
149151
# flask
150152
# flask-cors
151153
# omnipkg (pyproject.toml)
152-
yarl==1.23.0
154+
yarl==1.24.2
153155
# via aiohttp
154156

155157
# The following packages are considered to be unsafe in a requirements file:

requirements.txt

Lines changed: 27 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -4,60 +4,61 @@
44
#
55
# pip-compile --no-annotate --output-file=requirements.txt --strip-extras pyproject.toml
66
#
7-
aiohappyeyeballs==2.6.1
8-
aiohttp==3.13.5 ; python_version >= "3.9"
7+
aiohappyeyeballs==2.6.2
8+
aiohttp==3.14.0 ; python_version >= "3.9"
99
aiosignal==1.4.0
1010
annotated-doc==0.0.4
11-
attrs==25.4.0
12-
authlib==1.6.9 ; python_version >= "3.9"
11+
attrs==26.1.0
12+
authlib==1.7.2 ; python_version >= "3.9"
1313
blinker==1.9.0
1414
boolean-py==5.0
1515
cachecontrol==0.14.4
16-
certifi==2026.2.25
16+
certifi==2026.5.20
1717
cffi==2.0.0
18-
charset-normalizer==3.4.6
19-
click==8.3.1
20-
cryptography==46.0.7 ; python_version >= "3.8"
21-
cyclonedx-python-lib==11.7.0
18+
charset-normalizer==3.4.7
19+
click==8.4.1
20+
cryptography==48.0.0 ; python_version >= "3.8" and platform_system != "Emscripten"
21+
cyclonedx-python-lib==11.8.0
2222
defusedxml==0.7.1
23-
filelock==3.25.2 ; python_version >= "3.10"
23+
filelock==3.29.1 ; python_version >= "3.10"
2424
flask==3.1.3 ; python_version >= "3.9"
25-
flask-cors==6.0.2 ; python_version >= "3.8"
25+
flask-cors==6.0.3 ; python_version >= "3.8"
2626
frozenlist==1.8.0
27-
idna==3.11
27+
idna==3.18
2828
itsdangerous==2.2.0
2929
jinja2==3.1.6
30+
joserfc==1.7.0
3031
license-expression==30.4.4
31-
markdown-it-py==4.0.0
32+
markdown-it-py==4.2.0
3233
markupsafe==3.0.3
3334
mdurl==0.1.2
3435
msgpack==1.1.2
3536
multidict==6.7.1
3637
packageurl-python==0.17.6
37-
packaging==26.0 ; python_version >= "3.10"
38+
packaging==26.2 ; python_version >= "3.10"
3839
pip-api==0.0.34
3940
pip-audit==2.10.0
4041
pip-requirements-parser==32.0.1
41-
platformdirs==4.9.4
42-
propcache==0.4.1
43-
psutil==7.2.2
42+
platformdirs==4.10.0
43+
propcache==0.5.2
44+
psutil==7.2.2 ; platform_system != "Emscripten"
4445
py-serializable==2.1.0
4546
pycparser==3.0
4647
pygments==2.20.0 ; python_version >= "3.9"
4748
pyparsing==3.3.2
48-
requests==2.33.1
49-
rich==14.3.3
49+
requests==2.34.2
50+
rich==15.0.0
5051
shellingham==1.5.4
5152
sortedcontainers==2.4.0
52-
tomli==2.4.0
53+
tomli==2.4.1
5354
tomli-w==1.2.0
54-
typer==0.24.1
55+
typer==0.26.7
5556
typing-extensions==4.15.0
56-
urllib3==2.6.3 ; python_version >= "3.10"
57-
uv==0.10.11 ; python_version >= "3.8"
58-
uv-ffi==0.10.8.post7 ; python_version >= "3.8" and python_version <= "3.14"
59-
werkzeug==3.1.6 ; python_version >= "3.9"
60-
yarl==1.23.0
57+
urllib3==2.7.0 ; python_version >= "3.10"
58+
uv==0.11.19 ; python_version >= "3.8"
59+
uv-ffi==0.10.8.post15 ; python_version >= "3.8" and python_version <= "3.14" and platform_system != "Emscripten"
60+
werkzeug==3.1.8 ; python_version >= "3.9"
61+
yarl==1.24.2
6162

6263
# The following packages are considered to be unsafe in a requirements file:
6364
# pip

src/omnipkg/core.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13744,10 +13744,11 @@ def smart_uninstall(
1374413744
bubble_dir = dist_info_path.parent
1374513745

1374613746
# A critical sanity check to ensure we're deleting the correct directory.
13747+
def _norm(s): return s.replace('-', '_').lower()
1374713748
expected_bubble_name = f"{canonicalize_name(item_name)}-{item_version}"
1374813749
expected_bubble_name_alt = f"{item_name}-{item_version}"
1374913750
expected_bubble_name_underscored = f"{item_name.replace('-', '_')}-{item_version}"
13750-
if bubble_dir.name in (expected_bubble_name, expected_bubble_name_alt, expected_bubble_name_underscored) and bubble_dir.is_dir():
13751+
if _norm(bubble_dir.name) in (_norm(expected_bubble_name), _norm(expected_bubble_name_alt), _norm(expected_bubble_name_underscored)) and bubble_dir.is_dir():
1375113752
safe_print(_("🗑️ Deleting bubble directory: {}").format(bubble_dir))
1375213753
shutil.rmtree(bubble_dir, ignore_errors=True)
1375313754
else:

0 commit comments

Comments
 (0)