Skip to content

Commit be9369d

Browse files
1minds3t1minds3t
committed
docs: add deep-dive pages for C-Extension and TensorFlow demos
- 📚 New Demo #3 (C-Extension Switching): Documents the 'Nuclear' NumPy/SciPy hot-swap logic and binary path injection. - 📚 New Demo #4 (TensorFlow Dependency): Documents complex nested dependency graph resolution and stack management. - 🧭 Nav: Updated mkdocs.yml and .pages.yml to include the new tutorials in the Demos section.
1 parent 11e1f0e commit be9369d

4 files changed

Lines changed: 259 additions & 0 deletions

File tree

‎docs/demos/.pages.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
nav:
22
- index.md
33
- demos_rich_module_switching.md
4+
- demos_c_extension_switching.md
5+
- demos_tensorflow_dependency_switching.md
46
title: Demos

‎docs/demos/demos_c_extension_switching.md‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: C Extension Switching
3+
doc_type: demo
4+
status: draft
5+
generated: true
6+
created: '2026-01-07'
7+
builder: omnipkg-docbuilder
8+
builder_version: 2.1.0
9+
section: demos
10+
---
11+
12+
# C Extension Switching
13+
14+
## Overview
15+
16+
# NumPy + SciPy Stress Test
17+
18+
!!! danger "The Holy Grail of Python Conflicts"
19+
This demo performs an action considered "impossible" in standard Python: **Swapping C-extension modules (NumPy/SciPy) mid-execution without crashing the interpreter.**
20+
21+
This demo is the ultimate stress test for OmniPkg's **LibResolver**. It forces the interpreter to load, unload, and reload incompatible C-extension binaries (shared objects/DLLs) within the same process.
22+
23+
## Usage
24+
25+
**Prerequisite:** This demo requires **Python 3.11** (due to specific binary wheels used in the test).
26+
**Try it now:** Cloud users can click the play button to run this demo instantly:
27+
```bash
28+
omnipkg demo 3
29+
```
30+
To run this demo locally:
31+
32+
```bash
33+
# Run Demo #3: The "Nuclear" Option
34+
omnipkg demo 3
35+
```
36+
37+
## What You'll Learn
38+
39+
1. **C-Extension Hot-Swapping**: How OmniPkg manages the memory lifecycle of shared libraries (`.so` / `.pyd`).
40+
2. **Binary Compatibility**: How it ensures `numpy==1.24.3` (linked to older BLAS) doesn't crash when swapped for `1.26.4`.
41+
3. **The "Nuclear" Swap**: Watch it combine incompatible pairs (`numpy==1.24.3 + scipy==1.12.0`) vs (`numpy==1.26.4 + scipy==1.16.1`) in real-time.
42+
43+
## The Code
44+
45+
This script performs a series of high-speed swaps between incompatible versions of NumPy and SciPy, verifying that the math operations (which rely on C code) return correct results every time.
46+
47+
```python title="demo_numpy_scipy.py"
48+
from omnipkg.loader import omnipkgLoader
49+
import numpy as np
50+
51+
# 1. Baseline: NumPy 1.26.4 (Main Env)
52+
print(f"Baseline Version: {np.__version__}")
53+
# Output: 1.26.4
54+
55+
# 2. The "Impossible" Swap: NumPy 1.24.3
56+
# This involves unloading C-extensions and reloading older binaries
57+
with omnipkgLoader("numpy==1.24.3"):
58+
import numpy as np
59+
60+
# Verify we are running C-code from 1.24.3
61+
print(f"Swapped Version: {np.__version__}")
62+
# Output: 1.24.3
63+
64+
# Verify math accuracy (ensure memory isn't corrupted)
65+
print(f"Array Sum: {np.array([1, 2, 3]).sum()}")
66+
# Output: 6
67+
68+
# 3. Restore: Back to 1.26.4
69+
# OmniPkg handles the cleanup and re-linking automatically
70+
import numpy as np
71+
print(f"Restored Version: {np.__version__}")
72+
# Output: 1.26.4
73+
```
74+
75+
## Live Execution Log
76+
77+
Pay attention to the **Activation Time**. OmniPkg swaps these massive C-libraries in **~118ms**.
78+
79+
```text title="omnipkg demo 3"
80+
(evocoder_env) minds3t@aiminingrig:~/omnipkg$ omnipkg demo 3
81+
82+
============================================================
83+
🚀 STEP 3: Executing the Nuclear Test
84+
============================================================
85+
86+
💥 NUMPY VERSION JUGGLING:
87+
88+
⚡ Switching to numpy==1.24.3
89+
🧹 Purging 1 module(s) from memory...
90+
- 🔒 STRICT mode
91+
- 🔩 Activating binary path: .../numpy-1.24.3/bin
92+
⚡ HEALED in 21,684.8 μs
93+
✅ Version: 1.24.3
94+
🔢 Array sum: 6
95+
⚡ Activation time: 118.52ms
96+
🎯 Version verification: PASSED
97+
98+
🌀 omnipkg loader: Deactivating numpy==1.24.3...
99+
✅ Environment restored.
100+
⏱️ Swap Time: 47,614.668 μs
101+
102+
⚡ Switching to numpy==1.26.4
103+
✅ Version: 1.26.4
104+
🔢 Array sum: 6
105+
⚡ Activation time: 119.48ms
106+
🎯 Version verification: PASSED
107+
108+
🤯 NUMPY + SCIPY VERSION MIXING:
109+
110+
🌀 COMBO: numpy==1.24.3 + scipy==1.12.0
111+
🧪 numpy: 1.24.3, scipy: 1.12.0
112+
🔗 Compatibility check: [1. 2. 3.]
113+
🎯 Version verification: BOTH PASSED!
114+
⚡ Total combo time: 162.69ms
115+
116+
🌀 COMBO: numpy==1.26.4 + scipy==1.16.1
117+
🧪 numpy: 1.26.4, scipy: 1.16.1
118+
🔗 Compatibility check: [1. 2. 3.]
119+
🎯 Version verification: BOTH PASSED!
120+
⚡ Total combo time: 130.39ms
121+
122+
🚨 OMNIPKG SURVIVED NUCLEAR TESTING! 🎇
123+
```
124+
125+
## How It Works
126+
127+
1. **Memory Purge**: Before swapping, OmniPkg forces a garbage collection and manually unregisters the C-extension modules from `sys.modules`.
128+
2. **Binary Path Injection**: It injects the `bin` path of the target version into `LD_LIBRARY_PATH` (Linux) or `PATH` (Windows) to ensure the dynamic linker finds the correct `.so` / `.dll` files.
129+
3. **Strict Mode**: For packages identified as "C-Heavy" (like NumPy), OmniPkg enables Strict Mode, which performs deeper verification to prevent segmentation faults.

‎docs/demos/demos_tensorflow_dependency_switching.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
title: Tensorflow Dependency Switching
3+
doc_type: demo
4+
status: draft
5+
generated: true
6+
created: '2026-01-07'
7+
builder: omnipkg-docbuilder
8+
builder_version: 2.1.0
9+
section: demos
10+
---
11+
12+
# Tensorflow Dependency Switching
13+
14+
## Overview
15+
16+
This log demonstrates **Complex Dependency Tree Management**.
17+
18+
While the previous demo showed swapping a single binary, this one shows swapping **TensorFlow** (a massive package) along with its sub-dependencies (`typing_extensions`, `keras`) in a nested context. It proves OmniPkg handles the "Butterfly Effect" of dependencies—changing one package updates the entire graph instantly.
19+
20+
# TensorFlow Test
21+
22+
!!! note "Demo Prerequisite"
23+
This demo requires **Python 3.11**.
24+
25+
This demo validates OmniPkg's ability to handle **massive, complex dependency graphs**. Unlike simple packages, TensorFlow brings along a huge tree of dependencies (Keras, TensorBoard, Protobuf, etc.). OmniPkg must ensure *all* of them are swapped correctly to prevent ABI mismatches.
26+
27+
## Usage
28+
29+
```bash
30+
# Run Demo #4: Complex Dependency Graph Switching
31+
omnipkg demo 4
32+
```
33+
34+
## What You'll Learn
35+
36+
1. **Deep Graph Swapping**: How activating `tensorflow==2.13.0` automatically pulls in the correct `keras`, `typing_extensions`, and `numpy` versions.
37+
2. **Nested Contexts**: How to use `omnipkgLoader` inside another `omnipkgLoader` block (Inception-style environment management).
38+
3. **Strict Mode Verification**: Verifying that no "cloaked" modules (old versions) leak into the new context.
39+
40+
## The Code
41+
42+
This script performs a nested activation: it first activates a specific version of `typing_extensions`, and then *inside that block*, activates an old version of `tensorflow` that requires a *different* `typing_extensions`. OmniPkg handles this conflict resolution in real-time.
43+
44+
```python title="demo_tensorflow.py"
45+
from omnipkg.loader import omnipkgLoader
46+
47+
# 1. Outer Context: Activate older typing_extensions
48+
with omnipkgLoader("typing_extensions==4.5.0"):
49+
import typing_extensions as te
50+
print(f"Outer Version: {te.__version__}")
51+
# Output: 4.5.0
52+
53+
# 2. Inner Context: Activate TensorFlow 2.13.0
54+
# TensorFlow 2.13.0 might require a different typing_extensions.
55+
# OmniPkg creates a NEW nested bubble for this block.
56+
with omnipkgLoader("tensorflow==2.13.0"):
57+
import tensorflow as tf
58+
import typing_extensions as te_inner
59+
60+
print(f"Inner TF Version: {tf.__version__}")
61+
# Output: 2.13.0
62+
63+
print(f"Inner TE Version: {te_inner.__version__}")
64+
# Output: Matches TF's requirement (automatically resolved)
65+
66+
# Verify the complex graph works by creating a model
67+
model = tf.keras.Sequential([tf.keras.layers.Dense(1)])
68+
print("Model created successfully")
69+
70+
# 3. Back to Outer Context
71+
# TensorFlow is unloaded; typing_extensions reverts to 4.5.0
72+
import typing_extensions as te_back
73+
print(f"Restored Version: {te_back.__version__}")
74+
# Output: 4.5.0
75+
```
76+
77+
## Live Execution Log
78+
79+
Notice the **Nested activation (depth=2)** line in the logs. This confirms that OmniPkg is maintaining a stack of environments and correctly popping them off as the code exits the `with` blocks.
80+
81+
```text title="omnipkg demo 4"
82+
(evocoder_env) minds3t@aiminingrig:~/omnipkg$ omnipkg demo 4
83+
84+
================================================================================
85+
🚀 🚨 OMNIPKG TENSORFLOW DEPENDENCY SWITCHING TEST 🚨
86+
================================================================================
87+
88+
--- Nested Loader Test ---
89+
🌀 Testing nested loader usage...
90+
✅ Outer context - Typing Extensions: 4.5.0
91+
92+
🚀 Fast-activating tensorflow==2.13.0 ...
93+
📂 Searching for bubble: .../tensorflow-2.13.0
94+
📊 Bubble: 40 packages, 0 conflicts
95+
🧹 Purging 40 module(s) from memory...
96+
- 🔒 STRICT mode
97+
⚡ HEALED in 25,743.3 μs
98+
✅ Inner context - TensorFlow: 2.13.0
99+
✅ Inner context - Typing Extensions: 4.5.0
100+
✅ Nested loader test: Model created successfully
101+
102+
🌀 omnipkg loader: Deactivating tensorflow==2.13.0 [depth=2]...
103+
✅ Environment restored.
104+
⏱️ Swap Time: 26,032.537 μs
105+
106+
🌀 omnipkg loader: Deactivating typing_extensions==4.5.0...
107+
✅ Environment restored.
108+
⏱️ Swap Time: 125,331.240 μs
109+
110+
================================================================================
111+
🚀 STEP 3: Test Results Summary
112+
================================================================================
113+
Test 1 (TensorFlow 2.13.0 Bubble): ✅ PASSED
114+
Test 2 (Dependency Switching): ✅ PASSED
115+
Test 3 (Nested Loaders): ✅ PASSED
116+
117+
Overall: 3/3 tests passed
118+
🎉 DEMO PASSED! 🎉
119+
```
120+
121+
## How It Works
122+
123+
1. **Graph Resolution**: When `tensorflow==2.13.0` is requested, the LibResolver walks its dependency tree. It identifies 40+ sub-dependencies.
124+
2. **Memory Purge**: It unloads all 40 modules from `sys.modules` to ensure no stale code remains.
125+
3. **Stack Management**: The `omnipkgLoader` tracks the "depth" of the context. When the inner block exits, it doesn't just clear `sys.path`—it restores the `sys.path` of the *outer* block (restoring `typing_extensions==4.5.0`).

‎mkdocs.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,10 @@ nav:
2727
- CI Verification: platform_support/platform_support_ci_verification.md
2828
- Demos:
2929
- Overview: demos/index.md
30+
- C Extension Switching: demos/demos_c_extension_switching.md
3031
- Rich Module Switching: demos/demos_rich_module_switching.md
32+
- Tensorflow Dependency Switching:
33+
demos/demos_tensorflow_dependency_switching.md
3134
- UV Binary Switching: demos/demos_uv_binary_switching.md
3235
theme:
3336
name: material

0 commit comments

Comments
 (0)