Skip to content

Commit 4cb5197

Browse files
0xrinegade0xrinegade
committed
Fix cross-compilation for aarch64-unknown-linux-gnu
1 parent 521d3c2 commit 4cb5197

File tree

6 files changed

+312
-5
lines changed

6 files changed

+312
-5
lines changed

‎.github/workflows/release.yml‎

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -135,11 +135,29 @@ jobs:
135135
path: target/distrib/
136136
merge-multiple: true
137137

138-
- name: Set up OpenSSL (macOS)
139-
if: runner.os == 'macOS'
138+
- name: Set up OpenSSL and cross-compilation environment
140139
run: |
141-
brew install openssl@3
142-
echo "OPENSSL_DIR=$(brew --prefix openssl@3)" >> $GITHUB_ENV
140+
# Make the cross-compilation setup script executable
141+
chmod +x cross-compile-setup.sh
142+
143+
# Source the script to set up the environment
144+
if [[ "${{ matrix.targets }}" == *"aarch64-unknown-linux-gnu"* ]]; then
145+
source ./cross-compile-setup.sh
146+
# Export variables to GITHUB_ENV so they persist across steps
147+
echo "OPENSSL_DIR=$OPENSSL_DIR" >> $GITHUB_ENV
148+
echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=$CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER" >> $GITHUB_ENV
149+
echo "CC_aarch64_unknown_linux_gnu=$CC_aarch64_unknown_linux_gnu" >> $GITHUB_ENV
150+
echo "CXX_aarch64_unknown_linux_gnu=$CXX_aarch64_unknown_linux_gnu" >> $GITHUB_ENV
151+
echo "PKG_CONFIG_ALLOW_CROSS=$PKG_CONFIG_ALLOW_CROSS" >> $GITHUB_ENV
152+
echo "PKG_CONFIG_PATH=$PKG_CONFIG_PATH" >> $GITHUB_ENV
153+
echo "CARGO_BUILD_RUSTFLAGS=$CARGO_BUILD_RUSTFLAGS" >> $GITHUB_ENV
154+
if [ -n "$PKG_CONFIG_AARCH64_UNKNOWN_LINUX_GNU_SYSROOT" ]; then
155+
echo "PKG_CONFIG_AARCH64_UNKNOWN_LINUX_GNU_SYSROOT=$PKG_CONFIG_AARCH64_UNKNOWN_LINUX_GNU_SYSROOT" >> $GITHUB_ENV
156+
fi
157+
elif [[ "${{ runner.os }}" == "macOS" ]]; then
158+
brew install openssl@3
159+
echo "OPENSSL_DIR=$(brew --prefix openssl@3)" >> $GITHUB_ENV
160+
fi
143161
144162
- name: Install dependencies
145163
run: |

‎cli/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ log = "0.4"
3131
env_logger = "0.10"
3232
dirs = "5.0"
3333
toml = "0.8"
34-
reqwest = { version = "0.11", features = ["json"] }
34+
reqwest = { version = "0.11", features = ["json", "native-tls-vendored"] }
3535
chrono = "0.4"
3636
shellexpand = "3.1"
3737
tempfile = "3.8"

‎cross-compile-setup.sh‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
#!/bin/bash
2+
set -e
3+
4+
# This script sets up the environment for cross-compiling to aarch64-unknown-linux-gnu
5+
# with proper OpenSSL configuration
6+
7+
# Install required packages if not already installed
8+
if [ "$(uname)" = "Darwin" ]; then
9+
# macOS
10+
brew install pkg-config openssl@3
11+
export OPENSSL_DIR=$(brew --prefix openssl@3)
12+
13+
# Install cross-compilation tools
14+
brew tap messense/macos-cross-toolchains
15+
brew install aarch64-unknown-linux-gnu
16+
17+
# Set up environment variables for cross-compilation
18+
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-unknown-linux-gnu-gcc
19+
export CC_aarch64_unknown_linux_gnu=aarch64-unknown-linux-gnu-gcc
20+
export CXX_aarch64_unknown_linux_gnu=aarch64-unknown-linux-gnu-g++
21+
22+
# Configure pkg-config for cross-compilation
23+
export PKG_CONFIG_ALLOW_CROSS=1
24+
export PKG_CONFIG_PATH="/usr/local/opt/openssl@3/lib/pkgconfig"
25+
26+
# Use the vendored OpenSSL feature for reqwest
27+
export CARGO_BUILD_RUSTFLAGS="--cfg reqwest_unstable"
28+
else
29+
# Linux
30+
sudo apt-get update
31+
sudo apt-get install -y pkg-config libssl-dev gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
32+
33+
# Set up environment variables for cross-compilation
34+
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
35+
export CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc
36+
export CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++
37+
38+
# Configure pkg-config for cross-compilation
39+
export PKG_CONFIG_ALLOW_CROSS=1
40+
export PKG_CONFIG_AARCH64_UNKNOWN_LINUX_GNU_SYSROOT=/usr/aarch64-linux-gnu
41+
export PKG_CONFIG_PATH="/usr/lib/aarch64-linux-gnu/pkgconfig"
42+
43+
# Use the vendored OpenSSL feature for reqwest
44+
export CARGO_BUILD_RUSTFLAGS="--cfg reqwest_unstable"
45+
fi
46+
47+
echo "Cross-compilation environment set up for aarch64-unknown-linux-gnu"
48+
echo "Run 'source cross-compile-setup.sh' before building"
49+
50+
# Print the current environment for debugging
51+
echo "Current environment:"
52+
echo "OPENSSL_DIR=$OPENSSL_DIR"
53+
echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=$CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER"
54+
echo "PKG_CONFIG_ALLOW_CROSS=$PKG_CONFIG_ALLOW_CROSS"
55+
echo "PKG_CONFIG_PATH=$PKG_CONFIG_PATH"

‎website/docs/knowledge_graph.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# lessVM Knowledge Graph
22

3+
## System Architecture
4+
35
```mermaid
46
graph TB
57
%% Core Components
@@ -120,3 +122,67 @@ This visualization helps understand:
120122
- Resource monitoring
121123
- Error handling paths
122124
- Testing coverage
125+
126+
## Build & Deployment
127+
128+
```mermaid
129+
graph TB
130+
%% Build System
131+
BuildSystem[Build System]
132+
CrossCompilation[Cross Compilation]
133+
ArtifactGeneration[Artifact Generation]
134+
ReleaseWorkflow[Release Workflow]
135+
136+
%% Dependencies
137+
OpenSSL[OpenSSL]
138+
PkgConfig[pkg-config]
139+
CargoFeatures[Cargo Features]
140+
141+
%% Platforms
142+
MacOS[macOS]
143+
Linux[Linux]
144+
ARM64[ARM64]
145+
X86_64[x86_64]
146+
147+
%% Relationships
148+
BuildSystem --> |uses| CrossCompilation
149+
BuildSystem --> |produces| ArtifactGeneration
150+
BuildSystem --> |integrated with| ReleaseWorkflow
151+
152+
CrossCompilation --> |depends on| OpenSSL
153+
CrossCompilation --> |configured by| PkgConfig
154+
CrossCompilation --> |leverages| CargoFeatures
155+
156+
CrossCompilation --> |targets| MacOS
157+
CrossCompilation --> |targets| Linux
158+
CrossCompilation --> |targets| ARM64
159+
CrossCompilation --> |targets| X86_64
160+
161+
%% Component Types
162+
classDef build fill:#f9f,stroke:#333,stroke-width:2px
163+
classDef dependency fill:#ff9,stroke:#333,stroke-width:2px
164+
classDef platform fill:#9f9,stroke:#333,stroke-width:2px
165+
166+
class BuildSystem,CrossCompilation,ArtifactGeneration,ReleaseWorkflow build
167+
class OpenSSL,PkgConfig,CargoFeatures dependency
168+
class MacOS,Linux,ARM64,X86_64 platform
169+
```
170+
171+
The build system diagram shows:
172+
173+
1. Build Components (Pink)
174+
- Build System for compiling code
175+
- Cross Compilation for multi-platform support
176+
- Artifact Generation for creating distributable packages
177+
- Release Workflow for automating deployment
178+
179+
2. Dependencies (Yellow)
180+
- OpenSSL for secure communications
181+
- pkg-config for library detection
182+
- Cargo Features for conditional compilation
183+
184+
3. Target Platforms (Green)
185+
- Support for multiple operating systems (macOS, Linux)
186+
- Support for multiple architectures (ARM64, x86_64)
187+
188+
For detailed information about cross-compilation for aarch64-unknown-linux-gnu, see [Cross-Compilation Knowledge Graph](knowledge_graph_cross_compilation.html).

‎website/docs/knowledge_graph_cross_compilation.html‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8">
5+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
6+
<title>Cross-Compilation Knowledge Graph - lessVM</title>
7+
<link rel="stylesheet" href="../styles.css">
8+
</head>
9+
<body>
10+
<header>
11+
<nav>
12+
<a href="../index.html">Home</a>
13+
<a href="architecture.html">Architecture</a>
14+
<a href="instructions.html">Instructions</a>
15+
<a href="examples.html">Examples</a>
16+
<a href="solana.html">Solana</a>
17+
<a href="cli.html">CLI</a>
18+
<a href="knowledge_graph.html">Knowledge Graph</a>
19+
</nav>
20+
</header>
21+
<main>
22+
<h1>Cross-Compilation Knowledge Graph</h1>
23+
24+
<h2>Cross-Compilation for aarch64-unknown-linux-gnu</h2>
25+
26+
<h3>Problem</h3>
27+
<p>When cross-compiling the LessVM CLI for aarch64-unknown-linux-gnu, OpenSSL cannot be found via pkg-config because pkg-config is not configured for cross-compilation.</p>
28+
29+
<h3>Solution</h3>
30+
<ol>
31+
<li>Use the vendored OpenSSL feature in reqwest to avoid the need for pkg-config during cross-compilation.</li>
32+
<li>Set up the cross-compilation environment with appropriate environment variables.</li>
33+
</ol>
34+
35+
<h3>Implementation Details</h3>
36+
37+
<h4>1. Modified Dependencies</h4>
38+
<p>Added the <code>native-tls-vendored</code> feature to reqwest in cli/Cargo.toml:</p>
39+
<pre><code>reqwest = { version = "0.11", features = ["json", "native-tls-vendored"] }</code></pre>
40+
41+
<p>This feature makes reqwest use a vendored version of OpenSSL, eliminating the need for pkg-config to find a system-installed version.</p>
42+
43+
<h4>2. Environment Setup Script</h4>
44+
<p>Created a script (<code>cross-compile-setup.sh</code>) that sets up the necessary environment variables for cross-compilation:</p>
45+
46+
<h5>For macOS:</h5>
47+
<ul>
48+
<li>Installs required packages: pkg-config, openssl@3</li>
49+
<li>Installs cross-compilation toolchain for aarch64-unknown-linux-gnu</li>
50+
<li>Sets up environment variables: CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER, CC_aarch64_unknown_linux_gnu, etc.</li>
51+
<li>Configures pkg-config for cross-compilation: PKG_CONFIG_ALLOW_CROSS, PKG_CONFIG_PATH</li>
52+
</ul>
53+
54+
<h5>For Linux:</h5>
55+
<ul>
56+
<li>Installs required packages: pkg-config, libssl-dev, gcc-aarch64-linux-gnu, g++-aarch64-linux-gnu</li>
57+
<li>Sets up environment variables for cross-compilation</li>
58+
<li>Configures pkg-config for cross-compilation</li>
59+
</ul>
60+
61+
<h3>Usage</h3>
62+
<ol>
63+
<li>Source the environment setup script before building:
64+
<pre><code>source cross-compile-setup.sh</code></pre>
65+
</li>
66+
<li>Build for the target platform:
67+
<pre><code>cargo build --target aarch64-unknown-linux-gnu</code></pre>
68+
</li>
69+
</ol>
70+
71+
<h3>Technical Explanation</h3>
72+
<p>The issue occurs because pkg-config is not configured to look for libraries in the sysroot for the target platform. When cross-compiling, pkg-config needs to be told where to find libraries for the target architecture.</p>
73+
74+
<p>The vendored OpenSSL feature in reqwest is the simplest solution as it bundles OpenSSL with the build, avoiding the need for pkg-config to find it. For other dependencies that might require OpenSSL, the environment variables set in the script ensure that pkg-config can find the appropriate libraries.</p>
75+
76+
<h3>Decision Making Process</h3>
77+
<ol>
78+
<li>Identified that the issue was related to OpenSSL not being found during cross-compilation</li>
79+
<li>Considered two approaches:
80+
<ul>
81+
<li>Configure pkg-config for cross-compilation</li>
82+
<li>Use vendored dependencies where possible</li>
83+
</ul>
84+
</li>
85+
<li>Implemented both approaches for maximum compatibility:
86+
<ul>
87+
<li>Modified reqwest to use vendored OpenSSL</li>
88+
<li>Created a script to set up the cross-compilation environment</li>
89+
</ul>
90+
</li>
91+
</ol>
92+
93+
<h3>References</h3>
94+
<ul>
95+
<li><a href="https://rust-lang.github.io/rustup/cross-compilation.html">Rust Cross-Compilation Guide</a></li>
96+
<li><a href="https://pkg-config.freedesktop.org/pkg-config-guide.html">pkg-config Cross-Compilation Documentation</a></li>
97+
<li><a href="https://docs.rs/reqwest/latest/reqwest/">reqwest Documentation</a></li>
98+
</ul>
99+
</main>
100+
<footer>
101+
<p>&copy; 2025 lessVM Team</p>
102+
</footer>
103+
</body>
104+
</html>

‎website/docs/knowledge_graph_cross_compilation.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Cross-Compilation Knowledge Graph
2+
3+
## Cross-Compilation for aarch64-unknown-linux-gnu
4+
5+
### Problem
6+
When cross-compiling the LessVM CLI for aarch64-unknown-linux-gnu, OpenSSL cannot be found via pkg-config because pkg-config is not configured for cross-compilation.
7+
8+
### Solution
9+
1. Use the vendored OpenSSL feature in reqwest to avoid the need for pkg-config during cross-compilation.
10+
2. Set up the cross-compilation environment with appropriate environment variables.
11+
12+
### Implementation Details
13+
14+
#### 1. Modified Dependencies
15+
Added the `native-tls-vendored` feature to reqwest in cli/Cargo.toml:
16+
```toml
17+
reqwest = { version = "0.11", features = ["json", "native-tls-vendored"] }
18+
```
19+
20+
This feature makes reqwest use a vendored version of OpenSSL, eliminating the need for pkg-config to find a system-installed version.
21+
22+
#### 2. Environment Setup Script
23+
Created a script (`cross-compile-setup.sh`) that sets up the necessary environment variables for cross-compilation:
24+
25+
- For macOS:
26+
- Installs required packages: pkg-config, openssl@3
27+
- Installs cross-compilation toolchain for aarch64-unknown-linux-gnu
28+
- Sets up environment variables: CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER, CC_aarch64_unknown_linux_gnu, etc.
29+
- Configures pkg-config for cross-compilation: PKG_CONFIG_ALLOW_CROSS, PKG_CONFIG_PATH
30+
31+
- For Linux:
32+
- Installs required packages: pkg-config, libssl-dev, gcc-aarch64-linux-gnu, g++-aarch64-linux-gnu
33+
- Sets up environment variables for cross-compilation
34+
- Configures pkg-config for cross-compilation
35+
36+
### Usage
37+
1. Source the environment setup script before building:
38+
```bash
39+
source cross-compile-setup.sh
40+
```
41+
42+
2. Build for the target platform:
43+
```bash
44+
cargo build --target aarch64-unknown-linux-gnu
45+
```
46+
47+
### Technical Explanation
48+
The issue occurs because pkg-config is not configured to look for libraries in the sysroot for the target platform. When cross-compiling, pkg-config needs to be told where to find libraries for the target architecture.
49+
50+
The vendored OpenSSL feature in reqwest is the simplest solution as it bundles OpenSSL with the build, avoiding the need for pkg-config to find it. For other dependencies that might require OpenSSL, the environment variables set in the script ensure that pkg-config can find the appropriate libraries.
51+
52+
### Decision Making Process
53+
1. Identified that the issue was related to OpenSSL not being found during cross-compilation
54+
2. Considered two approaches:
55+
- Configure pkg-config for cross-compilation
56+
- Use vendored dependencies where possible
57+
3. Implemented both approaches for maximum compatibility:
58+
- Modified reqwest to use vendored OpenSSL
59+
- Created a script to set up the cross-compilation environment
60+
61+
### References
62+
- [Rust Cross-Compilation Guide](https://rust-lang.github.io/rustup/cross-compilation.html)
63+
- [pkg-config Cross-Compilation Documentation](https://pkg-config.freedesktop.org/pkg-config-guide.html)
64+
- [reqwest Documentation](https://docs.rs/reqwest/latest/reqwest/)

0 commit comments

Comments
 (0)