feat: Rust crate integration and package manager

[mjm918]

Description

Enable naml to use Rust crates directly, leveraging the Rust ecosystem. This is the foundation for naml's package manager.

Goals

1. Use Rust crates from naml code - Access the entire crates.io ecosystem
2. Automatic binding generation - No manual FFI declarations needed
3. Package manager - Resolve, download, and build dependencies
4. Type mapping - Seamless conversion between Rust and naml types

---

Proposed Syntax

naml.toml - Dependency Declaration

[package]
name = "my-app"
version = "0.1.0"

[dependencies]
# naml packages (from naml registry)
some-naml-lib = "1.0"

[crates]
# Rust crates (from crates.io)
serde = "1.0"
serde_json = "1.0"
reqwest = { version = "0.11", features = ["json", "blocking"] }
tokio = { version = "1.0", features = ["rt-multi-thread", "macros"] }
regex = "1.10"

[crates.uuid]
version = "1.0"
features = ["v4", "serde"]

naml Code - Using Rust Crates

use crate::serde_json::*;
use crate::reqwest;
use crate::regex::Regex;

fn main() {
    // Use serde_json
    var data: JsonValue = json_parse("{\"name\": \"naml\"}");
    var name: string = data["name"] as string;
    
    // Use regex
    var re: Regex = Regex.new("\\d+");
    var found: bool = re.is_match("hello123");
    
    // Use reqwest (blocking)
    var response: string = reqwest.blocking.get("https://api.example.com")
        .text();
}

---

Type Mapping

Automatic Type Conversion

Rust Type	naml Type
i64, i32, isize	int
u64, u32, usize	uint
f64, f32	float
bool	bool
String, &str	string
Vec<u8>, &[u8]	bytes
Vec<T>	[T]
Option<T>	option<T>
HashMap<K, V>	map<K, V>
Result<T, E>	Converts to throws/catch

Struct Mapping

// Rust crate defines:
pub struct User {
    pub name: String,
    pub age: i64,
}

// naml can use directly:
use crate::mylib::User;

var user: User = User { name: "Alice", age: 30 };
println("Name: {}", user.name);

Result/Error Handling

// Rust function:
pub fn parse_config(path: &str) -> Result<Config, ConfigError>;

// naml sees it as:
// fn parse_config(path: string) -> Config throws ConfigError;

var config: Config = parse_config("config.toml") catch e {
    println("Error: {}", e.message());
} ?? default_config();

---

Architecture

Package Manager Flow

naml build
    │
    ├─→ Parse naml.toml
    │
    ├─→ Resolve dependencies
    │   ├─→ naml packages (naml registry)
    │   └─→ Rust crates (crates.io)
    │
    ├─→ Download and cache
    │   └─→ ~/.naml/cache/crates/
    │
    ├─→ Generate bindings
    │   └─→ Analyze Rust crate public API
    │   └─→ Generate naml type declarations
    │   └─→ Generate FFI glue code
    │
    ├─→ Build Rust dependencies
    │   └─→ Compile to static library
    │
    └─→ Compile naml code
        └─→ Link against Rust libraries

Directory Structure

~/.naml/
├── cache/
│   ├── crates/           # Downloaded Rust crates
│   │   ├── serde-1.0.0/
│   │   └── reqwest-0.11.0/
│   └── naml/             # Downloaded naml packages
├── bin/                  # Installed binaries
└── registry/             # Package index cache

Project Structure

my-project/
├── naml.toml
├── naml.lock             # Lockfile for reproducible builds
├── src/
│   └── main.naml
├── target/
│   ├── debug/
│   └── release/
└── .naml/
    └── bindings/         # Generated Rust bindings
        ├── serde_json.naml
        └── reqwest.naml

---

CLI Commands

# Initialize new project
naml init

# Add dependencies
naml add serde_json              # Add naml package
naml add --crate serde           # Add Rust crate
naml add --crate reqwest --features json,blocking

# Remove dependencies
naml remove serde_json

# Update dependencies
naml update                      # Update all
naml update serde               # Update specific

# Build project
naml build                       # Debug build
naml build --release            # Release build

# Run project
naml run                        # Run with JIT
naml run --release              # Run release build

# Show dependency tree
naml tree

---

Implementation Phases

Phase 1: Basic Package Manager

• Parse naml.toml with [crates] section
• Resolve crate versions from crates.io
• Download and cache crates
• Generate naml.lock file

Phase 2: Binding Generation

• Parse Rust crate public API (using syn/quote)
• Generate naml type declarations
• Generate FFI glue code
• Handle basic types (primitives, String, Vec, Option)

Phase 3: Compilation Integration

• Build Rust crates as static libraries
• Link naml code against Rust libraries
• Runtime interop for heap types

Phase 4: Advanced Features

• Handle complex Rust types (traits, generics)
• Async Rust (tokio) integration
• Rust macro expansion
• naml package registry

---

Challenges & Considerations

1. Rust generics - May need to monomorphize at binding generation
2. Lifetimes - naml doesn't have lifetimes, need to handle ownership
3. Traits - Map to naml interfaces where possible
4. Async - Bridge tokio/async-std with naml's spawn
5. Build times - Rust compilation can be slow, need good caching

---

Prior Art

• PyO3 - Python-Rust interop
• wasm-bindgen - JS-Rust interop for WASM
• cbindgen - Generate C headers from Rust
• uniffi - Mozilla's cross-language bindings

---

Questions to Resolve

1. Should we generate bindings at build time or ahead of time?
2. How to handle Rust crates that use proc macros?
3. Should naml have its own registry or use crates.io directly?
4. How to handle crate features elegantly in naml syntax?