Sheaf is a functional language for machine learning that combines the expressiveness of Lisp with the performance of modern ML compilers.

Highlights

• Clojure for tensors: homoiconicity, immutability, minimalist syntax, threading macros
• REPL-driven development: immediate tensor shapes and dtypes, inline documentation, and environment inspection
• GPU-first: compiles to StableHLO and runs on CUDA, Metal, Vulkan, and CPUs through IREE
• Reverse-mode autodiff: ahead-of-time automatic differentiation
• JIT compilation: pure functions are automatically JIT-compiled
• Single native binary: runs on Linux (x86_64, aarch64) and macOS (Apple Silicon) with no runtime dependencies
• LLM-native: built-in context generation for AI assistants

Sample code

Define a model, differentiate, get gradients:

sheaf> (def x (tensor [[1.0 0.0] [0.0 1.0]]))    ;; inputs
sheaf> (def y (tensor [[1.0 1.0 0.0 0.0]         ;; targets
                       [0.0 0.0 1.0 1.0]]))
sheaf> (def W (zeros '[2 4]))                    ;; weights

sheaf> ((value-and-grad
           (fn [W] (mse-loss (@ x W) y)))        ;; loss + gradients
         W)
=> [0.5 [[-0.25 -0.25 0.0 0.0]
        [0.0 0.0 -0.25 -0.25]]]

Transformer block with residual connections:

(defn transformer-block [x layer-p config]
  (as-> x h
    ;; 1. Self-Attention
    (-> h
        (layer-norm (get layer-p :ln1) 2)
        (multi-head-attention layer-p config)
        (first) ;; Get the attention output, ignore weights
        (+ h))  ;; Residual 1

    ;; 2. MLP
    (-> h
        (layer-norm (get layer-p :ln2) 2)
        (mlp (get layer-p :mlp))
        (+ h)))) ;; Residual 2

Use macros to derive three compiled graphs from the same layer list:

(defmacro defresidual [name args & layers] ...)   ;; generates residual graph
(defmacro definspect  [name args & layers] ...)   ;; generates monitoring graph

(defmodel    net         (x) [linear :h1 gelu] [linear :h2 gelu])
(defresidual res-net     (x) [linear :h1 gelu] [linear :h2 gelu])
(definspect  inspect-net (x) [linear :h1 gelu] [linear :h2 gelu])

Check out the examples for more code samples.

Install

• Download the binary tarball from https://github.com/sheaf-lang/sheaf/releases
• Download the examples: https://github.com/sheaf-lang/sheaf/releases/download/v2.1.0/sheaf-examples.tar.gz

Links

• Website
• Quick Start
• Documentation
• Reference
• Examples