pure-go-sgd

A bare-metal implementation of Stochastic Gradient Descent and Backpropagation written in pure Go with a single dependency (gonum for matrix primitives).

Designed for low-latency environments (e.g., high-frequency financial data, edge inference) where Python/CGO overhead or heavy ML framework dependencies introduce unacceptable latency.

Overview

This engine is designed for high-frequency financial trading systems and embedded inference where Python's GIL and runtime overhead are unacceptable. It compiles to a single, statically linked binary with microsecond-latency inference capabilities.

It provides a raw, from-scratch implementation of:

• Linear Algebra: Direct use of gonum/mat.Dense for optimized matrix operations.
• Backpropagation: Explicit chain rule implementation for full gradient computation.
• Optimizers: Adam (Adaptive Moment Estimation) and standard SGD.
• Activations: ReLU, Sigmoid, Tanh, and Softmax (with numerical stability fixes).

Why This Exists

Most Go ML libraries are wrappers around C/C++ (TensorFlow, PyTorch) or Python bridges. This introduces:

1. CGO Overhead: Context switching costs that kill HFT latency.
2. Deployment Hell: Managing .so/.dll dependencies.
3. Opaque Logic: You can't debug the kernel.

This engine solves these issues. Pure Go. Readable source. Debuggable via dlv. One dependency, zero CGO.

Installation

go get github.com/mlhher/pure-go-sgd

Quick Start (XOR Example)

See cmd/example/main.go for a runnable proof.

package main

import (
    "github.com/mlhher/pure-go-sgd/pkg/neural"
    "gonum.org/v1/gonum/mat"
)

func main() {
    // 1. Define Architecture
    // 2 Inputs -> 4 Hidden (Tanh) -> 1 Output (Sigmoid)
    net := neural.NewNetwork(2, []int{4, 1}, 
        neural.NewActivators([]neural.ActivationIdentifier{neural.TANH, neural.SIGMOID}),
        0.01, 0.0,
    )

    // 2. Initialize Adam Optimizer
    optimizer := neural.NewAdamOptimizer(0.01)
    optimizer.Initialize(net.GetLayers())

    // 3. Training Loop
    inputs := mat.NewDense(4, 2, []float64{0,0, 0,1, 1,0, 1,1})
    targets := mat.NewDense(4, 1, []float64{0, 1, 1, 0})

    for i := 0; i < 2000; i++ {
        wGrads, bGrads := net.ComputeGradients(inputs, targets)
        optimizer.Update(wGrads, bGrads, net)
    }

    // 4. Inference
    output := net.Predict(inputs, false)
}

Core Mathematics

The engine implements the standard backpropagation algorithm for weight gradients:

$$ \frac{\partial C}{\partial w^l} = \delta^l (a^{l-1})^T $$

Loss Function (MSE)

$$ C(w,b) = \frac{1}{2n} \sum_x || y(x) - a^L(x) ||^2 $$

Backpropagation

The error $\delta^l$ for layer $l$ is computed recursively:

$$ \delta^l = ((w^{l+1})^T \delta^{l+1}) \odot \sigma'(z^l) $$

Gradients are explicitly computed and applied via TrainBatch (SGD) or AdamOptimizer.

Verified on MNIST

Out-of-the-box MNIST classification with a 784→64→64→10 MLP and Adam:

Epoch 0:  Accuracy = 94.42%
Epoch 5:  Accuracy = 96.91%
Epoch 19: Accuracy = 97.73%

94% accuracy after a single epoch. No hyperparameter tuning, no preprocessing beyond pixel / 255.

Autoencoder Showcase

The autoencoder (784→256→128→32→128→256→784) converges to MSE 0.016. It generates high-fidelity reconstructions:

Original	Reconstructed

Note: The images above are for illustration. Run the test below to generate your own local comparisons.

Run it yourself:

go test ./pkg/neural/ -run=TestMNIST -v -timeout 300s

Performance

Benchmarked on a consumer-grade processor (Intel i5):

Benchmark	Network	Time
Forward pass (small)	2→4→1	862 ns
Forward pass (MNIST)	784→128→64→10	28.7 µs
Backprop (XOR batch)	2→4→1, 4 samples	5.18 µs

Sub-microsecond inference for small networks. Under 30µs for real-world architectures.

Reproduce:

go test ./pkg/neural/ -bench=. -benchtime=3s -short -run=^$

• Concurrency: Thread-safe prediction (Predict). Multi-threaded gradient accumulation via Train/TrainAdam.
• Memory: Zero-allocation hot paths (reused matrices where possible).

Licensing

This project is licensed under the AGPLv3.

It is free and open-source for any project that also adopts the AGPLv3.

Commercial Use: If you intend to use this library internally within a proprietary, closed-source backend (e.g., proprietary trading algorithms, closed SaaS risk models) where you cannot comply with the AGPLv3 open-source requirements, a commercial license is required.

Please contact the author for a dual-license commercial agreement.