All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Upgraded to ORT v1.26.0
- Upgraded to ORT GenAI v0.14.0
- Do not specify go patch version in go.mod
- Fixes for release docker image build and push
- Fix panic on double-close (thanks @sananguliyev)
- Upgraded to ORT v1.25.1
- Test structure improvements
- Documentation and examples improvements (thanks @kepae)
- Upgraded to ORT v1.25.0
- Breaking: Added context.Context to many functions for graceful cancellation
- Experimental: Added WithGenerativeEngine to ORT, to enable concurrent inferencing and intelligent batching
- Added locks to protect against concurrent model and pipeline initialisations
- Added 2-bit bucketing to XLA backend for reduced padding overhead
- Upgraded to GoMLX v0.27.3
- Upgraded to ORT v1.24.4
- Upgraded to ORT GenAI v0.13.1
- Added QuestionAnsweringPipeline
- Added ConstrainedGeneration, Tools and Guidance to TextGenerationPipeline
- Upgraded to GoMLX v0.27.2
- Fixes multiple quantization issues in simple Go backend
- Upgraded to ORT v1.24.4
- Upgraded to ORT GenAI v0.12.2
- Replaced Go Tokenizers with GoMLX version
- Supports Pair Tokenization
- Further reduces dependencies
- Fix: Allow use of SOFTMAX in TextClassificationPipeline
- Breaking: Remove CLI to simplify dependencies (can be reintroduced as a separate library on request)
- Upgraded to GoMLX v0.27.1
- Includes performance improvements, new onnx ops and quantization types for Go and XLA backends
- See here for details
- Upgraded to ORT v1.24.3
- Upgraded to ORT GenAI v0.12.1
- Upgraded to Tokenizers v1.26.0
- Added WithProfiling and WithOptimizedModelFilePath options
- bugfix: correct wrong input token creation in feature extraction for models with eos tokens.
- Upgraded to ORT GenAI v0.12.0
- Upgraded to ORT v1.24.2
- Upgraded to Go v1.26.0
- Upgraded tokenizers to v1.25.0
- Added WithLogSeverityLevel, WithEnvLoggingLevel, WithGraphOptimizationLevel and WithExtraExecutionProvider options
- Separated TensorRT and NvTensorRTRTX
- Memory usage and bugfixes to generative pipelines
First release of 2026!
- Added TabularPipeline to run onnx exports of classical ML models (regression or classification) from, e.g., sklearn
- Added multimodal support for text generation pipeline (kudos to @ajroetker)
- Memory mapped loading of onnx models on native fs to reduce memory usage
- Explicit XLA compilation cache control via WithGoMLXBatchBuckets and WithGoMLXSequenceBuckets.
- Finalize device memory earlier in XLA sessions, reducing memory pressure when using TPU/GPU.
- Tokenizer can handle/ignore additional image tensor types.
- Allow 4D dimensions for multimodal featureExtraction pipelines.
- Allow multiple ONNX files/models to load from the same model directory.
- Restore support for GLIBC 2.34 in XLA
- Also disable XLA dependency autoinstall in training mode (will implement global C dependency autoinstall in near future!)
Thanks @ajroetker for your contributions!
- Integrated ONNX Runtime GenAI backend for significantly faster generative inference and broad model support.
- Added ObjectDetection pipeline
- Added WithTPU option to NewXLASession
- FeatureExtractionPipeline now supports image inputs, enabling vision models like CLIP
- Updated Onnx Runtime to 1.23.2, and GoMLX to 0.26.0
- ORT Gen AI has strong requirements on the name of the base ORT library. It should not be renamed from the release zip (e.g. libonnxruntime.so)
- WithOnnxLibraryPath should now be the folder contining the ORT library. The library name is now inferred from the current operating system.
- XLA now uses go-xla to manage PJRT dependencies, see our Dockerfile for details
- Model loading path could potentially duplicate paths (thanks @ajroetker)
- breaking: GetStatistics on a session returns a map of pipeline name to statistics object
- breaking: GetStatistics now returns a Statistics struct for the pipelines rather than a list of strings
- breaking: pipelineBackends has been renamed to backends
- update of onnxruntime_go, goMLX, gopjrt
- support splitIntoWords for tokenClassificationPipeline
- Support models that do not utilize attention masks in FeatureExtractionPipeline
- Bump onnx-gomlx to v0.3.2 for expanded model support in Go sessions.
- Update Go, Tokenizers, OnnxRuntime and GoMLX dependencies
- Compatibility with NHWC and NCHW formats in Image Classification Pipeline
- Update to new goMLX project structure (0.24.0+)
- remove the dependency on python when installing goMLX libraries
- Update Go, Tokenizers, and GoMLX dependencies
- XLA now uses StableHLO
- XLA CUDA backend is now CUDA 13
- Fix: use right tokenization and token type IDs for Bert-style sentence pair in cross encoder
- Performance improvement: do not pad inputs to POW2 when using simplego backend
- Apply small input performance fix to goMLX backend
- Fix performance regression for small inputs
- The new TextGenerationPipeline allows you to run generative models such as Gemma and Phi in golang. Kudos to Riley Oh for getting this one over the line!
- Currently only implemented for the ORT backend. Implementations for XLA and GO backend coming soon!
- See the documentation for how to get started
- The CrossEncoderPipeline implements the equivalent of sentence transformers' Cross Encoder. Kudos to Fábio Correia for providing the initial implementation
- The ImageClassificationPipeline implements the equivalent of Hugging Face's Image Classification pipeline
- The training session to fine-tune embeddings now accept TrainEval and Eval datasets to compute in-sample and test statistics
- The training session now implements early stopping based on the loss on the Eval dataset. Early stopping is evaluated at the end of each training epoch.
- The training session now accepts a layer freezing configuration to specify which layers of the transformer will be frozen during fine-tuning
- The go tokenizer now supports unigram tokenization
- Updated go to 1.25.0
- Upgraded GoMLX to 0.22.1
- Upgraded to latest Rust and Go tokenizers
- Generalised output tensor types, added int64 support (preparation for text generation pipeline)
- Dependency updates and go 1.24.5
- Tokenizers obey max_position_embeddings, closing issue #73
- Provided default ORT path in darwin/mac
- Dependency updates and go 1.24.4
- Run inference and fine-tuning, all from native Go!
- Added
NewGoSession()backend, enabling Hugot to run in pure Go environments without C dependencies - Implements our most requested community feature using SimpleGo from GoMLX
- Optimized for simpler workloads and environments where cgo is restricted
- Works best with smaller models like all-MiniLM-L6-v2
- Performance sweet spot: batches of approximately 32 inputs per call
- The Go backend currently performs 5-20x slower than C backends
- For performance-critical applications, we recommend using C backends (XLA or ORT)
- Significant speed improvements expected with the introduction of SIMD in Go 1.25
- The Go backend and tokenizer currently lack support for certain operators
- If you encounter compatibility issues, please open an issue
- As a temporary workaround, use a C backend until the issue is resolved
- Build Tag Requirements Updated:
- For
NewORTSession(): Add-tags "ORT"or-tags "ALL"to your build arguments - For
NewXLASession(): Add-tags "XLA"or-tags "ALL"to your build arguments
- For
- SimpleGo backend and SugarMe tokenizers for running Hugot without C dependencies
- Upgraded CUDA libraries to 12.9
- Upgraded GoMLX for performance improvements and bugfixes.