usls is a cross-platform Rust library powered by ONNX Runtime for efficient inference of SOTA vision and multi-modal models(typically under 1B parameters).
# CPU
cargo run -r --example yolo -- --task detect --ver 8 --scale n --dtype fp16 # q8, q4, q4f16
# NVIDIA CUDA
cargo run -r -F cuda --example yolo -- --device cuda:0 # YOLOv8-n detect by default
# NVIDIA TensorRT
cargo run -r -F tensorrt --example yolo -- --device tensorrt:0
# Apple Silicon CoreML
cargo run -r -F coreml --example yolo -- --device coreml
# Intel OpenVINO
cargo run -r -F openvino -F ort-load-dynamic --example yolo -- --device openvino:CPU
# And other EPs...
Add the following to your Cargo.toml
:
[dependencies]
# Recommended: Use GitHub version
usls = { git = "https://github.com/jamjamjon/usls", features = [ "cuda" ] }
# Alternative: Use crates.io version
usls = "latest-version"
Click to expand
Model | Task / Description | Example |
---|---|---|
BEiT | Image Classification | demo |
ConvNeXt | Image Classification | demo |
FastViT | Image Classification | demo |
MobileOne | Image Classification | demo |
DeiT | Image Classification | demo |
DINOv2 | Vision Embedding | demo |
YOLOv5 | Image Classification Object Detection Instance Segmentation |
demo |
YOLOv6 | Object Detection | demo |
YOLOv7 | Object Detection | demo |
YOLOv8 YOLO11 |
Object Detection Instance Segmentation Image Classification Oriented Object Detection Keypoint Detection |
demo |
YOLOv9 | Object Detection | demo |
YOLOv10 | Object Detection | demo |
YOLOv12 | Object Detection | demo |
RT-DETR | Object Detection | demo |
RF-DETR | Object Detection | demo |
PP-PicoDet | Object Detection | demo |
DocLayout-YOLO | Object Detection | demo |
D-FINE | Object Detection | demo |
DEIM | Object Detection | demo |
RTMPose | Keypoint Detection | demo |
DWPose | Keypoint Detection | demo |
RTMW | Keypoint Detection | demo |
RTMO | Keypoint Detection | demo |
SAM | Segment Anything | demo |
SAM2 | Segment Anything | demo |
MobileSAM | Segment Anything | demo |
EdgeSAM | Segment Anything | demo |
SAM-HQ | Segment Anything | demo |
FastSAM | Instance Segmentation | demo |
YOLO-World | Open-Set Detection With Language | demo |
GroundingDINO | Open-Set Detection With Language | demo |
CLIP | Vision-Language Embedding | demo |
jina-clip-v1 | Vision-Language Embedding | demo |
jina-clip-v2 | Vision-Language Embedding | demo |
mobileclip | Vision-Language Embedding | demo |
BLIP | Image Captioning | demo |
DB(PaddleOCR-Det) | Text Detection | demo |
FAST | Text Detection | demo |
LinkNet | Text Detection | demo |
SVTR(PaddleOCR-Rec) | Text Recognition | demo |
SLANet | Tabel Recognition | demo |
TrOCR | Text Recognition | demo |
YOLOPv2 | Panoptic Driving Perception | demo |
DepthAnything v1 DepthAnything v2 |
Monocular Depth Estimation | demo |
DepthPro | Monocular Depth Estimation | demo |
MODNet | Image Matting | demo |
Sapiens | Foundation for Human Vision Models | demo |
Florence2 | a Variety of Vision Tasks | demo |
Moondream2 | Open-Set Object Detection Open-Set Keypoints Detection Image Caption Visual Question Answering |
demo |
OWLv2 | Open-Set Object Detection | demo |
SmolVLM(256M, 500M) | Visual Question Answering | demo |
RMBG(1.4, 2.0) | Image Segmentation Background Removal |
demo |
BEN2 | Image Segmentation Background Removal |
demo |
MediaPipe: Selfie-segmentation | Image Segmentation | demo |
Swin2SR | Image Super-Resolution and Restoration | demo |
APISR | Real-World Anime Super-Resolution | demo |
ort-download-binaries
(default): Automatically downloads prebuilt ONNXRuntime binaries for supported platformsort-load-dynamic
: Dynamic linking to ONNXRuntime libraries (Guide)video
: Enable video stream reading and writing (via video-rs and minifb)cuda
: NVIDIA CUDA GPU acceleration supporttensorrt
: NVIDIA TensorRT optimization for inference accelerationcoreml
: Apple CoreML acceleration for macOS/iOS devicesopenvino
: Intel OpenVINO toolkit for CPU/GPU/VPU accelerationonednn
: Intel oneDNN (formerly MKL-DNN) for CPU optimizationdirectml
: Microsoft DirectML for Windows GPU accelerationxnnpack
: Google XNNPACK for mobile and edge device optimizationrocm
: AMD ROCm platform for GPU accelerationcann
: Huawei CANN (Compute Architecture for Neural Networks) supportrknpu
: Rockchip NPU accelerationacl
: Arm Compute Library for Arm processorsnnapi
: Android Neural Networks API supportarmnn
: Arm NN inference enginetvm
: Apache TVM tensor compiler stackqnn
: Qualcomm Neural Network SDKmigraphx
: AMD MIGraphX for GPU accelerationvitis
: Xilinx Vitis AI for FPGA accelerationazure
: Azure Machine Learning integration
See issues or open a new discussion.
Contributions are welcome! If you have suggestions, bug reports, or want to add new features or models, feel free to open an issue or submit a pull request.
This project is licensed under LICENSE.