Search results

Propagate actual model dtype to compressed cache instead of hardcoded F32

bugSomething isn't workingSomething isn't working

mistral.rs

performance

Status: Open.

#47 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Per-layer locks for compressed cache instead of single global Mutex

mistral.rs

performance

Status: Open.

#46 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

DoS protection: memory limit for compressed KV cache in server mode

bugSomething isn't workingSomething isn't working

mistral.rs

Status: Open.

#45 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Mistral.rs integration safety: input validation, guards, and error handling

bugSomething isn't workingSomething isn't working

mistral.rs

Status: Open.

#44 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Multi-Token Prediction (MTP) support in mistral.rs

enhancementNew feature or requestNew feature or request

mistral.rs

Status: Open.

#41 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Custom Flash Attention with QJL bias correction — investigate TQ3 quality rescue

research

Status: Open.

#40 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

GPU CI — ephemeral RunPod runner for CUDA tests

infrastructure

Status: Open.

#39 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

SIMD AVX2/NEON intrinsics for WHT, bit-packing, and codebook lookup

performance

Status: Open.

#38 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Norm correction — scale by original_norm / reconstruction_norm

quality

Status: Open.

#37 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Sparse V dequantization — only dequant where attention weight > threshold

performance

Status: Open.

#36 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Reuse fused attention output buffers instead of fresh allocation

performance

Status: Open.

#35 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026

Reuse CUDA quant output buffers instead of fresh allocation

performance

Status: Open.

#34 In SaschaOnTour/turboquant;

· SaschaOnTour opened on Apr 11, 2026