Search results

How can I reproduce the result of the RULER?

Status: Open.

#75 In mit-han-lab/omniserve;

· josephydu opened on Nov 7, 2025

suggestion on FP8 tensor core in H100

Status: Open.

#74 In mit-han-lab/omniserve;

· lisuying214 opened on Oct 21, 2025

Potential redundant global→shared copies for B in <code>w4a8_per_group</code> GEMM kernel (<code>kernels/csrc/qgemm/w4a8_per_group/gemm_cuda.cu</code>)

Status: Open.

#73 In mit-han-lab/omniserve;

· reoLantern opened on Sep 17, 2025

Can I change the cache block size to another value rather than 64?

Status: Open.

#71 In mit-han-lab/omniserve;

· Dinngger opened on Aug 18, 2025

how to export model into huggingface safetensors format ?

Status: Open.

#70 In mit-han-lab/omniserve;

· artetaout opened on Jun 16, 2025

Release artifacts (models) on Hugging Face

Status: Open.

#68 In mit-han-lab/omniserve;

· NielsRogge opened on Apr 22, 2025

Omniserve seems hard to install, especially in a corporate intranet environment, any Docker image?

Status: Open.

#67 In mit-han-lab/omniserve;

· sisrfeng opened on Apr 11, 2025

Is there any eviction method like H2O to handle the infinite length?

Status: Open.

#66 In mit-han-lab/omniserve;

· zZzZ9zZ9 opened on Mar 29, 2025

Can lserver be used for non quantized versions of initial LLMs?

Status: Open.

#65 In mit-han-lab/omniserve;

· zZzZ9zZ9 opened on Mar 22, 2025

Why use layer norm instead of RMS norm?

Status: Open.

#61 In mit-han-lab/omniserve;

· YudiZh opened on Feb 27, 2025

Weight Protective Quantization Range -119, 119

Status: Open.

#60 In mit-han-lab/omniserve;

· Jzz24 opened on Feb 26, 2025

Why Activation-aware Reorder works?

Status: Open.

#55 In mit-han-lab/omniserve;

· siahuat0727 opened on Feb 7, 2025