Skip to content

Issues: huggingface/text-generation-inference

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

96 Open 1,220 Closed
96 Open 1,220 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

TGI fails to run Llama 3.1-405B with AMD 8 X MI300x
#2613 opened Oct 5, 2024 by Bihan
4 tasks
huggingface_hub.errors.GenerationError: Request failed during generation: Server error:
#2608 opened Oct 4, 2024 by ivanhe123
2 of 4 tasks
Server error: transport error
#2593 opened Oct 1, 2024 by ismael-dm
2 of 4 tasks
Remove max_stop_sequences by default
#2584 opened Sep 29, 2024 by sestinj
3 of 4 tasks
How to turn on the KV cache when serve a model?
#2583 opened Sep 28, 2024 by hahmad2008
4 tasks
OutOfMemory error running Meta-Llama-3.1-405B-Instruct-fp8 on 8xH100
#2572 opened Sep 26, 2024 by ad01bl
1 of 4 tasks
2
Deploy error for Llama-3.2-vision-11B: "Sharded is not supported for AutoModel"
#2571 opened Sep 26, 2024 by xuan1905
1 of 4 tasks
2
Question: What is preferred way to cite TGI/repo? Didnt see a citation file.
#2569 opened Sep 26, 2024 by elegantmoose
Passing an image_url to a text-only model should fail explicitly
#2565 opened Sep 25, 2024 by Wauplin
4 tasks
Image for arm64 (Macbook Pro)
#2560 opened Sep 24, 2024 by arsentievalex
1 of 4 tasks
Inconsistent Behavior with Multi-LoRA Deployment
#2559 opened Sep 24, 2024 by charlatan-101
2 of 4 tasks
tgi server :: tool_choice="auto" behaves like tool_choice="required" from OpenAI spec
#2549 opened Sep 23, 2024 by mottoslo
2 of 4 tasks
@drbh
Host/CPU memory usage for prefix cache
#2544 opened Sep 21, 2024 by josephrocca
Error: Backend(Warmup(Generation("Hidden size mismatch"))) when launch Mixtral-8x22B-v0.1
#2543 opened Sep 21, 2024 by alexhegit
1 of 4 tasks
Docker container for version 2.3.0 CUDA detection broken
#2542 opened Sep 20, 2024 by JoeGonzalez0886
1 of 4 tasks
1
How to serve local models with python package (not docker)
#2541 opened Sep 20, 2024 by hahmad2008
4 tasks
2
Support for returning a CompletionUsage object when streaming=True
#2531 opened Sep 17, 2024 by andrewrreed
xpu/cpu: docker images referenced in documentation do not exist
#2530 opened Sep 17, 2024 by dvrogozh
3
* HTTP 1.0, assume close after body < HTTP/1.0 503 Service Unavailable
#2526 opened Sep 17, 2024 by aditivw
4 tasks
Add response_format input parameter to v1/chat/completions endpoint
#2523 opened Sep 16, 2024 by ktrapeznikov
tgi server launch fails with latest-rocm docker image.
#2522 opened Sep 13, 2024 by gurpreet-dhami
3 of 4 tasks
RuntimeError: weight model.embed_tokens.weight does not exist
#2509 opened Sep 11, 2024 by jayus71
3 of 4 tasks
Multi-LORA feature question-2
#2506 opened Sep 9, 2024 by imran3180
Multi-LORA feature question
#2505 opened Sep 9, 2024 by imran3180
Cant install on Ubuntu 22.04 with Cuda 11.8
#2504 opened Sep 9, 2024 by Oxi84
1 of 4 tasks
2
ProTip! Mix and match filters to narrow down what you’re looking for.