docs: add AWS (EC2/SageMaker) deployment + benchmarking guide#3352
docs: add AWS (EC2/SageMaker) deployment + benchmarking guide#3352alvarobartt merged 3 commits intohuggingface:mainfrom
Conversation
- Add an AWS deployment tutorial for EC2 + SageMaker - Fix SageMaker example indentation and link to the new guide - Add the new guide to the docs toctree
KOKOSde
force-pushed
the
docs/aws-deployment-benchmarks
branch
from
8d9a09d to
ae18647
Compare
|
Quick ping on this one. Happy to update anything if you want changes. |
|
lgtm but pinging @alvarobartt for a quick review |
|
This adds a guide for deployment based on Sagemaker SDK v2 that has been deprecated by v3, and TGI itself has been deprecated, so I see little value in merging this PR. |
There was a problem hiding this comment.
Thanks for your contribution @KOKOSde!
As my colleague @tengomucho mentioned, the AWS SageMaker SDK is now on v3.0 which means that the former v2.0 (used in this example) is deprecated; and on top Text Generation Inference (TGI) is on maintenance mode at the moment, meaning we won't be actively contributing to it anymore in favour of contributing to other Transformers-based inference engines as vLLM or SGLang! 🤗
Regardless of that, the PR looks good to me and I'd be happy to merge as it might still have value, but I'd add a couple things here and there as per the review below!
P.S. Apologies for missing this earlier and getting back to you just now 🙏🏻
4 participants
Docs: add an AWS deployment + benchmarking guide for TGI.
Docs-only change.