Open
Description
🐛 Describe the bug
Kserve's TorchserveModel load doesn't take into consideration the fact that torchserve has auth enabled by default.
Error logs
INFO:kserve:Copying contents of /mnt/models/model-store to local
INFO:root:Loading model .. 1 of 10 tries..
INFO:root:The model layoutlm-pytorch-model is not ready
INFO:root:Sleep 30 seconds for load layoutlm-pytorch-model..
...
...
...
INFO:root:Loading model .. 2 of 10 tries..
2024-08-29T10:16:01,232 [INFO ] epollEventLoopGroup-3-1 ACCESS_LOG - /127.0.0.1:50126 "GET /models/model?customized=false HTTP/1.1" 400 4
Installation instructions
Using docker image pytorch/torchserve-kfs:0.11.1
to serve a model
Model Packaging
Model was packaged using torch-model-archiver
and the same mar file would work on the older pytorch/torchserve-kfs:0.8.0
config.properties
No response
Versions
Copied from within docker image pytorch/torchserve-kfs:0.11.1
------------------------------------------------------------------------------------------
Environment headers
------------------------------------------------------------------------------------------
Torchserve branch:
torchserve==0.11.1
torch-model-archiver==0.11.1
Python version: 3.9 (64-bit runtime)
Python executable: /home/venv/bin/python
Versions of relevant python libraries:
captum==0.6.0
numpy==1.24.3
pillow==10.3.0
psutil==5.9.8
requests==2.32.0
requests-oauthlib==2.0.0
torch==2.3.0+cpu
torch-model-archiver==0.11.1
torch-workflow-archiver==0.2.14
torchaudio==2.3.0+cpu
torchserve==0.11.1
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
transformers==4.42.4
wheel==0.42.0
torch==2.3.0+cpu
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
torchaudio==2.3.0+cpu
Java Version:
OS: Ubuntu 20.04.6 LTS
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: N/A
CMake version: N/A
Environment:
library_path (LD_/DYLD_):
Repro instructions
- create model mar file
- create
config.properties
file and DO NOT disable auth - run the image
pytorch/torchserve-kfs:0.11.1
- should fail to load as the management request doesn't include auth token
Possible Solution
Read key_file.json
and use the token in the request
Metadata
Metadata
Assignees
Labels
No labels