Skip to content

asr whisper model in torchserve  #2578

Open
@navaneethakarli

Description

@navaneethakarli

🐛 Describe the bug

I am trying to surve ASR whisper model in torchserve but i am facing worker issue

Error logs

main org.pytorch.serve.servingsdk.impl.PluginsManager - Loading snapshot serializer plugin...
2023-09-08T11:47:27,120 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: torchserve_stt/model_store/speechtotext.mar
2023-09-08T11:47:44,718 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model speechtotext
2023-09-08T11:47:44,718 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model speechtotext
2023-09-08T11:47:44,718 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model speechtotext loaded.
2023-09-08T11:47:44,719 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: speechtotext, count: 1
2023-09-08T11:47:44,727 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:44,729 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2023-09-08T11:47:44,836 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://127.0.0.1:8080
2023-09-08T11:47:44,836 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel.
2023-09-08T11:47:44,837 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://127.0.0.1:8081
2023-09-08T11:47:44,837 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2023-09-08T11:47:44,838 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
Model server started.
2023-09-08T11:47:45,044 [WARN ] pool-3-thread-1 org.pytorch.serve.metrics.MetricCollector - worker pid is not available yet.
2023-09-08T11:47:45,631 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,633 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:64.30083847045898|#Level:Host|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,633 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:1803.2123374938965|#Level:Host|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,633 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:96.6|#Level:Host|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUtilization.Percent:0.02629618267081562|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUsed.Megabytes:12.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - GPUUtilization.Percent:0.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:53097.89453125|#Level:Host|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,634 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:10246.71875|#Level:Host|#hostname:black,timestamp:1694153865
2023-09-08T11:47:45,635 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:17.4|#Level:Host|#hostname:black,timestamp:1694153865
2023-09-08T11:47:46,079 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127192
2023-09-08T11:47:46,084 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:46,087 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:46,087 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127192
2023-09-08T11:47:46,087 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:46,088 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:46,088 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change null -> WORKER_STARTED
2023-09-08T11:47:46,093 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:46,100 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:46,103 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD to backend at: 1694153866103
2023-09-08T11:47:46,135 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - model_name: speechtotext, batchSize: 1
2023-09-08T11:47:46,682 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Backend worker process died.
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 253, in
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - worker.run_server()
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 221, in run_server
2023-09-08T11:47:46,683 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket)
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 184, in handle_connection
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg)
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py", line 131, in load_model
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - service = model_loader.load(
2023-09-08T11:47:46,684 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_loader.py", line 135, in load
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - initialize_fn(service.context)
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/tmp/models/47afe52012fe4966ae64cb2af806c171/handler.py", line 85, in initialize
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - self.processor = WhisperProcessor.from_pretrained(model_dir)
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/processing_utils.py", line 215, in from_pretrained
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, **kwargs)
2023-09-08T11:47:46,685 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/processing_utils.py", line 259, in _get_arguments_from_pretrained
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - args.append(attribute_class.from_pretrained(pretrained_model_name_or_path, **kwargs))
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1841, in from_pretrained
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - return cls._from_pretrained(
2023-09-08T11:47:46,686 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2004, in _from_pretrained
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - tokenizer = cls(*init_inputs, **init_kwargs)
2023-09-08T11:47:46,686 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - File "/home/batman/navaneetha/my_name/lib/python3.8/site-packages/transformers/models/whisper/tokenization_whisper.py", line 294, in init
2023-09-08T11:47:46,686 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:46,687 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - with open(merges_file, encoding="utf-8") as merges_handle:
2023-09-08T11:47:46,687 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - TypeError: expected str, bytes or os.PathLike object, not NoneType
2023-09-08T11:47:46,687 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:213) [model-server.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:46,698 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: speechtotext, error: Worker died.
2023-09-08T11:47:46,698 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:46,698 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery start timestamp: 1694153866698
2023-09-08T11:47:46,698 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:46,699 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:46,699 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 1 seconds.
2023-09-08T11:47:46,713 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:46,713 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:47,700 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:48,978 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127281
2023-09-08T11:47:48,979 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127281
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:48,987 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:48,987 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:48,988 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:48,989 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:48,989 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:48,990 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:48,990 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:48,991 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:48,991 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:48,991 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:48,991 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:48,991 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 1 seconds.
2023-09-08T11:47:49,003 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:49,003 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:49,992 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:51,274 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127305
2023-09-08T11:47:51,275 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127305
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:51,283 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:51,283 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:51,284 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:51,285 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:51,285 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:51,285 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:51,286 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:51,286 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:51,286 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:51,286 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:51,286 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:51,287 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 2 seconds.
2023-09-08T11:47:51,300 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:51,300 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:53,287 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:54,567 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127341
2023-09-08T11:47:54,567 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127341
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:54,576 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:54,576 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:54,578 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:54,578 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:54,578 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:54,578 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:54,579 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:54,579 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:54,580 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:54,580 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:54,580 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 3 seconds.
2023-09-08T11:47:54,593 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:54,593 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:57,581 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:47:58,848 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127382
2023-09-08T11:47:58,849 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127382
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:47:58,857 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:47:58,857 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:47:58,858 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:47:58,859 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:47:58,859 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:47:58,861 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:47:58,861 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:47:58,862 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:47:58,862 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:47:58,862 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:47:58,862 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:58,862 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 5 seconds.
2023-09-08T11:47:58,873 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:47:58,873 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:03,863 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:05,216 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127418
2023-09-08T11:48:05,216 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:05,224 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127418
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:05,225 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:05,225 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:05,227 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:05,227 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:05,227 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:05,227 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:05,228 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:05,228 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:05,228 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:05,228 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:05,229 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 8 seconds.
2023-09-08T11:48:05,240 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:05,240 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:13,229 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:14,608 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127471
2023-09-08T11:48:14,609 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127471
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:14,617 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:14,617 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:14,619 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:14,619 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:14,619 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:14,619 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:14,620 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:14,620 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:14,620 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:14,620 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:14,620 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 13 seconds.
2023-09-08T11:48:14,631 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:14,631 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:27,621 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:28,901 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127510
2023-09-08T11:48:28,901 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:28,909 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127510
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:28,910 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:28,910 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:28,913 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:28,913 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:28,913 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:28,913 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:28,913 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:28,913 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:28,914 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:28,914 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:28,914 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 21 seconds.
2023-09-08T11:48:28,924 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:28,924 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:45,629 [INFO ] pool-3-thread-2 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - DiskAvailable.Gigabytes:64.30076217651367|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - DiskUsage.Gigabytes:1803.2124137878418|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - DiskUtilization.Percent:96.6|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,630 [INFO ] pool-3-thread-2 TS_METRICS - GPUMemoryUtilization.Percent:0.02629618267081562|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - GPUMemoryUsed.Megabytes:12.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - GPUUtilization.Percent:0.0|#Level:Host,DeviceId:0|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - MemoryAvailable.Megabytes:53042.8046875|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUsed.Megabytes:10301.76953125|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:45,631 [INFO ] pool-3-thread-2 TS_METRICS - MemoryUtilization.Percent:17.5|#Level:Host|#hostname:black,timestamp:1694153925
2023-09-08T11:48:49,914 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:48:51,243 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127607
2023-09-08T11:48:51,243 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127607
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:48:51,252 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:48:51,252 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:48:51,253 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:48:51,256 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:48:51,256 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:48:51,256 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:48:51,256 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:48:51,256 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:48:51,257 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:48:51,257 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:48:51,257 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:51,257 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 34 seconds.
2023-09-08T11:48:51,270 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:48:51,270 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:49:25,258 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/batman/navaneetha/my_name/bin/python, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml]
2023-09-08T11:49:26,828 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=127660
2023-09-08T11:49:26,829 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2023-09-08T11:49:26,837 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Successfully loaded /home/batman/navaneetha/my_name/lib/python3.8/site-packages/ts/configs/metrics.yaml.
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - [PID]127660
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Torch worker started.
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
2023-09-08T11:49:26,838 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STOPPED -> WORKER_STARTED
2023-09-08T11:49:26,838 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2023-09-08T11:49:26,841 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2023-09-08T11:49:26,841 [INFO ] W-9000-speechtotext_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2023-09-08T11:49:26,841 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2023-09-08T11:49:26,841 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:513) ~[?:?]
at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:675) ~[?:?]
at org.pytorch.serve.wlm.Model.pollBatch(Model.java:276) ~[model-server.jar:?]
at org.pytorch.serve.wlm.BatchAggregator.getRequest(BatchAggregator.java:34) ~[model-server.jar:?]
at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:186) [model-server.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2023-09-08T11:49:26,842 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-speechtotext_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2023-09-08T11:49:26,842 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery failed again
2023-09-08T11:49:26,842 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stderr
2023-09-08T11:49:26,842 [WARN ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9000-speechtotext_1.0-stdout
2023-09-08T11:49:26,842 [INFO ] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9000 in 55 seconds.
2023-09-08T11:49:26,853 [INFO ] W-9000-speechtotext_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stdout
2023-09-08T11:49:26,853 [INFO ] W-9000-speechtotext_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9000-speechtotext_1.0-stderr
2023-09-08T11:50:21,844 [DEBUG] W-9000-speechtotext_1.0 org.pytorch.serve.wlm.WorkerLi

Installation instructions

i am not using Docker i am using server

Model Packaing

#!/bin/bash
set -euo pipefail

mkdir -p model_store

Extra files add all files necessary for processor

torch-model-archiver --model-name speechtotext --version 1.0 --serialized-file model/pytorch_model.bin --handler ./handler.py --extra-files "model/config.json,model/special_tokens_map.json,model/tokenizer_config.json,model/vocab.json,model/preprocessor_config.json" -f
mv speechtotext.mar model_store

config.properties

No response

Versions

absl-py==1.4.0
accelerate==0.22.0
aiohttp==3.8.5
aiosignal==1.3.1
aniso8601==9.0.1
ansi2html==1.8.0
arrow==1.2.3
asttokens==2.2.1
async-timeout==4.0.2
attrs==23.1.0
backcall==0.2.0
blinker==1.6.2
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
click==8.1.6
cmake==3.27.0
colorama==0.4.6
coloredlogs==15.0.1
contourpy==1.1.0
cycler==0.11.0
datasets==2.14.2
decorator==5.1.1
dill==0.3.7
enum-compat==0.0.3
evaluate==0.4.0
exceptiongroup==1.1.3
executing==1.2.0
ffmpeg==1.4
ffmpeg-python==0.2.0
filelock==3.12.2
Flask==2.3.2
Flask-RESTful==0.3.10
flatbuffers==23.5.26
fonttools==4.42.1
frozenlist==1.4.0
fsspec==2023.6.0
future==0.18.3
huggingface-hub==0.16.4
humanfriendly==10.0
idna==3.4
importlib-metadata==6.8.0
importlib-resources==6.0.1
iniconfig==2.0.0
install==1.3.5
ipython==8.12.2
itsdangerous==2.1.2
jedi==0.19.0
Jinja2==3.1.2
joblib==1.3.1
kiwisolver==1.4.5
lit==16.0.6
lxml==4.9.3
MarkupSafe==2.1.3
matplotlib==3.7.2
matplotlib-inline==0.1.6
mpmath==1.3.0
multidict==6.0.4
multiprocess==0.70.15
networkx==3.1
nltk==3.8.1
numpy==1.24.4
nvgpu==0.9.0
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.2.10.91
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusparse-cu11==11.7.4.91
nvidia-nccl-cu11==2.14.3
nvidia-nvtx-cu11==11.7.91
onnx==1.14.0
onnxruntime==1.15.1
optimum==1.10.1
packaging==23.1
pandas==2.0.3
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==10.0.0
pluggy==1.2.0
portalocker==2.7.0
prompt-toolkit==3.0.39
protobuf==4.23.4
psutil==5.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==12.0.1
pycparser==2.21
Pygments==2.16.1
pynvml==11.4.1
pyparsing==3.0.9
pyreadline3==3.4.1
pytest==7.4.0
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0.1
regex==2023.6.3
requests==2.31.0
responses==0.18.0
rouge-score==0.1.2
sacrebleu==2.3.1
sacremoses==0.0.53
safetensors==0.3.1
scikit-learn==1.3.0
scipy==1.10.1
sentencepiece==0.1.99
six==1.16.0
sklearn==0.0.post7
soundfile==0.12.1
stack-data==0.6.2
sympy==1.12
tabulate==0.9.0
termcolor==2.3.0
threadpoolctl==3.2.0
tokenizers==0.13.3
tomli==2.0.1
torch==2.0.0
torch-model-archiver==0.8.1
torchaudio==2.0.1
torchdata==0.6.1
torchserve==0.8.1
torchtext==0.15.2
torchvision==0.15.1
tqdm==4.65.0
traitlets==5.9.0
transformers==4.31.0
triton==2.0.0
typing-extensions==4.7.1
tzdata==2023.3
urllib3==2.0.4
wcwidth==0.2.6
Werkzeug==2.3.6
whisper==1.1.10
xxhash==3.3.0
yarl==1.9.2
zipp==3.16.2

Repro instructions

#!/bin/bash
set -euo pipefail

mkdir -p model_store

Extra files add all files necessary for processor

torch-model-archiver --model-name speechtotext --version 1.0 --serialized-file model/pytorch_model.bin --handler ./handler.py --extra-files "model/config.json,model/special_tokens_map.json,model/tokenizer_config.json,model/vocab.json,model/preprocessor_config.json" -f
mv speechtotext.mar model_store

Possible Solution

please give me any solution for this

Metadata

Metadata

Assignees

No one assigned

    Labels

    exampletriagedIssue has been reviewed and triaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions