e2m_111_api | [2024-06-06 21:30:43 +0800] [1] [CRITICAL] WORKER TIMEOUT (pid:209)
e2m_111_api | 2024-06-06 21:30:43,785 - filelock - DEBUG - Attempting to release lock 140291417052432 on /root/.cache/huggingface/hub/.locks/models--vikp--surya_rec/6a08bbdeda630f5f546f9cbae5a8a1bbed57be61a8f04c2a01c028544e0c90e7.lock
e2m_111_api | 2024-06-06 21:30:43,785 - filelock - DEBUG - Lock 140291417052432 released on /root/.cache/huggingface/hub/.locks/models--vikp--surya_rec/6a08bbdeda630f5f546f9cbae5a8a1bbed57be61a8f04c2a01c028544e0c90e7.lock
0%| | 0/1 [08:20<?, ?it/s]
e2m_111_api | [2024-06-06 21:30:43 +0800] [209] [ERROR] Error handling request /api/v1/convert
e2m_111_api | Traceback (most recent call last):
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/sync.py", line 135, in handle
e2m_111_api | self.handle_request(listener, req, client, addr)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/sync.py", line 178, in handle_request
e2m_111_api | respiter = self.wsgi(environ, resp.start_response)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1498, in __call__
e2m_111_api | return self.wsgi_app(environ, start_response)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1473, in wsgi_app
e2m_111_api | response = self.full_dispatch_request()
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
e2m_111_api | rv = self.dispatch_request()
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
e2m_111_api | return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/flasgger/utils.py", line 305, in wrapper
e2m_111_api | return function(*args, **kwargs)
e2m_111_api | File "/app/api/blueprints/v1/routes.py", line 115, in convert_route
e2m_111_api | resp_dict, code = file_to_markdown( # Response
e2m_111_api | File "/app/api/blueprints/v1/controllers.py", line 27, in file_to_markdown
e2m_111_api | resp: ResponseData = converter.convert(**kwargs)
e2m_111_api | File "/app/api/core/converters/base_converter.py", line 75, in convert
e2m_111_api | raw = self.process(**kwargs)
e2m_111_api | File "/app/api/core/converters/doc/pdf_converter.py", line 179, in process
e2m_111_api | raw = _ocr_high_parse_pdf(
e2m_111_api | File "/app/api/core/converters/doc/pdf_converter.py", line 87, in _ocr_high_parse_pdf
e2m_111_api | full_text, out_meta, image_data = _parse_pdf_and_return_markdown(
e2m_111_api | File "/app/api/core/converters/doc/pdf_converter.py", line 30, in _parse_pdf_and_return_markdown
e2m_111_api | full_text, images, out_meta = convert_single_pdf(pdf_file, model_list, langs=langs)
e2m_111_api | File "/app/api/core/marker/convert.py", line 98, in convert_single_pdf
e2m_111_api | pages, ocr_stats = run_ocr(
e2m_111_api | File "/app/api/core/marker/ocr/recognition.py", line 74, in run_ocr
e2m_111_api | rec_model = setup_recognition_model(lang_tokens)
e2m_111_api | File "/app/api/core/marker/models.py", line 51, in setup_recognition_model
e2m_111_api | rec_model = load_recognition_model(langs=langs)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/surya/model/recognition/model.py", line 38, in load_model
e2m_111_api | model = LangVisionEncoderDecoderModel.from_pretrained(checkpoint, config=config, torch_dtype=dtype)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 371, in from_pretrained
e2m_111_api | return super().from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3351, in from_pretrained
e2m_111_api | resolved_archive_file = cached_file(pretrained_model_name_or_path, filename, **cached_file_kwargs)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/transformers/utils/hub.py", line 399, in cached_file
e2m_111_api | resolved_file = hf_hub_download(
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
e2m_111_api | return fn(*args, **kwargs)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download
e2m_111_api | return _hf_hub_download_to_cache_dir(
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1367, in _hf_hub_download_to_cache_dir
e2m_111_api | _download_to_tmp_and_move(
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1884, in _download_to_tmp_and_move
e2m_111_api | http_get(
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 539, in http_get
e2m_111_api | for chunk in r.iter_content(chunk_size=DOWNLOAD_CHUNK_SIZE):
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 820, in generate
e2m_111_api | yield from self.raw.stream(chunk_size, decode_content=True)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/urllib3/response.py", line 1043, in stream
e2m_111_api | data = self.read(amt=amt, decode_content=decode_content)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/urllib3/response.py", line 935, in read
e2m_111_api | data = self._raw_read(amt)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/urllib3/response.py", line 862, in _raw_read
e2m_111_api | data = self._fp_read(amt, read1=read1) if not fp_closed else b""
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/urllib3/response.py", line 845, in _fp_read
e2m_111_api | return self._fp.read(amt) if amt is not None else self._fp.read()
e2m_111_api | File "/usr/local/lib/python3.10/http/client.py", line 466, in read
e2m_111_api | s = self.fp.read(amt)
e2m_111_api | File "/usr/local/lib/python3.10/socket.py", line 705, in readinto
e2m_111_api | return self._sock.recv_into(b)
e2m_111_api | File "/usr/local/lib/python3.10/ssl.py", line 1307, in recv_into
e2m_111_api | return self.read(nbytes, buffer)
e2m_111_api | File "/usr/local/lib/python3.10/ssl.py", line 1163, in read
e2m_111_api | return self._sslobj.read(len, buffer)
e2m_111_api | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base.py", line 203, in handle_abort
e2m_111_api | sys.exit(1)
e2m_111_api | SystemExit: 1
e2m_111_api | [2024-06-06 21:30:43 +0800] [209] [INFO] Worker exiting (pid: 209)
e2m_111_api | -> Cannot close object, library is destroyed. This may cause a memory leak!
e2m_111_api | -> Cannot close object, library is destroyed. This may cause a memory leak!
e2m_111_api | -> Cannot close object, library is destroyed. This may cause a memory leak!
e2m_111_api | [2024-06-06 21:30:45 +0800] [1] [ERROR] Worker (pid:209) was sent SIGKILL! Perhaps out of memory?
Describe the bug
OCR-High click convert：Error handling request /api/v1/convert
To Reproduce
Steps to reproduce the behavior:
Desktop (please complete the following information):