Error occurred while using Docker for build

[XmySz]

Hello, I encountered an error while building histomicstk using Docker (for use in DSA). The error occurred in 'Collecting pyvips (from large image [all])'. I tried to cancel the -- find_link parameter, but this would result in more errors. Here is the complete error record:

(base) F:\Data\PycharmProjects\HistomicsTK
docker build .

[+] Building 28.1s (9/19)                                                                                                                docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                     0.0s
 => => transferring dockerfile: 3.71kB                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/python:3.12-slim                                                                                      1.9s
 => [auth] library/python:pull token for registry-1.docker.io                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                        0.0s
 => => transferring context: 2.57kB                                                                                                                      0.0s
 => [ 1/14] FROM docker.io/library/python:3.12-slim@sha256:c24c34b502635f1f7c4e99dc09a2cbd85d480b7dcfd077198c6b5af138906390                              0.0s
 => [internal] load build context                                                                                                                        0.0s
 => => transferring context: 22.64kB                                                                                                                     0.0s
 => CACHED [ 2/14] RUN apt-get update &&     apt-get install --yes --no-install-recommends     curl     ca-certificates     git     wget     build-esse  0.0s
 => CACHED [ 3/14] RUN mkdir -p /HistomicsTK                                                                                                             0.0s
 => ERROR [ 4/14] RUN pip install --no-cache-dir --upgrade pip setuptools &&     pip install --no-cache-dir 'bokeh>=0.12.14,<3' &&     pip install --n  26.1s 
------
 > [ 4/14] RUN pip install --no-cache-dir --upgrade pip setuptools &&     pip install --no-cache-dir 'bokeh>=0.12.14,<3' &&     pip install --no-cache-dir 'la
rge-image[all]' --find-links https://girder.github.io/large_image_wheels &&     pip install --no-cache-dir girder-client &&     pip install --no-cache-dir nim
fa numpy scipy Pillow pandas scikit-image scikit-learn imageio 'shapely[vectorized]' opencv-python-headless sqlalchemy matplotlib 'dask[dataframe]' distributed &&     rm -rf /root/.cache/pip/* &&     rdfind -minsize 32768 -makehardlinks true -makeresultsfile false /usr/local:
1.443 Requirement already satisfied: pip in /usr/local/lib/python3.12/site-packages (24.2)
1.884 Collecting setuptools
2.022   Downloading setuptools-74.1.2-py3-none-any.whl.metadata (6.7 kB)
2.061 Downloading setuptools-74.1.2-py3-none-any.whl (1.3 MB)
2.204    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 11.6 MB/s eta 0:00:00
2.218 Installing collected packages: setuptools
2.892 Successfully installed setuptools-74.1.2
2.893 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering y
our system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
3.655 Collecting bokeh<3,>=0.12.14
3.752   Downloading bokeh-2.4.3-py3-none-any.whl.metadata (14 kB)
3.805 Collecting Jinja2>=2.9 (from bokeh<3,>=0.12.14)
3.829   Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
4.099 Collecting numpy>=1.11.3 (from bokeh<3,>=0.12.14)
4.131   Downloading numpy-2.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
4.207 Collecting packaging>=16.8 (from bokeh<3,>=0.12.14)
4.244   Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
4.488 Collecting pillow>=7.1.0 (from bokeh<3,>=0.12.14)
4.511   Downloading pillow-10.4.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (9.2 kB)
4.590 Collecting PyYAML>=3.10 (from bokeh<3,>=0.12.14)
4.613   Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
4.681 Collecting tornado>=5.1 (from bokeh<3,>=0.12.14)
4.705   Downloading tornado-6.4.1-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
4.744 Collecting typing-extensions>=3.10.0 (from bokeh<3,>=0.12.14)
4.767   Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
4.841 Collecting MarkupSafe>=2.0 (from Jinja2>=2.9->bokeh<3,>=0.12.14)
4.864   Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
4.893 Downloading bokeh-2.4.3-py3-none-any.whl (18.5 MB)
6.511    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.5/18.5 MB 11.6 MB/s eta 0:00:00
6.563 Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
6.598 Downloading numpy-2.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.0 MB)
7.966    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.0/16.0 MB 11.8 MB/s eta 0:00:00
7.990 Downloading packaging-24.1-py3-none-any.whl (53 kB)
8.016 Downloading pillow-10.4.0-cp312-cp312-manylinux_2_28_x86_64.whl (4.5 MB)
8.403    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 11.8 MB/s eta 0:00:00
8.426 Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (767 kB)
8.495    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 767.5/767.5 kB 12.5 MB/s eta 0:00:00
8.519 Downloading tornado-6.4.1-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (436 kB)
8.580 Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
8.606 Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
8.688 Installing collected packages: typing-extensions, tornado, PyYAML, pillow, packaging, numpy, MarkupSafe, Jinja2, bokeh
11.92 Successfully installed Jinja2-3.1.4 MarkupSafe-2.1.5 PyYAML-6.0.2 bokeh-2.4.3 numpy-2.1.1 packaging-24.1 pillow-10.4.0 tornado-6.4.1 typing-extensions-4.12.2
11.92 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering y
our system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
12.73 Looking in links: https://girder.github.io/large_image_wheels
14.99 Collecting large-image[all]
15.08   Downloading large_image-1.29.7-py3-none-any.whl.metadata (16 kB)
15.24 Collecting cachetools (from large-image[all])
15.26   Downloading cachetools-5.5.0-py3-none-any.whl.metadata (5.3 kB)
15.90 Collecting palettable (from large-image[all])
15.93   Downloading palettable-3.3.3-py2.py3-none-any.whl.metadata (3.3 kB)
15.93 Requirement already satisfied: Pillow in /usr/local/lib/python3.12/site-packages (from large-image[all]) (10.4.0)
15.93 Requirement already satisfied: numpy in /usr/local/lib/python3.12/site-packages (from large-image[all]) (2.1.1)
15.93 Requirement already satisfied: typing-extensions in /usr/local/lib/python3.12/site-packages (from large-image[all]) (4.12.2)
16.33 Collecting large-image-source-pil>=1.29.7 (from large-image-source-pil[all]>=1.29.7; extra == "all"->large-image[all])
16.35   Downloading large_image_source_pil-1.29.7-py3-none-any.whl.metadata (1.2 kB)
16.59 Collecting large-image-source-rasterio>=1.29.7 (from large-image[all])
16.62   Downloading large_image_source_rasterio-1.29.7-py3-none-any.whl.metadata (1.2 kB)
17.04 Collecting large-image-source-test>=1.29.7 (from large-image[all])
17.07   Downloading large_image_source_test-1.29.7-py3-none-any.whl.metadata (925 bytes)
17.38 **Collecting pyvips (from large-image[all])**
18.10   WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.u
rllib3.connection.HTTPSConnection object at 0x7f590cf48740>: Failed to establish a new connection: [Errno 111] Connection refused')': /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
18.60   WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.u
rllib3.connection.HTTPSConnection object at 0x7f590cf48800>: Failed to establish a new connection: [Errno 111] Connection refused')': /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
19.61   WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.u
rllib3.connection.HTTPSConnection object at 0x7f590d08b470>: Failed to establish a new connection: [Errno 111] Connection refused')': /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
21.61   WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.u
rllib3.connection.HTTPSConnection object at 0x7f590cf48ce0>: Failed to establish a new connection: [Errno 111] Connection refused')': /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
25.61   WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.u
rllib3.connection.HTTPSConnection object at 0x7f590cf49040>: Failed to establish a new connection: [Errno 111] Connection refused')': /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
25.62 ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /gi
rder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f590cf49250>: Failed to establish a new connection: [Errno 111] Connection refused'))
25.62
------
Dockerfile:45
--------------------
  44 |
  45 | >>> RUN pip install --no-cache-dir --upgrade pip setuptools && \
  46 | >>>     # Install bokeh to help debug dask \
  47 | >>>     pip install --no-cache-dir 'bokeh>=0.12.14,<3' && \
  48 | >>>     # Install large_image memcached and sources extras \
  49 | >>>     pip install --no-cache-dir 'large-image[all]' --find-links https://girder.github.io/large_image_wheels && \
  50 | >>>     # Install girder-client \
  51 | >>>     pip install --no-cache-dir girder-client && \
  52 | >>>     # Install some other dependencies here to save time in the histomicstk \
  53 | >>>     # install step \
  54 | >>>     pip install --no-cache-dir nimfa numpy scipy Pillow pandas scikit-image scikit-learn imageio 'shapely[vectorized]' opencv-python-headless sqlalchemy matplotlib 'dask[dataframe]' distributed && \
  55 | >>>     # clean up \
  56 | >>>     rm -rf /root/.cache/pip/* && \
  57 | >>>     # Make duplicate files not take extra space in the docker image \
  58 | >>>     rdfind -minsize 32768 -makehardlinks true -makeresultsfile false /usr/local
  59 |
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install --no-cache-dir --upgrade pip setuptools &&     pip install --no-cache-dir 'bokeh>=0.12.14,<3' &&     p
ip install --no-cache-dir 'large-image[all]' --find-links https://girder.github.io/large_image_wheels &&     pip install --no-cache-dir girder-client &&     p
ip install --no-cache-dir nimfa numpy scipy Pillow pandas scikit-image scikit-learn imageio 'shapely[vectorized]' opencv-python-headless sqlalchemy matplotlib
 'dask[dataframe]' distributed &&     rm -rf /root/.cache/pip/* &&     rdfind -minsize 32768 -makehardlinks true -makeresultsfile false /usr/local" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/nl08bjxtwj0nd5dc67f8p8r6h`
```

[manthey]

The relevant part of your error message is
Could not install packages due to an OSError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f590cf49250>: Failed to establish a new connection: [Errno 111] Connection refused')) 25.62

It looks like you have sort of network error reaching github from whatever environment you are building in. Maybe check your network and try again?

[dgutman]

Also if your main system uses a proxy to connect to the web, that is not necessarily passed through to the docker container depending on how docker is configured.

…

On Mon, Sep 9, 2024 at 10:53 AM David Manthey ***@***.***> wrote: The relevant part of your error message is Could not install packages due to an OSError: HTTPSConnectionPool(host=' raw.githubusercontent.com', port=443): Max retries exceeded with url: /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f590cf49250>: Failed to establish a new connection: [Errno 111] Connection refused')) 25.62 It looks like you have sort of network error reaching github from whatever environment you are building in. Maybe check your network and try again? — Reply to this email directly, view it on GitHub <#1128 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFODTRERLSMRF7GQANZ2BDZVWY6VAVCNFSM6AAAAABN2S543SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZYGM2DKNZXGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- David A Gutman, M.D. Ph.D. Associate Professor of Pathology Emory University School of Medicine

[XmySz]

The relevant part of your error message is Could not install packages due to an OSError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /girder/large_image_wheels/wheelhouse/pyvips-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f590cf49250>: Failed to establish a new connection: [Errno 111] Connection refused')) 25.62

It looks like you have sort of network error reaching github from whatever environment you are building in. Maybe check your network and try again?

I have configured a proxy on the host and can access GitHub normally. However, there is no proxy set in Docker (using the default settings). I successfully built it using the same steps a week ago, but even though I haven't changed anything since then, it keeps reporting network errors.