[FIX] Error psycopg2 when building 15.0 images

[josep-tecnativa]

Error when 15.0 is building:

#9 317.3 Collecting psycopg2>=2.2
#9 317.3   Downloading psycopg2-2.9.10.tar.gz (385 kB)
#9 317.3      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 385.7/385.7 kB 269.5 MB/s eta 0:00:00
#9 317.4   Preparing metadata (setup.py): started
#9 317.6   Preparing metadata (setup.py): finished with status 'error'
#9 317.6   error: subprocess-exited-with-error
#9 317.6   
#9 317.6   × python setup.py egg_info did not run successfully.
#9 317.6   │ exit code: 1
#9 317.6   ╰─> [23 lines of output]
#9 317.6       running egg_info
#9 317.6       creating /tmp/pip-pip-egg-info-rrag0c7u/psycopg2.egg-info
#9 317.6       writing /tmp/pip-pip-egg-info-rrag0c7u/psycopg2.egg-info/PKG-INFO
#9 317.6       writing dependency_links to /tmp/pip-pip-egg-info-rrag0c7u/psycopg2.egg-info/dependency_links.txt
#9 317.6       writing top-level names to /tmp/pip-pip-egg-info-rrag0c7u/psycopg2.egg-info/top_level.txt
#9 317.6       writing manifest file '/tmp/pip-pip-egg-info-rrag0c7u/psycopg2.egg-info/SOURCES.txt'
#9 317.6       
#9 317.6       Error: pg_config executable not found.
#9 317.6       
#9 317.6       pg_config is required to build psycopg2 from source.  Please add the directory
#9 317.6       containing pg_config to the $PATH or specify the full executable path with the
#9 317.6       option:
#9 317.6       
#9 317.6           python setup.py build_ext --pg-config /path/to/pg_config build ...
#9 317.6       
#9 317.6       or with the pg_config option in 'setup.cfg'.
#9 317.6       
#9 317.6       If you prefer to avoid building psycopg2 from source, please install the PyPI
#9 317.6       'psycopg2-binary' package instead.
#9 317.6       
#9 317.6       For further information please check the 'doc/src/install.rst' file (also at
#9 317.6       <https://www.psycopg.org/docs/install.html>).
#9 317.6       
#9 317.6       [end of output] 

I have also tried pre-installing psycopg-binary but keeps trying to install and compile non binary version.

ping @ap-wtioit @Tardo

[samsagaz]

FYI we temp fixed the error in our servers creating a simple script inside build.sh

    ln -s /usr/lib/postgresql/17/bin/pg_config /usr/local/bin
    apt --yes --force-yes install libpq-dev

[Tardo]

It's strange... 10 hours ago version 2.8.6 was in use: https://github.com/Tecnativa/doodba/actions/runs/15066114599/job/42351273469#step:8:472

But the latest builds are using version ‘2.9.10’ which has been released since 16 Oct 2024: https://pypi.org/project/psycopg2/

doodba installs odoo first using setup.py: https://github.com/odoo/odoo/blob/15.0/setup.py#L40
which is "less controlled" than using requirements.txt: https://github.com/odoo/odoo/blob/15.0/requirements.txt#L39
**EDIT: When building the image, doodba uses requirements.txt, then when using the "onbuild", it invokes the script in build.d which uses setup.py
Why didn't the onbuild image have the psyco... package installed? That's the strange thing here... I've been looking at logs and haven't seen any bugs in that regard.

But I have no explanation why this has failed now...

[Tardo]

I already build correctly :) (try relaunching action for 13.0 to see if it passes correctly.)

[ap-wtioit]

Sorry for replying kind of late, got a longer weekend and had to fix a lot of expired tokens in our gitlab yesterday.

According to our logs our images always install 2.8.6 (which works) and 2.9.10 seems not to work for installing:

docker run --rm python:3.8-slim-bullseye pip install "psycopg2>=2.2"

But the requirements.txt for 15.0: https://github.com/OCA/OCB/blob/15.0/requirements.txt does specify psycopg2==2.8.6

And the psycopg>=2.2 seems to come from OCB/setup.py

So i guess somehow this is executed in the wrong order on github.

(We first build the images, tag them with ci-NNNN-NN.N-onbuild in our registry and use the tagged images for tests instead of building new ones but even with the new setUpClass method the tests worked for us, and if tests pass we tag the same images successfully tested as the new prod images NN.N-onbuild to make sure no dependencies change in debian repos / pypi between the test and the prod image)

[ap-wtioit]

Edit: this first error was a user/me error.
Ok what i found now, our image has the dependencies from requirement.txt installed in onbuild (and i suppose that should be how it's done), the tecnativa image does not:
docker run --rm tecnativa/doodba:15.0-onbuild pip install "psycopg>=2.2"

Unable to find image 'tecnativa/doodba:15.0-onbuild' locally
15.0-onbuild: Pulling from tecnativa/doodba
fa0650a893c2: Already exists 
70db9fc16b3a: Already exists 
d245b2373f63: Already exists 
f7d01992d34a: Already exists 
9dc54da0d5db: Pull complete 
f6e934e92e1e: Pull complete 
ee8ef9de9fa8: Pull complete 
2c9f7250c84a: Pull complete 
c5fd08840c71: Pull complete 
fefc66e1261f: Pull complete 
697f3c1df280: Pull complete 
34508a9f1d00: Pull complete 
186c4c32ab5d: Pull complete 
ccac2a142a3b: Pull complete 
7bd9a73f4ea1: Pull complete 
23691099ee6f: Pull complete 
Digest: sha256:ca59616f8af977b0a1d9e30e5b25b294cf9fc428f8639029d82cdb407705764e
Status: Downloaded newer image for tecnativa/doodba:15.0-onbuild
Collecting psycopg>=2.2
  Downloading psycopg-3.2.9-py3-none-any.whl (202 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 202.7/202.7 kB 11.3 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=4.6 in /usr/local/lib/python3.8/site-packages (from psycopg>=2.2) (4.13.2)
Collecting backports.zoneinfo>=0.2.0
  Downloading backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl (74 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.0/74.0 kB 147.9 MB/s eta 0:00:00
Installing collected packages: backports.zoneinfo, psycopg
Successfully installed backports.zoneinfo-0.2.1 psycopg-3.2.9
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip

vs
docker run --rm registry.fqdn/path/to/docker-odoo-base:ci-NNNNNN-15.0-onbuild pip install "psycopg2>=2.2"

Requirement already satisfied: psycopg2>=2.2 in /usr/local/lib/python3.8/site-packages (2.8.6)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip

Update:
had a typo in this test, should have been: docker run --rm tecnativa/doodba:15.0-onbuild pip install "psycopg2>=2.2"

Update2:
The last image used in this PR to test 15.0 had an empty requirements.txt therefore no dependencies were installed:

• https://github.com/Tecnativa/doodba/actions/runs/15106051823/job/42454968977?pr=660#step:8:44

FROM docker.io/tecnativa/doodba:15.0-onbuild@sha256:a8a240b5c8785bea9691c5d90caefa643907c5c98ef71a7a56ecdc20def1ce2c

=>
docker run --rm tecnativa/doodba:15.0-onbuild@sha256:a8a240b5c8785bea9691c5d90caefa643907c5c98ef71a7a56ecdc20def1ce2c ls -al

total 20
drwxr-xr-x 1 root root 4096 May 16 10:40 .
drwxr-xr-x 1 root root 4096 May 16 10:40 ..
drwxr-xr-x 4 root root 4096 May 16 10:40 auto
drwxr-xr-x 1 root root 4096 May 16 10:40 common
drwxr-xr-x 3 root root 4096 May 16 10:40 custom
-rw-r--r-- 1 root root    0 May 16 10:40 requirements.txt