Segmentation fault when migrate for upgrade

[someconservator]

upgrade 3.4.6 -> 3.5.1
Centos8Stream. latest.
there are:
ldap
gunicorn
sssd
on the server
after CentOS update to the latest i run ./upgrade.sh in a standard way and see:
Applying database migrations (python3 netbox/manage.py migrate)...
./upgrade.sh: line 84: 36246 Segmentation fault (core dumped) python3 netbox/manage.py migrate

and in messages:
kernel: traps: python3[38775] general protection fault ip:7f01cdf8f660 sp:7fff72adf5e0 error:0 in libkrb5-fcafa220.so.3.3[7f01cdf12000+d9000]

[candlerb]

Segmentation fault (core dumped) python3
...
error:0 in libkrb5-fcafa220.so.3.3

Your OS or hardware is broken.

This could be (1) disk data corruption, (2) RAM errors (google for "sig 11 FAQ"), (3) your system has been p0wned, (4) CentOS Stream has released a borked update (CentOS stream receives changes before RedHat; CentOS Stream users are essentially alpha testers for RHEL)

Out of all those, my money would be on (2) - especially if you reboot the machine and try again and it works, or it crashes in a different place with a different error.

If it crashes in exactly the same place with the same error, then it's more likely (1) or (4).

[someconservator]

Yes, developers never make mistakes that cause applications to work incorrectly. and netbox was extensively tested on python 3.11.2. And special thanks for the ubuntu suggestion! :)

[kkthxbye-code]

No he is correct that this is not a netbox issue. Netbox provides no compiled C modules and does not modify memory, as such the segfault must be caused by either python itself, a compiled dependency of either netbox or django that is failing or your system is messed up.

Given that the segfault happened in libkrb5 would suggest that the error might be with psycopg2 and its dependencies on your system. Try running migrate manually in verbose mode to see where it fails.

Also, you are being very rude to the person trying to help you. If you can't respond in a respectful manner, please refrain from responding at all.

[someconservator]

I have three standardized in our company out-of-the-box installations of Centos8Stream with our standardized settings (yes, I know about Red Hat pre-release, I'm RHCE). On each of these standardized servers, I installed the netbox step by step exactly according to your instructions. And I also update the netbox exactly in accordance with them. No self manual builds. Everything is only from manufacturers. On all these three installations, until yesterday, everything worked overwhelmingly correctly. CentosStream was updated yesterday. After that, the netbox-rq service stopped working. It was decided to update the netbox to the latest version. This update also fails with the specified error. Yes, I tried to migrate manually. It crashes with the same errors I mentioned earlier. Repeatedly. Reproducible: when I run the migration (manually or from the update script, not the point) it crashes and I see a kernel error in the logs.
Thus I suppose that the possible reasons are: 1. some error in the updated centos packages (this is already being worked out), 2. python 3.11.2 installed with this system update is somehow not correctly executing the netbox code. May be. I once also achieved this with my own scripts. And I'm not sure how well netbox works with this version of python. That's why I created this discussion (I would have created an bug issue, but for some reason the "submit" button is inactive, I didn’t have enough brains to figure it out). If everything is correct, then great. My job is done for you. Good luck with the development. By the way, you finally developed the custom field search. It's just wonderful! Probably soon the output will also look good.

[candlerb]

On all these three installations, until yesterday, everything worked overwhelmingly correctly. CentosStream was updated yesterday

And that doesn't tell you something? Maybe you get a glimpse of why so many people have abandoned CentOS Stream. It's nothing like the old CentOS, which was a clone of stable and well-tested RHEL.

I would have created an bug issue

What you are seeing is not a bug in Netbox. Netbox is written as Python scripts. If a Python script is able to make the interpreter crash, then it's a problem with the interpreter, or the underlying libraries it depends on, or third party compiled modules (which Netbox doesn't supply), or the hardware it's running on.

[someconservator]

candlerb! May I ask you not to try to help in this case? Please!

[someconservator]

I resolved problem.

1. rebuild venv for example with venv part of /opt/netbox/upgrade.sh (It may be necessery to install a development package for one of the new versions of Python).
2. restart netbox.

That's all

[nab8]

I just ran into the same thing on Rocky 9; there appears to be a problem with newer versions of psycopg2 in the newer RHEL releases.

[grocanar]

I got the same result with a fresh installation of netbox on Rocky9.
Does anyone have a clue to adress this problem?

[someconservator]

it could be psycopg. It is outdated and does not work very well with modern Red Hat family software. Try changing it to psycopg2. I just added 2 to the requirements.txt, although it might be better to do this in loacal_requirements.txt. But I haven't tried that. I just changed it in requirements.txt and it works for me.

[nab8]

I did similar - commented out psycopg & added a psycopg2-binary line like so in requirements.txt:

#psycopg[binary,pool]==3.1.18
psycopg2-binary==2.8.4

We just upgraded to 3.7.4 successfully with this change on Rocky 9.