fix: sigterm handling so NGINX gracefully exits#1
Conversation
After compiling nginx with the modifications:
```shell
make build-heroku-18
```
I tested that the changes actually worked:
```shell
apt-get update
apt-get install -y python3-venv
python3 -m venv venv
./venv/bin/pip install gunicorn
FORCE=1 bin/start-nginx
./venv/bin/gunicorn -b unix:/tmp/nginx.socket app:app
# in another terminal (should hang for 15s assuming app is setup correctly)
curl localhost:5000
# in another terminal
# get the PID
ps aux | grep master
# should be graceful, that is nginx should shutdown after it finishes serving the request
kill -TERM $PID
FORCE=1 bin/start-nginx
# should kill nginx without waiting
# curl returns an error:
# curl: (52) Empty reply from server
kill -QUIT $PID
```
The `app` used by `gunicorn` was the hello world with a sleep thrown in
so we mimic a long running request.
```python
import time
def app(environ, start_response):
time.sleep(15)
data = b"Hello, World!\n"
start_response("200 OK", [
("Content-Type", "text/plain"),
("Content-Length", str(len(data)))
])
return iter([data])
```
Based on heroku#56
|
I think we can specify the buildpack to use this branch if we want to test on staging before merging. |
|
I set this up on staging and it seems to run alright. Also checked the SHA of the binaries to make sure it's actually using the new binary since it's hard to replicate the shutdown behavior. Prod (old Buildpack): Staging: Local: |
edanaher
left a comment
There was a problem hiding this comment.
I missed this before merge, but this LGTM aside from a couple unnecessary variables.
Also, if you wanted to test this, I suspect rapidly hitting staging (with either a benchmarking tool or a couple simultaneous curls (e.g., parallel -j4 curl marshall-staging/some-endpoint) while you restart marshall would suffice to reproduce this. Ideally, though, you'd test before the update to confirm that this does consistently produce 503's/H13's, and then see after that they're gone.
| NGX_SHUTDOWN_SIGNAL=TERM | ||
| NGX_TERMINATE_SIGNAL=QUIT |
There was a problem hiding this comment.
I'm pretty sure these two lines aren't doing anything, since they're just setting shell variables that are never used. I suspect a previous version of this script used them, but they're no longer necessary.
After compiling nginx with the modifications:
I then tested that the changes actually worked:
The
appused bygunicornwas the hello world with a sleep thrown inso we mimic a long running request.
Based on heroku#56