Flask Auto Instrumentation dissapears on debug=True app reload

[NathanielRN]

Describe your environment
python 3.9
flask~=1.0
opentelemetry-api==1.3.0
opentelemetry-sdk==1.3.0
opentelemetry-instrumentation==0.22b0
opentelemetry-instrumentation-flask==0.22b0

Steps to reproduce

I created a very simple flask app my_app.py with either debug=False or debug=True:

from flask import Flask
from opentelemetry import trace

# Setup Flask App

app = Flask(__name__)

@app.route("/my_endpoint")
def call_http():
    return '{{"traceId": "{}"}}'.format(
        trace.get_current_span().get_span_context().trace_id
    )


if __name__ == "__main__":
    app.run(**{"port": 8080, "debug": False})

I then use auto-instrumentation to instrument the app with the ConsoleExporter:

opentelemetry-instrument --trace-exporter console_span python3 my_app.py

What is the expected behavior?

If I use debug=False everything works fine.

Console output:

{
    "name": "/my_endpoint",
    "context": {
        "trace_id": "0x317a86be6e1cbe5fa913f003154f2a32",
        "span_id": "0x2aafb7284d6168ea",
        "trace_state": "[]"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": null,
    "start_time": "2021-06-22T17:08:52.484880Z",
    "end_time": "2021-06-22T17:08:52.485936Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "http.method": "GET",
        "http.server_name": "127.0.0.1",
        "http.scheme": "http",
        "net.host.port": 8080,
        "http.host": "localhost:8080",
        "http.target": "/my_endpoint",
        "net.peer.ip": "127.0.0.1",
        "http.user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Firefox/78.0",
        "net.peer.port": 49317,
        "http.flavor": "1.1",
        "http.route": "/my_endpoint",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.3.0",
        "service.name": "unknown_service"
    }
}

Browser response is OK:

What is the actual behavior?

If I use debug=True it does not work

No console output:

No trace id that can be returned in the response:

Additional context

I used print statements to confirm that auto instrumentation does run and replace the right methods before the app starts. But when the Restarting with stat message appears it seems like the instrumentation is lost.

The fix might require investigating how auto reloader works and instrumenting some hidden "clean slate app" that the auto reloader replaces the currently running app with to restart the instrumentation.

[codeboten]

Is this a duplicate of #477?

[NathanielRN]

Thanks for pointing me to that, it was helpful because I found out that debug=True, use_reloader=False in this Slack thread works and better isolates what the problem is.

I also see that the docs were updated in open-telemetry/opentelemetry-python#1795

But I think we could still benefit from either:

• Solving the issue with the reloader so we can instrument even with it enabled (better fix)
• Wrap flask.Flask.__init__ so that there is an error if use_reloader=True (temporary fix)

It's an easy caveat to miss, and since flask is the first thing you tell auto instrumentation to run (i.e. in opentelemetry-instrument python my_flask_app.py) I think the error would save people some time given that they most likely want their flask app to be instrumented since they are using auto instrumentation in the first place.

[github-actions]

This issue was marked stale due to lack of activity. It will be closed in 30 days.

[glebignatieff]

An off top question to everyone. I'm wondering why the status.status_code in the example is UNSET whereas http.status_code is 200? Shouldn't it be OK instead?