TypeError when task is manually marked success

[yoichiwo7]

When the non-executed task is manually marked success via Web UI or CLI, the following TypeError will be raised while collecting dag duration metrics.

  File "/usr/local/lib/python3.7/site-packages/airflow_prometheus_exporter/prometheus_exporter.py", line xxx, in collect
    dag_duration_value = (dag.end_date - dag.start_date).total_seconds()
TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'

Detail

The dag duration uses TaskInstance's start_date to calculate the duration.

airflow-prometheus-exporter/airflow_prometheus_exporter/prometheus_exporter.py

Line 90 in 43862bf

func.min(TaskInstance.start_date).label("start_date"),

The problem is when the non-executed task is manually marked success, the corresponding TaskInstance record's start_date has no value. This will set the dag's start_date to None value and cause the TypeError mentioned in this issue.

To fix the problem, we need to add filter to dag_start_dt_query so that TaskInstance record without start_date/end_date won't be included in the query result. The filter should look like the following code.

airflow-prometheus-exporter/airflow_prometheus_exporter/prometheus_exporter.py

Lines 273 to 274 in 43862bf

	TaskInstance.start_date.isnot(None),
	TaskInstance.end_date.isnot(None),

We already applied the fix in our private environment and now it works without error.

[DongshengXiong-old]

Any update for this issue?

[jefffarkas]

We're also experiencing the same issue.

[abhishekray07]

Sorry for the issues. I just pushed a fix and released a new version. Could y'all test and see if it fixes the problem?

[DongshengXiong-old]

@abhishekray07 we still have this issue since we have to mark some DAGs as success with skipped tasks. After manually adding start_state and end_state for skipped tasks, metrics can show up correctly. Maybe you need to add one more check like:
TaskInstance.state != State.SKIPPED"