Identify what is leading integration tests run duration to be so long and fix it, if possible

[tatiana]

A few of our integration tests are taking almost double the time to run than others, as observed in the CI job run:

• Run-Integration-Tests (3.11, 2.6, 1.11): 15m 28s
• Run-Integration-Tests (3.11, 2.7, 1.11): 15m 52s
• Run-Integration-Tests (3.11, 2.8, 1.11): 21m 17s
• Run-Integration-Tests (3.11, 2.9, 1.11): 10m 10s
• Run-Integration-Tests (3.11, 2.10, 1.11): 17m 43s
• Run-Integration-Tests (3.11, 2.11, 1.11): 14m 52s
• Run-Integration-Tests (3.11, 3.0, 1.11): 20m 13s
• Run-Integration-Tests (3.11, 3.1, 1.11): 28m 56s

While we expect some level of variation, since not all DAGs run in all Airflow versions, we should identify what is leading to this - and if possible - fix it.

[michal-mrazek]

I have also noticed this while running tests locally, it is likely because the tests actually run the dags https://github.com/astronomer/astronomer-cosmos/blob/main/tests/utils.py#L71 so whole dbt project if I read it correctly

[tatiana]

@michal-mrazek yes, that's per design..! But it is a bit concerning that AF 3.1 would take twice as long to run the DAGs as AF 2.11 🙈 This type of test has helped us make Cosmos backwards compatible and avoid breaking features - especially during Airflow upgrades.

If we can't fix this in Cosmos, we can reach out to the Airflow team for them to prioritise improving the performance.

[michal-mrazek]

Okay, thank you for the explanation, fingers crossed 🤞