Support for Schedulers configuration

[ryanClift-sd]

Have a question? Please checkout our Slack Community or visit our Slack Archive.

Describe the Feature

Support for configuration attribute that allows setting the number of Apache Airflow Schedulers for the MWAA Environment as listed here Schedulers

Expected Behavior

Ability to set Scheduler Count for MWAA environment creation or updating

Use Case

We needed to increase the number of Airflow Schedulers run in our MWAA environment

Describe Ideal Solution

Ability to set the Schedulers configuration parameter

Alternatives Considered

Deploying MWAA terraform code without the parameter, once the environment is up then we manually adjust this single parameter and have reboot it manually via the UI

Additional Context

I couldn't find any reference to an actual airflow configuration parameter that I could provide via the "airflow_configuration_options" to adjust the number of schedulers that run for the airflow instance. I could change the DAG processing threads, but not the total # of schedulers that are running via the airflow configuration.

Airflow Scheduler Concept
AmazonMWAA Api Changelog 2021-05-26

• Might've been implemented sooner than this though
  AmazonMWAA Latest Docs - Schedulers