Airflow 1.10.9, 2020-02-07

Bug Fixes

• [AIRFLOW-6751] Pin Werkzeug (dependency of a number of our dependencies) to < 1.0.0 (#7377)

Airflow 1.10.8, 2020-02-07

New Features

• [AIRFLOW-4026] Add filter by DAG tags (#6489)
• [AIRFLOW-6613] Center dag on graph view load (#7238)
• [AIRFLOW-5843] Add conf option to Add DAG Run view (#7281)
• [AIRFLOW-4495] Allow externally triggered dags to run for future exec dates (#7038)

Improvements

• [AIRFLOW-6438] Filter DAGs returned by blocked (#7019)
• [AIRFLOW-6666] Resolve js-yaml advisory (#7283)
• [AIRFLOW-6632] Bump dagre-d3 to resolve lodash CVE advisory (#7280)
• [AIRFLOW-6667] Resolve serialize-javascript advisory (#7282)
• [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable (#7134)
• [AIRFLOW-6495] Load DAG only once when running a task using StandardTaskRunner (#7090)
• [AIRFLOW-6319] Add support for AWS Athena workgroups (#6871)
• [AIRFLOW-6677] Remove deprecation warning from SQLAlchmey (#7289)
• [AIRFLOW-6428] Fix import path for airflow.utils.dates.days_ago in Example DAGs (#7007)
• [AIRFLOW-6595] Use TaskNotFound exception instead of AirflowException (#7210)
• [AIRFLOW-6620] Mock celery in worker cli test (#7243)
• [AIRFLOW-6608] Change logging level for Bash & PyOperator Env exports
• [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker (#6633)
• [AIRFLOW-6359] Make Spark status_poll_interval explicit (#6978)
• [AIRFLOW-6359] spark_submit_hook.py status polling interval config (#6909)
• [AIRFLOW-6316] Use exampleinclude directives in tutorial.rst (#6868)
• [AIRFLOW-6519] Make TI logs constants in Webserver configurable (#7113)
• [AIRFLOW-6327] http_hook: Accept json= parameter for payload (#6886)
• [AIRFLOW-6261] flower_basic_auth eligible to _cmd (#6825)
• [AIRFLOW-6238] Filter dags returned by dag_stats
• [AIRFLOW-5616] Switch PrestoHook from pyhive to prestosql-client to s
• [AIRFLOW-6611] Add proxy_fix configs to default_airflow.cfg (#7236)
• [AIRFLOW-6557] Add test for newly added fields in BaseOperator (#7162)
• [AIRFLOW-6584] Pin cassandra driver (#7194)
• [AIRFLOW-6537] Fix backticks in rst files (#7140)
• [AIRFLOW-4428] Error if exec_date before default_args.start_date in trigger_dag (#6948)
• [AIRFLOW-6330] Show cli help when param blank or typo (#6883)
• [AIRFLOW-4113] Unpin boto3 (#6884)
• [AIRFLOW-6181] Add DebugExecutor (#6740)
• [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting (#7097)
• [AIRFLOW-6436] Cleanup for Airflow configs doc generator code (#7036)
• [AIRFLOW-6436] Add x_frame_enabled config in config.yml (#7024)
• [AIRFLOW-6436] Create & Automate docs on Airflow Configs (#7015)
• [AIRFLOW-6527] Make send_task_to_executor timeout configurable (#7143)
• [AIRFLOW-6272] Switch from npm to yarnpkg for managing front-end dependencies (#6844)
• [AIRFLOW-6350] Security - spark submit operator logging+exceptions should mask passwords
• [AIRFLOW-6358] Log details of failed task (#6908)
• [AIRFLOW-5149] Skip SLA checks config (#6923)
• [AIRFLOW-6057] Update template_fields of the PythonSensor (#6656)
• [AIRFLOW-4445] Mushroom cloud errors too verbose (#6952)
• [AIRFLOW-6394] Simplify github PR template (#6955)
• [AIRFLOW-5385] spark hook does not work on spark 2.3/2.4 (#6976)

Bug Fixes

• [AIRFLOW-6345] Ensure arguments to ProxyFix are integers (#6901)
• [AIRFLOW-6576] Fix scheduler crash caused by deleted task with sla misses (#7187)
• [AIRFLOW-6686] Fix syntax error constructing list of process ids (#7298)
• [AIRFLOW-6683] REST API respects store_serialized_dag setting (#7296)
• [AIRFLOW-6553] Add upstream_failed in instance state filter to WebUI (#7159)
• [AIRFLOW-6357] Highlight nodes in Graph UI if task id contains dots (#6904)
• [AIRFLOW-3349] Use None instead of False as value for encoding in StreamLogWriter (#7329)
• [AIRFLOW-6627] Email with incorrect DAG not delivered (#7250)
• [AIRFLOW-6637] Fix airflow test command in 1.10.x
• [AIRFLOW-6636] Avoid exceptions when printing task instance
• [AIRFLOW-6522] Clear task log file before starting to fix duplication in S3TaskHandler (#7120)
• [AIRFLOW-5501] Make default in_cluster value in KubernetesPodOperator respect config (#6124)
• [AIRFLOW-6514] Use RUNNING_DEPS to check run from UI (#6367)
• [AIRFLOW-6381] Remove styling based on DAG id from DAGs page (#6985)
• [AIRFLOW-6434] Add return statement back to DockerOperator.execute (#7013)
• [AIRFLOW-2516] Fix mysql deadlocks (#6988)
• [AIRFLOW-6528] Disable flake8 W503 line break before binary operator (#7124)
• [AIRFLOW-6517] Make merge_dicts function recursive (#7111)
• [AIRFLOW-5621] Failure callback is not triggered when marked Failed on UI (#7025)
• [AIRFLOW-6353] Security - ui - add click jacking defence (#6995)
• [AIRFLOW-6348] Security - cli.py is currently printing logs with password (#6915)
• [AIRFLOW-6323] Remove non-ascii letters from default config (#6878)
• [AIRFLOW-6506] Fix do_xcom_push defaulting to True in KubernetesPodOperator (#7122)
• [AIRFLOW-6516] BugFix: airflow.cfg does not exist in Volume Mounts (#7109)
• [AIRFLOW-6427] Fix broken example_qubole_operator dag (#7005)
• [AIRFLOW-6385] BugFix: SlackAPIPostOperator fails when blocks not set (#7022)
• [AIRFLOW-6347] BugFix: Can't get task logs when serialization is enabled (#7092)
• [AIRFLOW-XXXX] Fix downgrade of db migration 0e2a74e0fc9f (#6859)
• [AIRFLOW-6366] Fix migrations for MS SQL Server (#6920)
• [AIRFLOW-5406] Allow spark without kubernetes (#6921)
• [AIRFLOW-6229] SparkSubmitOperator polls forever if status json can't… (#6918)
• [AIRFLOW-6352] Security - ui - add login timeout (#6912)
• [AIRFLOW-6397] Ensure sub_process attribute exists before trying to kill it (#6958)
• [AIRFLOW-6400] Fix pytest not working on Windows (#6964)
• [AIRFLOW-6418] Remove SystemTest.skip decorator (#6991)
• [AIRFLOW-6425] Serialization: Add missing DAG parameters to Json Schema (#7002)

Misc/Internal

• [AIRFLOW-6467] Use self.dag i/o creating a new one (#7067)
• [AIRFLOW-6490] Improve time delta comparison in local task job tests (#7083)
• [AIRFLOW-5814] Implementing Presto hook tests (#6491)
• [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing (#6516)
• [AIRFLOW-XXXX] Move airflow-config-yaml pre-commit before pylint (#7108)
• [AIRFLOW-XXXX] Improve clarity of confirm message (#7110)
• [AIRFLOW-6662] install dumb init (#7300)
• [AIRFLOW-6705] One less chatty message at breeze initialisation (#7326)
• [AIRFLOW-6705] Less chatty integration/backend checks (#7325)
• [AIRFLOW-6662] Switch to --init docker flag for signal propagation (#7278)
• [AIRFLOW-6661] Fail after 50 failing tests (#7277)
• [AIRFLOW-6607] Get rid of old local scripts for Breeze (#7225)
• [AIRFLOW-6589] BAT tests run in pre-commit on bash script changes (#7203)
• [AIRFLOW-6592] Doc build is moved to test phase (#7208)
• [AIRFLOW-6641] Better diagnostics for kubernetes flaky tests (#7261)
• [AIRFLOW-6642] Make local task job test less flaky (#7262)
• [AIRFLOW-6643] Fix flakiness of kerberos tests
• [AIRFLOW-6638] Remove flakiness test from test_serialized_db remove
• [AIRFLOW-6701] Rat is downloaded from stable backup/mirrors (#7323)
• [AIRFLOW-6702] Dumping kind logs to file.io. (#7319)
• [AIRFLOW-6491] Improve handling of Breeze parameters (#7084)
• [AIRFLOW-6470] Avoid pipe to file when do curl (#7063)
• [AIRFLOW-6471] Add pytest-instafail plugin (#7064)
• [AIRFLOW-6462] Limit exported variables in Dockerfile/Breeze (#7057)
• [AIRFLOW-6465] Add bash autocomplete for airflow in Breeze (#7060)
• [AIRFLOW-6464] Add cloud providers CLI tools in Breeze (#7059)
• [AIRFLOW-6461] Remove silent flags in Dockerfile (#7052)
• [AIRFLOW-6459] Increase verbosity of pytest (#7049)
• [AIRFLOW-6370] Skip Cassandra tests if cluster is not up (#6926)
• [AIRFLOW-6511] Remove BATS docker containers (#7103)
• [AIRFLOW-6475] Remove duplication of volume mount specs in Breeze.. (#7065)
• [AIRFLOW-6489] Add BATS support for Bash unit testing (#7081)
• [AIRFLOW-6387] print details of success/skipped task (#6956)
• [AIRFLOW-6568] Add Emacs related files to .gitignore (#7175)
• [AIRFLOW-6575] Entropy source for CI tests is changed to unblocking (#7185)
• [AIRFLOW-6496] Separate integrations in tests (#7091)
• [AIRFLOW-6634] Set PYTHONPATH in interactive Breeze
• [AIRFLOW-6564] Additional diagnostics information on CI check failure (#7172)
• [AIRFLOW-6383] Add no trailing-whitespace pre-commit hook (#6941)

Doc only changes

• [AIRFLOW-XXXX] Consistency fixes in new documentation (#7207)
• [AIRFLOW-XXXX] Improve grammar and structure in FAQ doc (#7291)
• [AIRFLOW-XXXX] Fix email configuration link in CONTRIBUTING.rst (#7311)
• [AIRFLOW-XXXX] Update docs with new BranchPythonOperator behaviour (#4682)
• [AIRFLOW-XXXX] Fix Typo in scripts/ci/ci_run_airflow_testing.sh (#7235)
• [AIRFLOW-XXXX] Screenshot showing disk space configuration for OSX (#7226)
• [AIRFLOW-XXXX] Add mentoring information to contributing docs (#7202)
• [AIRFLOW-XXXX] Add rebase info to contributing (#7201)
• [AIRFLOW-XXXX] Increase verbosity of static checks in CI (#7200)
• [AIRFLOW-XXXX] Adds branching strategy to documentation (#7193)
• [AIRFLOW-XXXX] Move email configuration from the concept page (#7189)
• [AIRFLOW-XXXX] Update task lifecycle diagram (#7161)
• [AIRFLOW-XXXX] Fix reference in concepts doc (#7135)
• [AIRFLOW-XXXX] Clear debug docs (#7104)
• [AIRFLOW-XXXX] Fix typos and broken links in development docs (#7086)
• [AIRFLOW-XXXX] Clarify wait_for_downstream and execution_date (#6999)
• [AIRFLOW-XXXX] Add airflow dags show command guide (#7014)
• [AIRFLOW-XXXX] Update operation chaining documentation (#7018)
• [AIRFLOW-XXXX] Add .autoenv_leave.zsh to .gitignore (#6986)
• [AIRFLOW-XXXX] Fix development packages installation instructions (#6942)
• [AIRFLOW-XXXX] Update committers list (#7212)
• [AIRFLOW-XXXX] Move UPDATING changes into correct versions (#7166)
• [AIRFLOW-XXXX] Add Documentation for check_slas flag (#6974)
• [AIRFLOW-XXXX] Fix gcp keyfile_d...

Airflow 1.10.7, 2019-12-24

New Features

• [AIRFLOW-5088] [AIP-24] Persisting serialized DAG in DB for webserver scalability (#5992)
• [AIRFLOW-6083] Adding ability to pass custom configuration to AWS Lambda client. (#6678)
• [AIRFLOW-5117] Automatically refresh EKS API tokens when needed (#5731)
• [AIRFLOW-5118] Add ability to specify optional components in DataprocClusterCreateOperator (#5821)
• [AIRFLOW-5681] Allow specification of a tag or hash for the git_sync init container (#6350)
• [AIRFLOW-6025] Add label to uniquely identify creator of Pod (#6621)
• [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator) (#5489)
• [AIRFLOW-5751] add get_uri method to Connection (#6426)
• [AIRFLOW-6056] Allow EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id (#6655)
• [AIRFLOW-2694] Declare permissions in DAG definition (#4642)
• [AIRFLOW-4940] Add DynamoDB to S3 operator (#5663)
• [AIRFLOW-4161] BigQuery to Mysql Operator (#5711)
• [AIRFLOW-6041] Add user agent to the Discovery API client (#6636)
• [AIRFLOW-6089] Reorder setup.py dependencies and add ci (#6681)
• [AIRFLOW-5921] Add bulk_load_custom to MySqlHook (#6575)
• [AIRFLOW-5854] Add support for tty parameter in Docker related operators (#6542)
• [AIRFLOW-4758] Add GcsToGDriveOperator operator (#5822)

Improvements

• [AIRFLOW-3656] Show doc link for the current installed version (#6690)
• [AIRFLOW-5665] Add path_exists method to SFTPHook (#6344)
• [AIRFLOW-5729] Make InputDataConfig optional in Sagemaker's training config (#6398)
• [AIRFLOW-5045] Add ability to create Google Dataproc cluster with custom image from a different project (#5752)
• [AIRFLOW-6132] Allow to pass in tags for the AzureContainerInstancesOperator (#6694)
• [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization (#6715)
• [AIRFLOW-5947] Make the json backend pluggable for DAG Serialization (#6630)
• [AIRFLOW-6239] Filter dags return by last_dagruns (to only select visible dags, not all dags) (#6804)
• [AIRFLOW-6095] Filter dags returned by task_stats (to only select visible dags, not all dags) (#6684)
• [AIRFLOW-4482] Add execution_date to "trigger DagRun" API response (#5260)
• [AIRFLOW-1076] Add get method for template variable accessor (#6793)
• [AIRFLOW-5194] Add error handler to action log (#5883)
• [AIRFLOW-5936] Allow explicit get_pty in SSHOperator (#6586)
• [AIRFLOW-5474] Add Basic auth to Druid hook (#6095)
• [AIRFLOW-5726] Allow custom filename in RedshiftToS3Transfer (#6396)
• [AIRFLOW-5834] Option to skip serve_logs process with airflow worker (#6709)
• [AIRFLOW-5583] Extend the 'DAG Details' page to display the start_date / end_date (#6235)
• [AIRFLOW-6250] Ensure on_failure_callback always has a populated context (#6812)
• [AIRFLOW-6222] http hook logs response body for any failure (#6779)
• [AIRFLOW-6260] Drive _cmd config option by env var (AIRFLOW__CORE__SQL_ALCHEMY_CONN_CMD for example) (#6801)
• [AIRFLOW-6168] Allow proxy_fix middleware of webserver to be configurable (#6723)
• [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. (#6627)
• [AIRFLOW-4145] Allow RBAC roles permissions, ViewMenu to be over-ridable (#4960)
• [AIRFLOW-5928] Hive hooks load_file short circuit (#6582)
• [AIRFLOW-5313] Add params support for awsbatch_operator (#5900)
• [AIRFLOW-2227] Add delete method to Variable class (#4963)
• [AIRFLOW-5082] Add subject in AwsSnsHook (#5694)
• [AIRFLOW-5715] Make email, owner context available (#6385)
• [AIRFLOW-5345] Allow SqlSensor's hook to be customized by subclasses (#5946)
• [AIRFLOW-5417] Fix DB disconnects during webserver startup (#6023)
• [AIRFLOW-5730] Enable get_pandas_df on PinotDbApiHook (#6399)
• [AIRFLOW-3235] Add list function in AzureDataLakeHook (#4070)
• [AIRFLOW-5442] implementing get_pandas_df method for druid broker hook (#6057)
• [AIRFLOW-5883] Improve count() queries in a few places (#6532)
• [AIRFLOW-5811] Add metric for externally killed task count (#6466)
• [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook (#6432)
• [AIRFLOW-5766] Use httpbin.org in http_default (#6438)
• [AIRFLOW-5798] Set default ExternalTaskSensor.external_task_id (#6431)
• [AIRFLOW-5643] Reduce duplicated logic in S3Hook (#6313)
• [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. (#6199)
• [AIRFLOW-6192] Stop creating Hook from SFTPSensor.init (#6748)
• [AIRFLOW-5749][AIRFLOW-4162] Support the "blocks" component for the Slack operators (#6418)
• [AIRFLOW-5693] Support the "blocks" component for the Slack messages (#6364)
• [AIRFLOW-5714] Collect SLA miss emails only from tasks missed SLA (#6384)
• [AIRFLOW-5049] Add validation for src_fmt_configs in bigquery hook (#5671)
• [AIRFLOW-6177] Log DAG processors timeout event at error level, not info (#6731)
• [AIRFLOW-6180] Improve kerberos init in pytest conftest (#6735)
• [AIRFLOW-6159] Change logging level of the heartbeat message to DEBUG (#6716)
• [AIRFLOW-6144] Improve the log message of airflow scheduler (#6710)
• [AIRFLOW-6099] Add host name to task runner log (#6688)
• [AIRFLOW-6045] Error on failed execution of compile_assets (#6640)
• [AIRFLOW-5144] Add confirmation on delete button click (#6745)
• [AIRFLOW-6099] Add host name to task runner log (#6688)
• [AIRFLOW-5915] Add support for the new documentation theme (#6563)
• [AIRFLOW-5897] Allow setting -1 as pool slots value in webserver (#6550)
• [AIRFLOW-5888] Use psycopg2-binary for postgres operations (#6533)
• [AIRFLOW-5870] Allow -1 for pool size and optimise pool query (#6520)

Bug Fixes

• [AIRFLOW-XXX] Bump Jira version to fix issue with async
• [AIRFLOW-XXX] Add encoding to fix Cyrillic output when reading back task logs (#6631)
• [AIRFLOW-5304] Fix extra links in BigQueryOperator with multiple queries (#5906)
• [AIRFLOW-6268] Prevent (expensive) ajax calls on home page when no dags visible (#6839)
• [AIRFLOW-6259] Reset page to 1 with each new search for dags (#6828)
• [AIRFLOW-6185] SQLAlchemy Connection model schema not aligned with Alembic schema (#6754)
• [AIRFLOW-3632] Only replace microseconds if execution_date is None in trigger_dag REST API (#6380)
• [AIRFLOW-5458] Bump Flask-AppBuilder to 2.2.0 (for Python >= 3.6) (#6607)
• [AIRFLOW-5072] gcs_hook should download files once (#5685)
• [AIRFLOW-5744] Environment variables not correctly set in Spark submit operator (#6796)
• [AIRFLOW-3189] Remove schema from DbHook.get_uri response if None (#6833)
• [AIRFLOW-6195] Fixed TaskInstance attrs not correct on UI (#6758)
• [AIRFLOW-5889] Make polling for AWS Batch job status more resillient (#6765)
• [AIRFLOW-6043] Fix bug in UI when "filtering by root" to display section of dag (#6638)
• [AIRFLOW-6033] Fix UI Crash at "Landing Times" when task_id is changed (#6635)
• [AIRFLOW-3745] Fix viewer not able to view dag details (#4569)
• [AIRFLOW-6175] Fixes bug when tasks get stuck in "scheduled" state (#6732)
• [AIRFLOW-5463] Make Variable.set when replacing an atomic operation (#6807)
• [AIRFLOW-5582] Add get_autocommit to JdbcHook (#6232)
• [AIRFLOW-5867] Fix webserver unit_test_mode data type (#6517)
• [AIRFLOW-5819] Update AWSBatchOperator default value (#6473)
• [AIRFLOW-5709] Fix regression in setting custom operator resources. (#6331)
• [AIRFLOW-5658] Fix broken navigation links (#6374)
• [AIRFLOW-5727] SqoopHook: Build --connect parameter only if port/schema are defined (#6397)
• [AIRFLOW-5695] use RUNNING_DEPS to check run from UI (#6367)
• [AIRFLOW-6254] obscure conn extra in logs (#6817)
• [AIRFLOW-4824] Add charset handling for SqlAlchemy engine for MySqlHook (#6816)
• [AIRFLOW-6091] Add flushing in execute method for BigQueryCursor (#6683)
• [AIRFLOW-6256] Ensure Jobs table is cleared when resetting DB (#6818)
• [AIRFLOW-5224] Add encoding parameter to GoogleCloudStorageToBigQuery (#6297)
• [AIRFLOW-5179] Remove top level init.py (#5818)
• [AIRFLOW-5660] Attempt to find the task in DB from Kubernetes pod labels (#6340)
• [AIRFLOW-6241] Fix typo in airflow/gcp/operator/dataflow.py (#6806)
• [AIRFLOW-6171] Apply .airflowignore to correct subdirectories (#6784)
• [AIRFLOW-6018] Display task instance in table during backfilling (#6612)
• [AIRFLOW-6189] Reduce the maximum test duration to 8 minutes (#6744)
• [AIRFLOW-6141] Remove ReadyToRescheduleDep if sensor mode == poke (#6704)
• [AIRFLOW-6054] Add a command that starts the database consoles (#6653)
• [AIRFLOW-6047] Simplify the logging configuration template (#6644)
• [AIRFLOW-6017] Exclude PULL_REQUEST_TEMPLATE.md from RAT check (#6611)
• [AIRFLOW-4560] Fix Tez queue parameter name in mapred_queue (#5315)
• [AIRFLOW-2143] Fix TaskTries graph counts off-by-1 (#6526)
• [AIRFLOW-5873] KubernetesPodOperator fixes and test (#6523)
• [AIRFLOW-5869] BugFix: Some Deserialized tasks have no start_date (#6519)
• [AIRFLOW-4020] Remove DAG edit permissions from Viewer role (#4845)
• [AIRFLOW-6263] Fix broken WinRM integration (#6832)
• [AIRFLOW-5836] Pin azure-storage-blob version to <12 (#6486)
• [AIRFLOW-4488] Fix typo for non-RBAC UI in max_active_runs_per_dag (#6778)
• [AIRFLOW-5942] Pin PyMSSQL to <3.0 (#6592)
• [AIRFLOW-5451] SparkSubmitHook don't set default namespace (#6072)
• [AIRFLOW-6271] Printing log files read during load_test_config (#6842)
• [AIRFLOW-6308] Unpin Kombu for Python 3

Misc/Internal

• [AIRFLOW-6009] Switch off travis_wait for regular tests (#6600)
• [AIRFLOW-6226] Always reset warnings in tests
• [AIRFLOW-XXX] Remove cyclic imports and pylint hacks in Serialization (#6601)
• [AIRFLOW-XXX] Bump npm from 6.4.1 to 6.13.4 in /airflow/www (#6815)
• [AIRFLOW-XXX] Remove executable permission from file
• [AIRFLOW-XXX] Group AWS & Azure dependencies (old [emr] etc. extra still work)
• [AIRFLOW-5487] Fix unused warning var (#6111)
• [AIRFLOW-5925] Relax funcsigs and psutil version requirements (#6580)
• [AIRFLOW-...

Airflow 1.10.6rc1, 2019-10-18

New Features

• [AIRFLOW-4908] Implement BigQuery Hooks/Operators for update_dataset, patch_dataset and get_dataset (#5546)
• [AIRFLOW-4741] Optionally report task errors to Sentry (#5407)
• [AIRFLOW-4939] Add default_task_retries config (#5570)
• [AIRFLOW-5508] Add config setting to limit which StatsD metrics are emitted (#6130)
• [AIRFLOW-4222] Add cli autocomplete for bash & zsh (#5789)
• [AIRFLOW-3871] Operators template fields can now render fields inside objects (#4743)

Improvements

• [AIRFLOW-5127] Gzip support for CassandraToGoogleCloudStorageOperator (#5738)
• [AIRFLOW-5125] Add gzip support for AdlsToGoogleCloudStorageOperator (#5737)
• [AIRFLOW-5124] Add gzip support for S3ToGoogleCloudStorageOperator (#5736)
• [AIRFLOW-5653] Log AirflowSkipException in task instance log to make it clearer why tasks might be skipped (#6330)
• [AIRFLOW-5343] Remove legacy SQLAlchmey pessimistic pool disconnect handling (#6034)
• [AIRFLOW-5561] Relax httplib2 version required for gcp extra (#6194)
• [AIRFLOW-5657] Update the upper bound for dill dependency (#6334)
• [AIRFLOW-5292] Allow ECSOperator to tag tasks (#5891)
• [AIRFLOW-4939] Simplify Code for Default Task Retries (#6233)
• [AIRFLOW-5126] Read aws_session_token in extra_config of the aws hook (#6303)
• [AIRFLOW-5636] Allow adding or overriding existing Operator Links (#6302)
• [AIRFLOW-4965] Handle quote exceptions in GCP AI operators (v1.10) (#6304)
• [AIRFLOW-3783] Speed up Redshift to S3 UNload with HEADERs (#6309)
• [AIRFLOW-3388] Add support to Array Jobs for AWS Batch Operator (#6153)
• [AIRFLOW-4574] add option to provide private_key in SSHHook (#6104) (#6163)
• [AIRFLOW-5530] Fix typo in AWS SQS sensors (#6012)
• [AIRFLOW-5445] Reduce the required resources for the Kubernetes's sidecar (#6062)
• [AIRFLOW-5443] Use alpine image in Kubernetes's sidecar (#6059)
• [AIRFLOW-5344] Add --proxy-user parameter to SparkSubmitOperator (#5948)
• [AIRFLOW-3888] HA for Hive metastore connection (#4708)
• [AIRFLOW-5269] Reuse session in Scheduler Job from health endpoint (#5873)
• [AIRFLOW-5153] Option to force delete non-empty BQ datasets (#5768)
• [AIRFLOW-4443] Document LatestOnly behavior for external trigger (#5214)
• [AIRFLOW-2891] Make DockerOperator container_name be templateable (#5696)
• [AIRFLOW-2891] allow configurable docker_operator container name (#5689)
• [AIRFLOW-4285] Update task dependency context definition and usage (#5079)
• [AIRFLOW-5142] Fixed flaky Cassandra test (#5758)
• [AIRFLOW-5218] Less polling of AWS Batch job status (#5825)
• [AIRFLOW-4956] Fix LocalTaskJob heartbeat log spamming (#5589)
• [AIRFLOW-3160] Load latest_dagruns asynchronously on home page (#5339)
• [AIRFLOW-5560] Allow no confirmation on reset dags in airflow backfill command (#6195)
• [AIRFLOW-5280] conn: Remove aws_default's default region name (#5879)
• [AIRFLOW-5528] end_of_log_mark should not be a log record (#6159)
• [AIRFLOW-5526] Update docs configuration due to migration of GCP docs (#6154)
• [AIRFLOW-4835] Refactor operator render_template (#5461)

Bug Fixes

• [AIRFLOW-5459] Use a dynamic tmp location in Dataflow operator (#6078)
• [Airflow 4923] Fix Databricks hook leaks API secret in logs (#5635)
• [AIRFLOW-5133] Keep original env state in provide_gcp_credential_file (#5747)
• [AIRFLOW-5497] Update docstring in airflow/utils/dag_processing.py (#6314)
• Revert/and then rework "[AIRFLOW-4797] Improve performance and behaviour of zombie detection (#5511)" to improve performance (#5908)
• [AIRFLOW-5634] Don't allow editing of DagModelView (#6308)
• [AIRFLOW-4309] Remove Broken Dag error after Dag is deleted (#6102)
• [AIRFLOW-5387] Fix "show paused" pagination bug (#6100)
• [AIRFLOW-5489] Remove unneeded assignment of variable (#6106)
• [AIRFLOW-5491] mark_tasks pydoc is incorrect (#6108)
• [AIRFLOW-5492] added missing docstrings (#6107)
• [AIRFLOW-5503] Fix tree view layout on HDPI screen (#6125)
• [AIRFLOW-5481] Allow Deleting Renamed DAGs (#6101)
• [AIRFLOW-3857] spark_submit_hook cannot kill driver pod in Kubernetes (#4678)
• [AIRFLOW-4391] Fix tooltip for None-State Tasks in 'Recent Tasks' (#5909)
• [AIRFLOW-5554] Require statsd 3.3.0 minimum (#6185)
• [AIRFLOW-5306] Fix the display of links when they contain special characters (#5904)
• [AIRFLOW-3705] Fix PostgresHook get_conn to use conn_name_attr (#5841)
• [AIRFLOW-5581] Cleanly shutdown KubernetesJobWatcher for safe Scheduler shutdown on SIGTERM (#6237)
• [AIRFLOW-5634] Don't allow disabled fields to be edited in DagModelView (#6307)
• [AIRFLOW-4833] Allow to set Jinja env options in DAG declaration (#5943)
• [AIRFLOW-5408] Fix env variable name in Kubernetes template (#6016)
• [AIRFLOW-5102] Worker jobs should terminate themselves if they can't heartbeat (#6284)
• [AIRFLOW-5572] Clear task reschedules when clearing task instances (#6217)
• [AIRFLOW-5543] Fix tooltip disappears in tree and graph view (RBAC UI) (#6174)
• [AIRFLOW-5444] Fix action_logging so that request.form for POST is logged (#6064)
• [AIRFLOW-5484] fix PigCliHook has incorrect named parameter (#6112)
• [AIRFLOW-5342] Fix MSSQL breaking task_instance db migration (#6014)
• [AIRFLOW-5556] Add separate config for timeout from scheduler dag processing (#6186)
• [AIRFLOW-4858] Deprecate "Historical convenience functions" in airflow.configuration (#5495) (#6144)
• [AIRFLOW-774] Fix long-broken DAG parsing Statsd metrics (#6157)
• [AIRFLOW-5419] Use sudo to kill cleared tasks when running with impersonation (#6026) (#6176)
• [AIRFLOW-5537] Yamllint is not needed as dependency on host
• [AIRFLOW-5536] Better handling of temporary output files
• [AIRFLOW-5535] Fix name of VERBOSE parameter
• [AIRFLOW-5519] Fix sql_to_gcs operator missing multi-level default args by adding apply_defaults decorator (#6146)
• [AIRFLOW-5210] Make finding template files more efficient (#5815)
• [AIRFLOW-5447] Scheduler stalls because second watcher thread in default args (#6129)

Doc-only changes

• [AIRFLOW-5574] Fix Google Analytics script loading (#6218)
• [AIRFLOW-5588] Add Celery's architecture diagram (#6247)
• [AIRFLOW-5521] Fix link to GCP documentation (#6150)
• [AIRFLOW-5398] Update contrib example DAGs to context manager (#5998)
• [AIRFLOW-5268] Apply same DAG naming conventions as in literature (#5874)
• [AIRFLOW-5101] Fix inconsistent owner value in examples (#5712)
• [AIRFLOW-XXX] Fix typo - AWS DynamoDB Hook (#6319)
• [AIRFLOW-XXX] Fix Documentation for adding extra Operator Links (#6301)
• [AIRFLOW-XXX] Add section on task lifecycle & correct casing in docs (#4681)
• [AIRFLOW-XXX] Make it clear that 1.10.5 wasn't accidentally omitted from UPDATING.md (#6240)
• [AIRFLOW-XXX] Improve format in code-block directives (#6242)
• [AIRFLOW-XXX] Format Sendgrid docs (#6245)
• [AIRFLOW-XXX] Update to new logo (#6066)
• [AIRFLOW-XXX] Typo in FAQ - schedule_interval (#6291)
• [AIRFLOW-XXX] Add message about breaking change in DAG#get_task_instances in 1.10.4 (#6226)
• [AIRFLOW-XXX] Fix incorrect units in docs for metrics using Timers (#6152)
• [AIRFLOW-XXX] Fix backtick issues in .rst files & Add Precommit hook (#6162)
• [AIRFLOW-XXX] Update documentation about variables forcing answer (#6158)
• [AIRFLOW-XXX] Add a third way to configure authorization (#6134)
• [AIRFLOW-XXX] Add example of running pre-commit hooks on single file (#6143)
• [AIRFLOW-XXX] Add information about default pool to docs (#6019)
• [AIRFLOW-XXX] Make Breeze The default integration test environment (#6001)

Misc/Internal

• [AIRFLOW-5687] Upgrade pip to 19.0.2 in CI build pipeline (#6358) (#6361)
• [AIRFLOW-5533] Fixed failing CRON build (#6167)
• [AIRFLOW-5130] Use GOOGLE_APPLICATION_CREDENTIALS constant from library (#5744)
• [AIRFLOW-5369] Adds interactivity to pre-commits (#5976)
• [AIRFLOW-5531] Replace deprecated log.warn() with log.warning() (#6165)
• [AIRFLOW-4686] Make dags Pylint compatible (#5753)
• [AIRFLOW-4864] Remove calls to load_test_config (#5502)
• [AIRFLOW-XXX] Pin version of mypy so we are stable over time (#6198)
• [AIRFLOW-XXX] Add tests that got missed from #5127
• [AIRFLOW-4928] Move config parses to class properties inside DagBag (#5557)
• [AIRFLOW-5003] Making AWS Hooks pylint compatible (#5627)
• [AIRFLOW-5580] Add base class for system test (#6229)

Airflow 1.10.5, 2019-08-30

New Features

• [AIRFLOW-1498] Add feature for users to add Google Analytics to Airflow UI (#5850)
• [AIRFLOW-4074] Add option to add labels to Dataproc jobs (#5606)
• [AIRFLOW-4846] Allow specification of an existing secret containing git credentials for init containers (#5475)

Improvements

• [AIRFLOW-5335] Update GCSHook methods so they need min IAM perms (#5939)
• [AIRFLOW-2692] Allow AWS Batch Operator to use templates in job_name parameter (#3557)
• [AIRFLOW-4768] Add Timeout parameter in example_gcp_video_intelligence (#5862)
• [AIRFLOW-5165] Make Dataproc highly available (#5781)
• [AIRFLOW-5139] Allow custom ES configs (#5760)
• [AIRFLOW-5340] Fix GCP DLP example (#594)
• [AIRFLOW-5211] Add pass_value to template_fields BigQueryValueCheckOperator (#5816)
• [AIRFLOW-5113] Support icon url in slack web hook (#5724)
• [AIRFLOW-4230] bigquery schema update options should be a list (#5766)
• [AIRFLOW-1523] Clicking on Graph View should display related DAG run (#5866)
• [AIRFLOW-5027] Generalized CloudWatch log grabbing for ECS and SageMaker operators (#5645)
• [AIRFLOW-5244] Add all possible themes to default_webserver_config.py (#5849)
• [AIRFLOW-5245] Add more metrics around the scheduler (#5853)
• [AIRFLOW-5048] Improve display of Kubernetes resources (#5665)
• [AIRFLOW-5284] Replace deprecated log.warn by log.warning (#5881)
• [AIRFLOW-5276] Remove unused helpers from airflow.utils.helpers (#5878)
• [AIRFLOW-4316] Support setting kubernetes_environment_variables config section from env var (#5668)

Bug fixes

• [AIRFLOW-5168] Fix Dataproc operators that failed in 1.10.4 (#5928)
• [AIRFLOW-5136] Fix Bug with Incorrect template_fields in DataProc{*} Operators (#5751)
• [AIRFLOW-5169] Pass GCP Project ID explicitly to StorageClient in GCSHook (#5783)
• [AIRFLOW-5302] Fix bug in none_skipped Trigger Rule (#5902)
• [AIRFLOW-5350] Fix bug in the num_retires field in BigQueryHook (#5955)
• [AIRFLOW-5145] Fix rbac ui presents false choice to encrypt or not encrypt variable values (#5761)
• [AIRFLOW-5104] Set default schedule for GCP Transfer operators (#5726)
• [AIRFLOW-4462] Use datetime2 column types when using MSSQL backend (#5707)
• [AIRFLOW-5282] Add default timeout on kubeclient & catch HTTPError (#5880)
• [AIRFLOW-5315] TaskInstance not updating from DB when user changes executor_config (#5926)
• [AIRFLOW-4013] Mark success/failed is picking all execution date (#5616)
• [AIRFLOW-5152] Fix autodetect default value in GoogleCloudStorageToBigQueryOperator(#5771)
• [AIRFLOW-5100] Airflow scheduler does not respect safe mode setting (#5757)
• [AIRFLOW-4763] Allow list in DockerOperator.command (#5408)
• [AIRFLOW-5260] Allow empty uri arguments in connection strings (#5855)
• [AIRFLOW-5257] Fix ElasticSearch log handler errors when attemping to close logs (#5863)
• [AIRFLOW-1772] Google Updated Sensor doesnt work with CRON expressions (#5730)
• [AIRFLOW-5085] When you run kubernetes git-sync test from TAG, it fails (#5699)
• [AIRFLOW-5258] ElasticSearch log handler, has 2 times of hours (%H and %I) in _clean_execution_dat (#5864)
• [AIRFLOW-5348] Escape Label in deprecated chart view when set via JS (#5952)
• [AIRFLOW-5357] Fix Content-Type for exported variables.json file (#5963)
• [AIRFLOW-5109] Fix process races when killing processes (#5721)
• [AIRFLOW-5240] Latest version of Kombu is breaking airflow for py2

Misc/Internal

• [AIRFLOW-5111] Remove apt-get upgrade from the Dockerfile (#5722)
• [AIRFLOW-5209] Fix Documentation build (#5814)
• [AIRFLOW-5083] Check licence image building can be faster and moved to before-install (#5695)
• [AIRFLOW-5119] Cron job should always rebuild everything from scratch (#5733)
• [AIRFLOW-5108] In the CI local environment long-running kerberos might fail sometimes (#5719)
• [AIRFLOW-5092] Latest python image should be pulled locally in force_pull_and_build (#5705)
• [AIRFLOW-5225] Consistent licences can be added automatically for all JS files (#5827)
• [AIRFLOW-5229] Add licence to all other file types (#5831)
• [AIRFLOW-5227] Consistent licences for all .sql files (#5829)
• [AIRFLOW-5161] Add pre-commit hooks to run static checks for only changed files (#5777)
• [AIRFLOW-5159] Optimise checklicence image build (do not build if not needed) (#5774)
• [AIRFLOW-5263] Show diff on failure of pre-commit checks (#5869)
• [AIRFLOW-5204] Shell files should be checked with shellcheck and have identical licence (#5807)
• [AIRFLOW-5233] Check for consistency in whitespace (tabs/eols) and common problems (#5835)
• [AIRFLOW-5247] Getting all dependencies from NPM can be moved up in Dockerfile (#5870)
• [AIRFLOW-5143] Corrupted rat.jar became part of the Docker image (#5759)
• [AIRFLOW-5226] Consistent licences for all html JINJA templates (#5828)
• [AIRFLOW-5051] Coverage is not properly reported in the new CI system (#5732)
• [AIRFLOW-5239] Small typo and incorrect tests in CONTRIBUTING.md (#5844)
• [AIRFLOW-5287] Checklicence base image is not pulled (#5886)
• [AIRFLOW-5301] Some not-yet-available files from breeze are committed to master (#5901)
• [AIRFLOW-5285] Pre-commit pylint runs over todo files (#5884)
• [AIRFLOW-5288] Temporary container for static checks should be auto-removed (#5887)
• [AIRFLOW-5326] Fix teething proglems for Airflow breeze (#5933)
• [AIRFLOW-5206] All .md files should have all common licence, TOC (where applicable) (#5809)
• [AIRFLOW-5329] Easy way to add local files to docker (#5933)
• [AIRFLOW-4027] Make experimental api tests more stateless (#4854)

Doc-only changes

• [AIRFLOW-XXX] Fixed Azkaban link (#5865)
• [AIRFLOW-XXX] Remove duplicate lines from CONTRIBUTING.md (#5830)
• [AIRFLOW-XXX] Fix incorrect docstring parameter in SchedulerJob (#5729)

1.10.4rc3

[AIRFLOW-XXX] Update changelog for 1.10.4rc3

1.10.3

New Feature

• [AIRFLOW-4232] Add none_skipped trigger rule (#5032)
• [AIRFLOW-3971] Add Google Cloud Natural Language operators (#4980)
• [AIRFLOW-4069] Add Opsgenie Alert Hook and Operator (#4903)
• [AIRFLOW-3552] Fix encoding issue in ImapAttachmentToS3Operator (#5040)
• [AIRFLOW-3552] Add ImapAttachmentToS3Operator (#4476)
• [AIRFLOW-1526] Add dingding hook and operator (#4895)
• [AIRFLOW-3490] Add BigQueryHook's Ability to Patch Table/View (#4299)
• [AIRFLOW-3918] Add ssh private-key support to git-sync for KubernetesExecutor (#4777)
• [AIRFLOW-3659] Create Google Cloud Transfer Service Operators (#4792)
• [AIRFLOW-3939] Add Google Cloud Translate operator (#4755)
• [AIRFLOW-3541] Add Avro logical type conversion to bigquery hook (#4553)
• [AIRFLOW-4106] instrument staving tasks in pool (#4927)
• [AIRFLOW-2568] Azure Container Instances operator (#4121)
• [AIRFLOW-4107] instrument executor (#4928)
• [AIRFLOW-4033] record stats of task duration (#4858)
• [AIRFLOW-3892] Create Redis pub sub sensor (#4712)
• [AIRFLOW-4124] add get_table and get_table_location in aws_glue_hook and tests (#4942)
• [AIRFLOW-1262] Adds missing docs for email configuration (#4557)
• [AIRFLOW-3701] Add Google Cloud Vision Product Search operators (#4665)
• [AIRFLOW-3766] Add support for kubernetes annotations (#4589)
• [AIRFLOW-3741] Add extra config to Oracle hook (#4584)
• [AIRFLOW-1262] Allow configuration of email alert subject and body (#2338)
• [AIRFLOW-2985] Operators for S3 object copying/deleting (#3823)
• [AIRFLOW-2993] s3_to_sftp and sftp_to_s3 operators (#3828)
• [AIRFLOW-3799] Add compose method to GoogleCloudStorageHook (#4641)
• [AIRFLOW-3218] add support for poking a whole DAG (#4058)
• [AIRFLOW-3315] Add ImapAttachmentSensor (#4161)
• [AIRFLOW-3556] Add cross join set dependency function (#4356)

Improvement

• [AIRFLOW-3823] Exclude branch's downstream tasks from the tasks to skip (#4666)
• [AIRFLOW-3274] Add run_as_user and fs_group options for Kubernetes (#4648)
• [AIRFLOW-4247] Template Region on the DataprocOperators (#5046)
• [AIRFLOW-4008] add envFrom for Kubernetes Executor (#4952)
• [AIRFLOW-3947] Flash msg for no DAG-level access error (#4767)
• [AIRFLOW-3287] Moving database clean-up code into the CoreTest.tearDown() (#4122)
• [AIRFLOW-4058] Name models test file to get automatically picked up (#4901)
• [AIRFLOW-3830] Remove DagBag from /dag_details (#4831)
• [AIRFLOW-3596] Clean up undefined template variables. (#4401)
• [AIRFLOW-3573] Remove DagStat table (#4378)
• [AIRFLOW-3623] Fix bugs in Download task logs (#5005)
• [AIRFLOW-4173] Improve SchedulerJob.process_file() (#4993)
• [AIRFLOW-3540] Warn if old airflow.cfg file is found (#5006)
• [AIRFLOW-4000] Return response when no file (#4822)
• [AIRFLOW-3383] Rotate fernet keys. (#4225)
• [AIRFLOW-3003] Pull the krb5 image instead of building (#3844)
• [AIRFLOW-3862] Check types with mypy. (#4685)
• [AIRFLOW-251] Add option SQL_ALCHEMY_SCHEMA parameter to specify schema for metadata (#4199)
• [AIRFLOW-1814] Temple PythonOperator {op_args,op_kwargs} fields (#4691)
• [AIRFLOW-3730] Standarization use of logs mechanisms (#4556)
• [AIRFLOW-3770] Validation of documentation on CI] (#4593)
• [AIRFLOW-3866] Run docker-compose pull silently in CI (#4688)
• [AIRFLOW-3685] Move licence header check (#4497)
• [AIRFLOW-3670] Add stages to Travis build (#4477)
• [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef and secretRef (#4772)
• [AIRFLOW-3408] Remove outdated info from Systemd Instructions (#4269)
• [AIRFLOW-3202] add missing documentation for AWS hooks/operator (#4048)
• [AIRFLOW-3908] Add more Google Cloud Vision operators (#4791)
• [AIRFLOW-2915] Add example DAG for GoogleCloudStorageToBigQueryOperator (#3763)
• [AIRFLOW-3062] Add Qubole in integration docs (#3946)
• [AIRFLOW-3288] Add SNS integration (#4123)
• [AIRFLOW-3148] Remove unnecessary arg "parameters" in RedshiftToS3Transfer (#3995)
• [AIRFLOW-3049] Add extra operations for Mongo hook (#3890)
• [AIRFLOW-3559] Add missing options to DatadogHook. (#4362)
• [AIRFLOW-1191] Simplify override of spark submit command. (#4360)
• [AIRFLOW-3155] Add ability to filter by a last modified time in GCS Operator (#4008)
• [AIRFLOW-2864] Fix docstrings for SubDagOperator (#3712)
• [AIRFLOW-4062] Improve docs on install extra package commands (#4966)
• [AIRFLOW-3743] Unify different methods of working out AIRFLOW_HOME (#4705)
• [AIRFLOW-4002] Option to open debugger on errors in airflow test. (#4828)
• [AIRFLOW-3997] Extend Variable.get so it can return None when var not found (#4819)
• [AIRFLOW-4009] Fix docstring issue in GCSToBQOperator (#4836)
• [AIRFLOW-3980] Unify logger (#4804)
• [AIRFLOW-4076] Correct port type of beeline_default in init_db (#4908)
• [AIRFLOW-4046] Add validations for poke_interval & timeout for Sensor (#4878)
• [AIRFLOW-3744] Abandon the use of obsolete aliases of methods (#4568)
• [AIRFLOW-3865] Add API endpoint to get python code of dag by id (#4687)
• [AIRFLOW-3516] Support to create k8 worker pods in batches (#4434)
• [AIRFLOW-2843] Add flag in ExternalTaskSensor to check if external DAG/task exists (#4547)
• [AIRFLOW-2224] Add support CSV files in MySqlToGoogleCloudStorageOperator (#4738)
• [AIRFLOW-3895] GoogleCloudStorageHook/Op create_bucket takes optional resource params (#4717)
• [AIRFLOW-3950] Improve AirflowSecurityManager.update_admin_perm_view (#4774)
• [AIRFLOW-4006] Make better use of Set in AirflowSecurityManager (#4833)
• [AIRFLOW-3917] Specify alternate kube config file/context when running out of cluster (#4859)
• [AIRFLOW-3911] Change Harvesting DAG parsing results to DEBUG log level (#4729)
• [AIRFLOW-3584] Use ORM DAGs for index view. (#4390)
• [AIRFLOW-2821] Refine Doc "Plugins" (#3664)
• [AIRFLOW-3561] Improve queries (#4368)
• [AIRFLOW-3600] Remove dagbag from trigger (#4407)
• [AIRFLOW-3713] Updated documentation for GCP optional project_id (#4541)
• [AIRFLOW-2767] Upgrade gunicorn to 19.5.0 to avoid moderate-severity CVE (#4795)
• [AIRFLOW-3795] provide_context param is now used (#4735)
• [AIRFLOW-4012] Upgrade tabulate to 0.8.3 (#4838)
• [AIRFLOW-3623] Support download logs by attempts from UI (#4425)
• [AIRFLOW-2715] Use region setting when launching Dataflow templates (#4139)
• [AIRFLOW-3932] Update unit tests and documentation for safe mode flag. (#4760)
• [AIRFLOW-3932] Optionally skip dag discovery heuristic. (#4746)
• [AIRFLOW-3258] K8S executor environment variables section. (#4627)
• [AIRFLOW-3931] set network, subnetwork when launching dataflow template (#4744)
• [AIRFLOW-4095] Add template_fields for S3CopyObjectOperator & S3DeleteObjectsOperator (#4920)
• [AIRFLOW-2798] Remove needless code from models.py
• [AIRFLOW-3731] Constrain mysqlclient to <1.4 (#4558)
• [AIRFLOW-3139] include parameters into log.info in SQL operators, if any (#3986)
• [AIRFLOW-3174] Refine Docstring for SQL Operators & Hooks (#4043)
• [AIRFLOW-3933] Fix various typos (#4747)
• [AIRFLOW-3905] Allow using "parameters" in SqlSensor (#4723)
• [AIRFLOW-2761] Parallelize enqueue in celery executor (#4234)
• [AIRFLOW-3540] Respect environment config when looking up config file. (#4340)
• [AIRFLOW-2156] Parallelize Celery Executor task state fetching (#3830)
• [AIRFLOW-3702] Add backfill option to run backwards (#4676)
• [AIRFLOW-3821] Add replicas logic to GCP SQL example DAG (#4662)
• [AIRFLOW-3547] Fixed Jinja templating in SparkSubmitOperator (#4347)
• [AIRFLOW-3647] Add archives config option to SparkSubmitOperator (#4467)
• [AIRFLOW-3802] Updated documentation for HiveServer2Hook (#4647)
• [AIRFLOW-3817] Corrected task ids returned by BranchPythonOperator to match the dummy operator ids (#4659)
• [AIRFLOW-3782] Clarify docs around celery worker_autoscale in default_airflow.cfg (#4609)
• [AIRFLOW-1945] Add Autoscale config for Celery workers (#3989)
• [AIRFLOW-3590] Change log message of executor exit status (#4616)
• [AIRFLOW-3591] Fix start date, end date, duration for rescheduled tasks (#4502)
• [AIRFLOW-3709] Validate allowed_states for ExternalTaskSensor (#4536)
• [AIRFLOW-3522] Add support for sending Slack attachments (#4332)
• [AIRFLOW-3569] Add "Trigger DAG" button in DAG page (/www only) (#4373)
• [AIRFLOW-3569] Add "Trigger DAG" button in DAG page (/www_rbac only) (#4373)
• [AIRFLOW-3044] Dataflow operators accept templated job_name param (#3887)
• [AIRFLOW-3023] Fix docstring datatypes
• [AIRFLOW-2928] Use uuid4 instead of uuid1 (#3779)
• [AIRFLOW-2988] Run specifically python2 for dataflow (#3826)
• [AIRFLOW-3697] Vendorize nvd3 and slugify (#4513)
• [AIRFLOW-3692] Remove ENV variables to avoid GPL (#4506)
• [AIRFLOW-3907] Upgrade flask and set cookie security flags. (#4725)
• [AIRFLOW-3698] Add documentation for AWS Connection (#4514)
• [AIRFLOW-3616][AIRFLOW-1215] Add aliases for schema with underscore (#4523)
• [AIRFLOW-3375] Support returning multiple tasks with BranchPythonOperator (#4215)
• [AIRFLOW-3742] Fix handling of "fallback" for AirflowConfigParsxer.getint/boolean (#4674)
• [AIRFLOW-3742] Respect the fallback arg in airflow.configuration.get (#4567)
• [AIRFLOW-3789] Fix flake8 3.7 errors. (#4617)
• [AIRFLOW-3602] Improve ImapHook handling of retrieving no attachments (#4475)
• [AIRFLOW-3631] Update flake8 and fix lint. (#4436)

Bug fixes

• [AIRFLOW-4248] Fix 'FileExistsError' makedirs race in file_processor_handler (#5047)
• [AIRFLOW-4240] State-changing actions should be POST requests (#5039)
• [AIRFLOW-4246] Flask-Oauthlib needs downstream dependencies pinning due to breaking changes (#5045)
• [AIRFLOW-3887] Downgrade dagre-d3 to 0.4.18 (#4713)
• [AIRFLOW-3419] Fix S3Hook.select_key on Python3 (#4970)
• [AIRFLOW-4127] Correct AzureContainerInstanceHook._get_instance_view's return (#4945)
• [AIRFLOW-4172] Fix changes for driver class path option in Spark Submi...

1.7.0

Raw changelog

for the record, generated with:
~/node_modules/github-changes/bin/index.js -o airbnb -r airflow --only-pulls --use-commit-body --between-tags 1.6.2...1.7.0rc1 --token {your_gh_token_here}

1.7.0

• #1147 Enhance CLI Test command to accept a JSON-formatted dictionary of par… (@r39132)
• #975 statuses column on /admin shows only active or most recent dag_runs (@jtschoonhoven)
• #1127 FAQ entry about start_date (@mistercrunch)
• #1129 Showing active dag runs as in (3/16) in tooltip (@mistercrunch)
• #1144 More explicit zindex on modals (@mistercrunch)
• #1143 small fixes to previous bq project inclusion pr. (@mtagle)
• #1139 Allow specificiation of project in BigQuery Hook methods (@mtagle)
• #1110 Add date support to MySQL to GCS operator (@criccomini)
• #1135 Added start_date initialization for DagRun creation within schedule_dag(self, dag_id) (@RvN76)
• #1140 License check (@bolkedebruin)
• #1138 Add support for BigQuery User Defined Functions in BigQuery operator (@LeBlanc)
• #1132 Add custom email backends. (@jmcarp)
• #1119 Add gcloud-based GCSHook (@jlowin)
• #1134 Parameterizing DagBag import timeouts (@mistercrunch)
• #1045 Add startup scripts for upstart based systems (@d-lee)
• #1131 Adding ssh connection type to webform (@hyperborea)
• #792 add error handling for slack api (@abridgett)
• #1115 This patch adds license checking for Airflow. (@bolkedebruin)
• #1116 Add WePay and committer list to README.md (@criccomini)
• #1118 Don't force installation of GCP API client dependencies (@jlowin)
• #1090 Adding template support in qbol operator (@msumit)
• #1108 new company + link to pitfalls (@kretes)
• #1104 Add INT24 (MEDIUMINT) support to MySQL to Google cloud storage operator (@criccomini)
• #1076 SubDAG docs and examples (@nicktrav)
• #1101 modify datastore hook so that authorization is maintained for the lif… (@mtagle)
• #1099 Adding @mention in Airbnb and links to airbnb open source projects (@artwr)
• #1096 Make SqlAlchemy pool_recycle and pool_size configurable (@amread)
• #1070 Proper sqlalchemy syntax for desc (@mistercrunch)
• #1079 SID Oracle DB connection support (@biln)
• #1094 Adding to inits (@artwr)
• #1093 Add MySQL to BQ support for TINYINT (@criccomini)
• #1088 Fix MySQL to Google cloud storage scoping. (@criccomini)
• #1075 Pig hook and operator stub (@artwr)
• #1084 Replace deprecated flask.ext.* with flask_* (@jeffwidman)
• #1082 Cleanup Contributing.md (@jeffwidman)
• #1086 Default to 0 if no rows loaded in GCS to BQ operator. (@criccomini)
• #1080 Add MySQL->GCS, GCS->BQ operators (@criccomini)
• #1073 Fixing small issue with qbol operator and hook (@msumit)
• #1068 Add a new hook for google datastore (@mtagle)
• #1065 dag pausing should pause queued tasks as well (@mistercrunch)
• #1067 Add two methods to BigQueryBaseCursor: (@mtagle)
• #1064 Add Thumbtack to organizations using Airflow (@natekupp)
• #1063 Documenting task details doc_ feature (@mistercrunch)
• #1049 Allow the use of the autoconfig client (@bolkedebruin)
• #1056 Add output encoding option to BashOperator (@Attsun1031)
• #996 Added the dst filename to template_fields GoogleCloudStorageDownloadOperator (@0xR)
• #1051 Running unit tests with local executor (@mistercrunch)
• #1008 Add FTPSHook class. (@geeknam)
• #1055 Improve [Hide/Show all series] perforamce (@lumengxi)
• #1054 CLI's trigger_dag now accepts --conf as json (@mistercrunch)
• #1053 Add Kogan.com to the list of users (@geeknam)
• #1046 Documenting the cluster policy feature (@mistercrunch)
• #1044 Updating the Readme with a link to the TriggerDagRunOperator post (@r39132)
• #1043 Adding an example to illustrate the TriggerDagRunOperator (@r39132)
• #1039 Only set headers and delimiters for CSV exports in Google BigQuery hook (@criccomini)
• #1026 Minor documentation tweaks to the FAQ under the fernet key section (@r39132)
• #1025 Add notes on connection password encryption (@d-lee)
• #1018 add SSL support for SMTP (@bbrumi)
• #1020 Add BigQuery copy operator. (@criccomini)
• #964 Clean test database out in between unit test runs (@nicktrav)
• #1019 Add direct dependencies for Google cloud contribs (@criccomini)
• #985 [airflow][presto] Gracefully handle 503 errors and avoid eval() (@airbnb)
• #1013 Fixed issue 1012: pool not used with celery executor (@rdavison)
• #1011 drop the tmp table after ingestion (@dayzzz)
• #995 Fix LDAP error messages when login fails. (@criccomini)
• #1006 Fix forgetting to expose yesterday_ds_nodash and tomorrow_ds_nodash (@0xR)
• #1003 Update to the README such as adding Hootsuite (@r39132)
• #998 Pedantic documentation tweaks (@joshmarlow)
• #940 Refresh border coloring in graph view without having to refresh the entire page (@dinocow)
• #987 Add logout button to Airflow (@criccomini)
• #930 When either data_profiler_filter or superuser_filter aren't defined,… (@mtagle)
• #994 there could be dot in the key string, which is illegal in the hive ta… (@dayzzz)
• #993 Added yesterday_ds_nodash and tommorow_ds_nodash (@0xR)
• #992 Rename BashOperator.xcom_push member to avoid collision with parent class (@seregasheypak)
• #978 Fixing conflicting params in default_args (@mistercrunch, @nicktrav)
• #984 Add tests for params handling in Dag construction (@nicktrav)
• #986 [documentation] document that max_active_runs can prevent a DAG from running (@aoen)
• #983 Added destination_dataset_table to template_fieds of bigquery_operator (@0xR)
• #981 Add BigQuery PEP 249 support (@criccomini)
• #980 Support user-defined macros and params in dry-run backfills with task… (@r39132)
• #976 More verbose logging for are_dependencies_met when called from run (@mistercrunch)
• #944 Fix statuses in dag list for dags with dag_ids prefixed by a number (@wil5for)
• #942 Set effective user of (web)hdfs hooks using connection config, optionally overridden by constructor (@xadhoom)
• #965 Adding mock lib to devel extras_require (@mistercrunch)
• #963 Only schedule DagRuns between start and end dates (@nicktrav)
• [#948](https://git...

1.6.2

1.6.2 is mostly a release with bug fixes and a few relatively minor features. Thanks to all contributors!

v1.6.1

• Scheduler bugfix