Update dependency apache-airflow to v2.7.3 [SECURITY] #694
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
⚠ Artifact update problemRenovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is. ♻ Renovate will retry this branch, including artifacts, only when one of the following happens:
The artifact failure details are included below: File name: poetry.lock |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
==2.2.5->==2.7.3GitHub Vulnerability Alerts
CVE-2023-25754
Privilege Context Switching Error vulnerability in Apache Software Foundation Apache Airflow. This issue affects Apache Airflow: before 2.6.0.
CVE-2022-46651
Apache Airflow, versions before 2.6.3, is affected by a vulnerability that allows an unauthorized actor to gain access to sensitive information in Connection edit view. This vulnerability is considered low since it requires someone with access to Connection resources specifically updating the connection to exploit it. Users should upgrade to version 2.6.3 or later which has removed the vulnerability.
CVE-2023-22887
Apache Airflow, versions before 2.6.3, is affected by a vulnerability that allows an attacker to perform unauthorized file access outside the intended directory structure by manipulating the run_id parameter. This vulnerability is considered low since it requires an authenticated user to exploit it. It is recommended to upgrade to a version that is not affected
CVE-2023-35908
Apache Airflow, versions before 2.6.3, is affected by a vulnerability that allows unauthorized read access to a DAG through the URL. It is recommended to upgrade to a version that is not affected
CVE-2023-36543
Apache Airflow, versions before 2.6.3, has a vulnerability where an authenticated user can use crafted input to make the current request hang. It is recommended to upgrade to a version that is not affected
CVE-2023-39508
Execution with Unnecessary Privileges, : Exposure of Sensitive Information to an Unauthorized Actor vulnerability in Apache Software Foundation Apache Airflow.The "Run Task" feature enables authenticated user to bypass some of the restrictions put in place. It allows to execute code in the webserver context as well as allows to bypas limitation of access the user has to certain DAGs. The "Run Task" feature is considered dangerous and it has been removed entirely in Airflow 2.6.0.
This issue affects Apache Airflow: before 2.6.0.
CVE-2023-40273
The session fixation vulnerability allowed the authenticated user to continue accessing Airflow webserver even after the password of the user has been reset by the admin - up until the expiry of the session of the user. Other than manually cleaning the session database (for database session backend), or changing the secure_key and restarting the webserver, there were no mechanisms to force-logout the user (and all other users with that).
With this fix implemented, when using the database session backend, the existing sessions of the user are invalidated when the password of the user is reset. When using the securecookie session backend, the sessions are NOT invalidated and still require changing the secure key and restarting the webserver (and logging out all other users), but the user resetting the password is informed about it with a flash message warning displayed in the UI. Documentation is also updated explaining this behaviour.
Users of Apache Airflow are advised to upgrade to version 2.7.0 or newer to mitigate the risk associated with this vulnerability.
CVE-2023-37379
Apache Airflow, in versions prior to 2.7.0, contains a security vulnerability that can be exploited by an authenticated user possessing Connection edit privileges. This vulnerability allows the user to access connection information and exploit the test connection feature by sending many requests, leading to a denial of service (DoS) condition on the server. Furthermore, malicious actors can leverage this vulnerability to establish harmful connections with the server.
Users of Apache Airflow are strongly advised to upgrade to version 2.7.0 or newer to mitigate the risk associated with this vulnerability. Additionally, administrators are encouraged to review and adjust user permissions to restrict access to sensitive functionalities, reducing the attack surface.
CVE-2023-39441
Apache Airflow SMTP Provider before 1.3.0, Apache Airflow IMAP Provider before 3.3.0, and Apache Airflow before 2.7.0 are affected by the Validation of OpenSSL Certificate vulnerability.
The default SSL context with SSL library did not check a server's X.509 certificate. Instead, the code accepted any certificate, which could result in the disclosure of mail server credentials or mail contents when the client connects to an attacker in a MITM position.
Users are strongly advised to upgrade to Apache Airflow version 2.7.0 or newer, Apache Airflow IMAP Provider version 3.3.0 or newer, and Apache Airflow SMTP Provider version 1.3.0 or newer to mitigate the risk associated with this vulnerability
CVE-2023-40611
Apache Airflow, versions before 2.7.1, is affected by a vulnerability that allows authenticated and DAG-view authorized Users to modify some DAG run detail values when submitting notes. This could have them alter details such as configuration parameters, start date, etc.
Users should upgrade to version 2.7.1 or later which has removed the vulnerability.
CVE-2023-40712
Apache Airflow, versions before 2.7.1, is affected by a vulnerability that allows authenticated users who have access to see the task/dag in the UI, to craft a URL, which could lead to unmasking the secret configuration of the task that otherwise would be masked in the UI.
Users are strongly advised to upgrade to version 2.7.1 or later which has removed the vulnerability.
CVE-2023-42663
Apache Airflow, versions before 2.7.2, has a vulnerability that allows an authorized user with access to read specific DAGs only to read information about task instances in other DAGs. Users of Apache Airflow are advised to upgrade to version 2.7.2 or newer to mitigate the risk associated with this vulnerability.
CVE-2023-42792
Apache Airflow, in versions prior to 2.7.2, contains a security vulnerability that allows an authenticated user with limited access to some DAGs, to craft a request that could give the user write access to various DAG resources for DAGs that the user had no access to, thus, enabling the user to clear DAGs they shouldn't.
Users of Apache Airflow are strongly advised to upgrade to version 2.7.2 or newer to mitigate the risk associated with this vulnerability.
CVE-2023-42780
Apache Airflow, versions prior to 2.7.2, contains a security vulnerability that allows authenticated users of Airflow to list warnings for all DAGs, even if the user had no permission to see those DAGs. It would reveal the dag_ids and the stack-traces of import errors for those DAGs with import errors. Users of Apache Airflow are advised to upgrade to version 2.7.2 or newer to mitigate the risk associated with this vulnerability.
CVE-2023-47037
Apache Airflow, versions before 2.7.3, is affected by a vulnerability that allows authenticated and DAG-view authorized Users to modify some DAG run detail values when submitting notes. This could have them alter details such as configuration parameters, start date, etc. Users should upgrade to version 2.7.3 or later which has removed the vulnerability.
CVE-2023-42781
Apache Airflow, versions before 2.7.3, has a vulnerability that allows an authorized user who has access to read specific DAGs only, to read information about task instances in other DAGs. This is a different issue than CVE-2023-42663 but leading to similar outcome.
Users of Apache Airflow are advised to upgrade to version 2.7.3 or newer to mitigate the risk associated with this vulnerability.
Release Notes
apache/airflow (apache-airflow)
v2.7.3Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
"""""""""
codemirrorand extra (#35122)get_plugin_infofor class based listeners. (#35022)all_skippedtrigger rule asskippedif any task is inupstream_failedstate (#34392)Misc/Internal
"""""""""""""
pendulumrequirement to<3.0(#35336)sentry_sdkto1.33.0(#35298)@babel/traversefrom7.16.0 to 7.23.2in/airflow/www(#34988)undicifrom5.19.1 to 5.26.3in/airflow/www(#34971)SchedulerJobRunner(#34810)max_tis per query > parallelism(#34742)connexion<3.0upper bound (#35218)< 3.12(#35123)3.1.0(#34943)Doc Only Changes
""""""""""""""""
conn.extras(#35165)mysql-connector-pythonfrom recommended MySQL driver (#34287)set_downstreamexample (#35075)airflow_local_settings.pytemplate (#34826)'>'in provider section name (#34813)v2.7.2Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes
Bug Fixes
"""""""""
taskgroupis mapped (#34587)cluster_activityview not loading due tostandaloneDagProcessortemplating (#34274)loglevel=DEBUGin 'Not syncingDAG-levelpermissions' (#34268)access_control={}(#34114)ab_usertable in the CLI session (#34120)next_run_datasets_summaryendpoint (#34143)_run_task_sessionin mappedrender_template_fields(#33309)version_added(#34011)Doc Only Changes
""""""""""""""""
AUTH_REMOTE_USERfrom FAB in WSGI middleware example (#34721)Misc/Internal
"""""""""""""
astroidversion < 3 (#34658)os.path.splitexttoPath.*(#34352, #33669)pyproject.toml(#34014)isinstancein fab_security manager (#33760)isinstancecalls for the same object in a single call (#33767)str.splitlines()to split lines (#33592)len()(#33454)v2.7.1Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
CronTriggerTimetable is now less aggressive when trying to skip a run (#33404)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
When setting
catchup=False, CronTriggerTimetable no longer skips a run ifthe scheduler does not query the timetable immediately after the previous run
has been triggered.
This should not affect scheduling in most cases, but can change the behaviour if
a DAG is paused-unpaused to manually skip a run. Previously, the timetable (with
catchup=False) would only start a run after a DAG is unpaused, but with thischange, the scheduler would try to look at little bit back to schedule the
previous run that covers a part of the period when the DAG was paused. This
means you will need to keep a DAG paused longer (namely, for the entire cron
period to pass) to really skip a run.
Note that this is also the behaviour exhibited by various other cron-based
scheduling tools, such as
anacron.conf.set()becomes case insensitive to matchconf.get()behavior (#33452)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Also,
conf.get()will now break if used with non-string parameters.conf.set(section, key, value)used to be case sensitive, i.e.conf.set("SECTION", "KEY", value)and
conf.set("section", "key", value)were stored as two distinct configurations.This was inconsistent with the behavior of
conf.get(section, key), which was always converting the section and key to lower case.As a result, configuration options set with upper case characters in the section or key were unreachable.
That's why we are now converting section and key to lower case in
conf.settoo.We also changed a bit the behavior of
conf.get(). It used to allow objects that are not strings in the section or key.Doing this will now result in an exception. For instance,
conf.get("section", 123)needs to be replaced withconf.get("section", "123").Bug Fixes
"""""""""
MappedTaskGrouptasks not respecting upstream dependency (#33732)SECURITY_MANAGER_CLASSshould be a reference to class, not a string (#33690)get_url_for_loginin security manager (#33660)2.7.0 dbmigration job errors (#33652)groupbyin TIS duration calculation (#33535)dialect.namein custom SA types (#33503)end_dateis less thanutcnow(#33488)formatDurationmethod (#33486)conf.setcase insensitive (#33452)soft_failargument whenpokeis called (#33401)processor_subdir(#33357)<br>text in Provider's view (#33326)soft_failargument when ExternalTaskSensor runs in deferrable mode (#33196)expand_kwargsmethod (#32272)Misc/Internal
"""""""""""""
Pydantic1 compatibility (#34081, #33998)Pydantic2 (#33956)devel_onlyextra in Airflow's setup.py (#33907)FABto4.3.4in order to fix issues with filters (#33931)sqlalchemy to 1.4.24(#33892)OrderedDictwith plain dict (#33508)#33568, #33480, #33753, #33520, #33623)
Pydanticwarning aboutorm_moderename (#33220)Pydanticlimitation for version < 2 (#33507)Doc only changes
"""""""""""""""""
v2.7.0Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
Remove Python 3.7 support (#30963)
""""""""""""""""""""""""""""""""""
As of now, Python 3.7 is no longer supported by the Python community.
Therefore, to use Airflow 2.7.0, you must ensure your Python version is
either 3.8, 3.9, 3.10, or 3.11.
Old Graph View is removed (#32958)
""""""""""""""""""""""""""""""""""
The old Graph View is removed. The new Graph View is the default view now.
The trigger UI form is skipped in web UI if no parameters are defined in a DAG (#33351)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
If you are using
dag_run.confdictionary and web UI JSON entry to run your DAG you should either:Add params to your DAG <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/params.html#use-params-to-provide-a-trigger-ui-form>_show_trigger_form_if_no_paramsto bring back old behaviourThe "db init", "db upgrade" commands and "[database] load_default_connections" configuration options are deprecated (#33136).
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Instead, you should use "airflow db migrate" command to create or upgrade database. This command will not create default connections.
In order to create default connections you need to run "airflow connections create-default-connections" explicitly,
after running "airflow db migrate".
In case of SMTP SSL connection, the context now uses the "default" context (#33070)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The "default" context is Python's
default_ssl_contestinstead of previously used "none". Thedefault_ssl_contextprovides a balance between security and compatibility but in some cases,when certificates are old, self-signed or misconfigured, it might not work. This can be configured
by setting "ssl_context" in "email" configuration of Airflow.
Setting it to "none" brings back the "none" setting that was used in Airflow 2.6 and before,
but it is not recommended due to security reasons ad this setting disables validation of certificates and allows MITM attacks.
Disable default allowing the testing of connections in UI, API and CLI(#32052)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
For security reasons, the test connection functionality is disabled by default across Airflow UI,
API and CLI. The availability of the functionality can be controlled by the
test_connectionflag in thecoresection of the Airflowconfiguration (
airflow.cfg). It can also be controlled by theenvironment variable
AIRFLOW__CORE__TEST_CONNECTION.The following values are accepted for this config param:
Disabled: Disables the test connection functionality anddisables the Test Connection button in the UI.
This is also the default value set in the Airflow configuration.
2.
Enabled: Enables the test connection functionality andactivates the Test Connection button in the UI.
Hidden: Disables the test connection functionality andhides the Test Connection button in UI.
For more information on capabilities of users, see the documentation:
https://airflow.apache.org/docs/apache-airflow/stable/security/security_model.html#capabilities-of-authenticated-ui-users
It is strongly advised to not enable the feature until you make sure that only
highly trusted UI/API users have "edit connection" permissions.
The
xcomEntriesAPI disables support for thedeserializeflag by default (#32176)"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
For security reasons, the
/dags/*/dagRuns/*/taskInstances/*/xcomEntries/*API endpoint now disables the
deserializeoption to deserialize arbitraryXCom values in the webserver. For backward compatibility, server admins may set
the
[api] enable_xcom_deserialize_supportconfig to True to enable theflag and restore backward compatibility.
However, it is strongly advised to not enable the feature, and perform
deserialization at the client side instead.
Change of the default Celery application name (#32526)
""""""""""""""""""""""""""""""""""""""""""""""""""""""
Default name of the Celery application changed from
airflow.executors.celery_executortoairflow.providers.celery.executors.celery_executor.You should change both your configuration and Health check command to use the new name:
celery_app_nameconfiguration incelerysection) useairflow.providers.celery.executors.celery_executorairflow.providers.celery.executors.celery_executor.appThe default value for
scheduler.max_tis_per_queryis changed from 512 to 16 (#32572)""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
This change is expected to make the Scheduler more responsive.
scheduler.max_tis_per_queryneeds to be lower thancore.parallelism.If both were left to their default value previously, the effective default value of
scheduler.max_tis_per_querywas 32(because it was capped at
core.parallelism).To keep the behavior as close as possible to the old config, one can set
scheduler.max_tis_per_query = 0,in which case it'll always use the value of
core.parallelism.Some executors have been moved to corresponding providers (#32767)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
In order to use the executors, you need to install the providers:
apache-airflow-providers-celerypackage >= 3.3.0apache-airflow-providers-cncf-kubernetespackage >= 7.4.0apache-airflow-providers-daskexecutorpackage in any versionYou can achieve it also by installing airflow with
[celery],[cncf.kubernetes],[daskexecutor]extras respectively.Users who base their images on the
apache/airflowreference image (not slim) should be unaffected - the basereference image comes with all the three providers installed.
Improvement Changes
^^^^^^^^^^^^^^^^^^^
PostgreSQL only improvement: Added index on taskinstance table (#30762)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
This index seems to have great positive effect in a setup with tens of millions such rows.
New Features
""""""""""""
AIP-49 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-49+milestone%3A%22Airflow+2.7.0%22>_)AIP-51 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-51+milestone%3A%22Airflow+2.7.0%22>_)AIP-52 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-52+milestone%3A%22Airflow+2.7.0%22>_)AIP-53 <https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+milestone%3A%22Airflow+2.7.0%22+label%3Aprovider%3Aopenlineage>_)BranchExternalPythonOperator(#32787, #33360)Per-LocalTaskJobConfiguration (#32313)AirflowClusterPolicySkipDagexception (#32013)reactflowfor datasets graph (#31775)chainwhich doesn't require matched lists (#31927)--retryand--retry-delaytoairflow db check(#31836)sectionquery param in get config rest API (#30936)Scheduled->Queued->Runningtask state transition times (#30612)Improvements
""""""""""""
db upgradetodb migrateand addconnections create-default-connections(#32810, #33136)<=parallelism (#32572)isdisjointinstead ofnot intersection(#32616)dag_processorstatus. (#32382)[triggers.running](#32050)TriggerDagRunOperator: Addwait_for_completiontotemplate_fields(#31122)PythonVirtualenvOperatortermination log in alert (#31747)airflow dbcommands to SQLAlchemy 2.0 style (#31486)validatorsinto their own modules (#30802)get_logapi (#30729)Bug Fixes
"""""""""
Gantt chart:Use earliest/oldest ti dates if different than dag run start/end (#33215)virtualenvdetection for Pythonvirtualenvoperator (#33223)chmodairflow.cfg(#33118)max_active_runsreached its upper limit. (#31414)Configuration
📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Never, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate. View repository job log here.