登录

airflow

apacheApache-2.0活跃维护

用 Python 代码定义、调度和监控数据工作流，把 ETL/ELT 管道写成 DAG，告别手写 cron 和脚本拼接。

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

3.4.0发布于 2026-11-06 13:08 UTC47,430star +26 / 7 天18,330fork同步于 2026-11-10 05:19 UTC
工作流编排数据管道DAG调度ETLPython
对比 GitHub 官网airflowapacheapache-airflowautomationdag

怎么上手

由 README 提炼，执行前请确认
原文 
pip install 'apache-airflow==3.4.0' --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.4.0/constraints-3.10.txt"
硬件要求按 README 说明

README 未说明硬件要求，仅说明通过 PyPI 安装，且官方仅支持 pip 和 uv 安装方式

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

用 Python 代码定义、调度并监控工作流的编排平台，适合把数据管道当代码管理

用 Python 代码定义、调度和监控数据工作流，把 ETL/ELT 管道写成 DAG，告别手写 cron 和脚本拼接。

核心能力

  • 用 Python 编写 DAG 定义任务及其依赖关系
  • 调度器按依赖在 worker 阵列上执行任务
  • 提供丰富命令行工具对 DAG 做运维操作
  • Web UI 可视化生产管道、监控进度并排障
  • 工作流即代码，可维护、可版本化、可测试、可协作

容易踩的坑

直接 pip install apache-airflow 可能失败或装出不可用环境，需配合 constraints 文件安装；官方仅支持 pip 和 uv，用 Poetry 或 pip-tools 安装不受支持；constraints URL 中的版本号与 Python 版本需与要安装的 Airflow 版本对应

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本3.4.0
全部 release
  1. 3.4.0Apache Airflow 3.4.0最新正式版2026-11-06 · 2026-11-06 13:08 UTC

    📦 PyPI: https://pypi.org/project/apache-airflow/3.4.0/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.4.0/ 🛠 Release Notes: https://airflow.apache.org/docs/apache-airflow/3.4.0/release_notes.html 🐳 Docker Image: "docker pull apache/airflow:3.4.0" 🚏 Constraints: https://github.com/apache/airflow/tree/constraints-3.4.0

    Significant Changes

    Task Loops (#73005, #74222, #74275, #74345)

    Airflow 3.4.0 introduces Task Loops (AIP-111): a decorated task group can be repeated a number of times decided at runtime using .loop(). A conditional loop works like a do ... while loop - the body runs at least once and an until condition decides when to stop, bounded by max_iterations. Each iteration can use the previous iteration's result and iterations run one after another (tasks within an iteration can still run in parallel). Retried attempts are kept under a per-attempt UUID so each iteration's data is preserved and can be cleared selectively.

    For detailed usage instructions, see :doc:/authoring-and-scheduling/loops.

    .. note:: Downgrading to any version earlier than 3.4.0 can be slow, depending on the number of XCom rows that have been written after upgrading. Upgrading was tested on a 300m row XCom table, so please don't worry about upgrade time (we made sure that the XCom data didn't get touched or read during migration to keep it fast).

    This is because this release contains a migration that starts storing new XCom rows in an xcom_v2 table, and in order to downgrade we have to move those over to the old table, and there is no way of doing this without copying the written xcom values.

    The downgrade refuses to continue when it finds data it cannot map back safely. Each migration commits separately, so the later 3.4.0 migrations have already been reversed at that point: fix the data and re-run the downgrade, or run airflow db migrate to return to 3.4.

    On MySQL a failure after those checks leaves the schema partly downgraded, as MySQL cannot roll back DDL. Taking a backup before attempting the downgrade on MySQL is advised, so it can be restored in that case.

    Dag Importer for non-Python Dags (#72369, #73728, #73853, #74020, #74084, #74333, #74586)

    Airflow 3.4.0 can author an entire Dag - schedule, tasks, and dependencies - in a language other than Python. AIP-85 adds a pluggable Dag Importer in the Task SDK that teaches the Dag processor to recognize source formats beyond .py: each format registers an importer, the processor runs the matching language runtime, and the resulting Dags surface in Airflow like Python Dags. The formats that land on it are the TypeScript, Go, and Java language SDKs and a native YAML Dag format (a .yaml file with a $schema marker, validated against a schema and built into real operators, with explicit templating). The Dag Importer framework and the language SDKs are experimental in 3.4 and their APIs may change.

    For detailed usage instructions, see :doc:/administration-and-deployment/dag-importers.

    Unify executor workload queues under BaseExecutor.executor_queues (#63491)

    Executor workload state is now stored on the unified BaseExecutor.executor_queues mapping keyed by WorkloadType and scheduled through trigger_workloads. The previous per-type attributes and entrypoints (queued_tasks, queued_callbacks, supports_callbacks, trigger_tasks, order_queued_tasks_by_priority) and the Airflow 3.3 connection-test surface are folded into this mapping and kept as backward-compatible shims that emit RemovedInAirflow4Warning (to be removed in Airflow 4.0). Out-of-tree executors should declare supported_workload_types and implement trigger_workloads. Connection tests are now admitted under the same per-heartbeat budget as tasks and prioritized ahead of them. See `#63491 <https://github.com/apache/airflow/pull/

  2. 3.3.2Apache Airflow 3.3.22026-09-17 · 2026-09-17 11:10 UTC
    查看更新说明

    📦 PyPI: https://pypi.org/project/apache-airflow/3.3.2/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.3.2/ 🛠 Release Notes: https://airflow.apache.org/docs/apache-airflow/3.3.2/release_notes.html 🐳 Docker Image: "docker pull apache/airflow:3.3.2" 🚏 Constraints: https://github.com/apache/airflow/tree/constraints-3.3.2

    Significant Changes

    Backfill endpoints no longer disclose which backfill ids exist across Dags

    The four routes that name a backfill in their path -- GET /backfills/{backfill_id} and the pause, unpause and cancel routes -- resolved the Dag they authorize against from the dag_id supplied on the request whenever the path's id matched no row. An unknown id and a backfill on a Dag the caller cannot see therefore answered differently, which enumerates backfill ids across Dags.

    The backfill named in the path is now the only thing those routes authorize against.

    Behaviour changes:

    • Requesting a backfill on a Dag the caller cannot read now returns 404 (Backfill not found) -- the same response an unknown id gets -- instead of the 403 returned before. A caller who can read the Dag still gets 403 for a write they are not allowed to make.
    • A backfill_id in the path is never authorized against a dag_id in the request body or query string. GET /backfills, POST /backfills and POST /backfills/dry_run name no backfill in their path and keep authorizing off the request.
    • All four routes now answer an unknown id with the same detail, Backfill not found. The pause, unpause and cancel routes previously answered Could not find backfill with id {backfill_id}. Clients matching on detail must be updated. (#71113)

    An explicit credential now takes precedence over the session cookie

    get_user() is written to prefer an explicit bearer token, then OAuth2, then the session cookie, but that precedence was unreachable whenever a cookie was present. JWTRefreshMiddleware runs first, resolves a user from the _token cookie alone and stamps it on request.state, and get_user() returned that cached user before looking at either explicit credential. The effective order on every core-API route was cookie over bearer.

    A request carrying both a session cookie and an explicit credential therefore executed, and was recorded in the audit log, as the cookie's principal rather than the identity the client presented. The cached user is now honoured only when the request carries no explicit credential.

    Behaviour changes:

    A request carrying both a _token cookie and an Authorization: Bearer header

    is now resolved as the bearer token's principal, where it was previously resolved as the
cookie's. The same applies to a cookie combined with an OAuth2 token.

    An invalid or expired explicit credential is now rejected with 401/403 even

    when a valid ``_token`` cookie accompanies it. Previously the cookie silently took over
and the request succeeded as the cookie's principal; the failure is now loud.

    Requests carrying a single credential are unaffected. Cookie-only browser sessions keep

    the token-refresh behaviour of ``JWTRefreshMiddleware`` unchanged.

    Clients that relied on the cookie winning -- for example a browser-based tool that sent a

    service account's bearer token while a user session cookie was present, and expected the
user's identity to apply -- will now act as the bearer token's principal. Remove the
header, or the cookie, to select the intended identity explicitly. (#72225)

    Bug Fixes

    • Optimize the previous-task-instance lookup by removing a redundant dag_run join (#72944)
    • Revoke every credential presented to the logout endpoint, not just the session cookie (#72933)
    • API: Return HTTP 404 instead
  3. 3.3.1Apache Airflow 3.3.12026-08-12 · 2026-08-12 09:50 UTC
    查看更新说明

    📦 PyPI: https://pypi.org/project/apache-airflow/3.3.1/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.3.1/ 🛠 Release Notes: https://airflow.apache.org/docs/apache-airflow/3.3.1/release_notes.html 🐳 Docker Image: "docker pull apache/airflow:3.3.1" 🚏 Constraints: https://github.com/apache/airflow/tree/constraints-3.3.1

    Significant Changes

    Pandas 3 changes how DataFrame XComs are stored and read back (#71169)

    pandas 3 exposes its public classes from the pandas namespace, so a DataFrame is qualified as pandas.DataFrame instead of pandas.core.frame.DataFrame. XComs record that name alongside the serialized value, so the name written into the metadata database depends on the pandas version of the component that pushed the value. Airflow registers both names, and a DataFrame written by either pandas version can be read by either -- no configuration change is needed, and existing XComs stay readable.

    What you should do:

    • Roll this Airflow version out to every component before pandas 3 reaches any of them -- workers in particular. A component that predates this change cannot read a DataFrame XCom written under pandas 3, and fails the pull with:

      .. code-block:: text

      ImportError: pandas.DataFrame was not found in allow list for deserialization imports.
To allow it, add it to allowed_deserialization_classes in the configuration

      The message points at configuration, but the allow list is not the cause and changing it does not help. The rows are not corrupt: they become readable again as soon as the reader is upgraded.

    • Treat a downgrade as a one-way door for those XComs. Rolling back to an Airflow version without this change strands any DataFrame XCom written while on pandas 3, with the same error, until you roll forward again.

    • Review Dags that inspect the dtypes of a pulled DataFrame. The pandas version of the reader determines what a pulled DataFrame looks like, not the version that wrote it. Under pandas 3, a column of strings comes back as str rather than object, and its missing values come back as nan rather than None. Values are unchanged, but downstream code that branches on dtype == "object", checks cells with is None, or compares against a reference frame with DataFrame.equals() can behave differently after the upgrade.

    Fix 2.x to 3.0+ upgrade failure when a custom Dag bundle is configured (#70994)

    The 0082_3_1_0_make_bundle_name_not_nullable migration assigned every legacy row bundle_name='dags-folder', so triggering a DagRun raised Requested bundle 'dags-folder' is not configured. on any deployment that uses a bundle other than the default dags-folder. DagFileProcessorManager now runs a one-shot, best-effort backfill at startup that routes each affected Dag to the correct bundle based on its file path; unmatched Dags self-heal on the next successful parse (or run airflow dags reserialize to force it immediately).

    Team scoped values of options registered as sensitive are now hidden (#71099)

    Configuration options are registered as sensitive under their base section, so until now only the base spelling of an option was masked. A team scoped override -- set in a [<team>=<section>] config file section, or through an AIRFLOW__<TEAM>___<SECTION>__<KEY> environment variable -- was not recognized as the same option and was returned in full.

    Sensitivity is now decided after resolving the team scoped spelling back to the base option, so a team scoped value is masked exactly as the base value already was.

    Behaviour changes:

    • AirflowConfigParser.as_dict(display_sensitive=False), GET /config, GET /config/section/{section}/option/{option} and airflow config list now return < hidden > for a team scoped value of an option registered as sensitive. Deployments that read a
  4. java-sdk/1.0.0-beta1Apache Airflow Java SDK 1.0.0-beta1pre-release2026-07-13 · 2026-07-13 18:14 UTC
    查看更新说明

    First release of the Apache Airflow Java SDK (beta).

  5. 3.3.0Apache Airflow 3.3.02026-07-06 · 2026-07-06 15:16 UTC
    查看更新说明

    📦 PyPI: https://pypi.org/project/apache-airflow/3.3.0/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.3.0/ 🛠 Release Notes: https://airflow.apache.org/docs/apache-airflow/3.3.0/release_notes.html 🐳 Docker Image: "docker pull apache/airflow:3.3.0" 🚏 Constraints: https://github.com/apache/airflow/tree/constraints-3.3.0

    Significant Changes

    Asset Partitioning (#64571, #65447, #66030, #66848, #67184, #67475, #67716, #68978)

    Building on the asset partitioning introduced in 3.2.0, Airflow 3.3.0 substantially expands how a single upstream asset event fans out to partitioned downstream Dag runs. New partition mappers — RollupMapper (many-to-one), FanOutMapper (one-to-many), and FixedKeyMapper + SegmentWindow (categorical rollup) — compose with time windows (day/week/month/quarter/year) and a wait_policy (WaitForAll or MinimumCount(n)) to control when partitioned runs fire. Windows can fan out forward or backward in time, and total fan-out per upstream event is bounded by the new [scheduler] partition_mapper_max_downstream_keys config (configurable per mapper). Airflow 3.3.0 also adds the PartitionedAtRuntime timetable, which lets a Dag declare that its partition key(s) are assigned when the run starts rather than mapped from an upstream event.

    For detailed usage instructions, see :doc:/authoring-and-scheduling/assets.

    Task and Asset State Store (#65759, #66073, #66160, #66463, #66586, #66859, #67041, #67292, #67319)

    Airflow 3.3.0 introduces a first-class state store for tasks and assets (AIP-103). Tasks can persist arbitrary key-value state that survives across retries and runs via a new task_state_store accessor, and assets can carry their own state via asset_state_store — both available from the Task SDK. State is kept in the metadata database by default, or in a custom worker-side backend ([workers] state_store_backend), supports per-key retention with periodic garbage collection and an optional clear_on_success, and is fully manageable through the Core API and Execution API.

    For detailed usage instructions, see :doc:/core-concepts/task-and-asset-state-store.

    Pluggable Retry Policies (#65474)

    Task retry behaviour is now pluggable (AIP-105). In addition to a fixed retries count, you can attach a custom retry policy that decides whether and when a task is retried, enabling strategies such as retrying only on specific exceptions or backing off based on custom logic.

    For detailed usage instructions, see :ref:concepts:retry-policies.

    Language Task SDK (Java and Go) (#65958, #67161, #67635, #67699)

    Airflow 3.3.0 adds a Coordinator layer (AIP-108) that lets individual task implementations be written in non-Python languages while the Dag and its scheduling stay in Python. A task is declared in the Dag with @task.stub(queue=...); the worker routes it to a configured coordinator (JavaCoordinator for JVM languages, ExecutableCoordinator for self-contained native binaries such as Go) that runs the task in a language runtime and proxies Variables, Connections, and XComs back through the Execution API.

    .. warning::

    The Coordinator layer and the Java/Go SDKs are experimental in 3.3.0 and may change in future versions based on user feedback.

    For detailed usage instructions, see :doc:/authoring-and-scheduling/language-sdks/index.

    Dag bundle version on clear, rerun, and backfill (#63884)

    The new rerun_with_latest_version setting controls whether a cleared, rerun, or backfilled Dag run uses the latest bundle version or the original version from the initial run. The default is resolved by precedence: an explicit request parameter/CLI flag, then the Dag-level rerun_with_latest_version, then [core] rerun_with_latest_version, and finally False for clear/rerun and True for backfills (preserving historical behaviour). Airflow 2.x

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    Apache Airflow

    CategoryBadges
    LicenseLicense
    PyPIPyPI version PyPI - Python Version PyPI - Downloads
    ContainersDocker Pulls Docker Stars Artifact HUB
    CommunityContributors Slack Status Commit Activity LFX Health Score
    Dev toolsprek
    VersionBuild Status
    MainTests AMD main Tests ARM main
    3.xTests AMD 3.2 Tests ARM 3.2

    Apache Airflow (or simply Airflow) is a platform to programmatically author, schedule, and monitor workflows.

    When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.

    Use Airflow to author workflows (Dags) that orchestrate tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on Dags a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed.

    Table of contents

    Project Focus

    Airflow works best with workflows that are mostly static and slowly changing. When the Dag structure is similar from one run to the next, it clarifies the unit of work and continuity. Other similar projects include Luigi, Oozie and Azkaban.

    Beyond traditional data pipelines, Airflow is widely used to orchestrate machine learning workflows — training, retraining, evaluation, and deployment — and increasingly to orchestrate agentic and LLM-based workloads, coordinating the steps of an AI pipeline (data prep, tool calls, model invocation, evaluation) rather than acting as the agent itself. This isn't a new direction: teams have run ML and AI workloads on Airflow for years, and the ecosystem of providers supporting these use cases (see the AI & ML section of the Airflow registry) continues to grow.

    Airflow is commonly used to process data, but has the opinion that tasks should ideally be idempotent (i.e., results of the task will be the same, and will not create duplicated data in a destination system), and should not pass large quantities of data from one task to the next (though tasks can pass metadata using Airflow's XCom feature). For high-volume, data-intensive tasks, a best practice is to delegate to external services specializing in that type of work.

    Airflow is not a streaming solution, but it is often used to process real-time data, pulling data off streams in batches.

    Principles

    • Dynamic: Pipelines are defined in code, enabling dynamic dag generation and parameterization.
    • Extensible: The Airflow framework includes a wide range of built-in operators and can be extended to fit your needs.
    • Flexible: Airflow leverages the Jinja templating engine, allowing rich customizations.

    Requirements

    Apache Airflow is tested with:

    Main version (dev)Stable version (3.4.0)Deprecate version (2.11.2)
    Python3.11, 3.12, 3.13, 3.143.10, 3.11, 3.12, 3.13, 3.143.10, 3.11, 3.12
    PlatformAMD64/ARM64AMD64/ARM64AMD64/ARM64(*)
    Kubernetes1.31, 1.32, 1.33, 1.34, 1.35, 1.36, 1.371.30, 1.31, 1.32, 1.33, 1.34, 1.351.26, 1.27, 1.28, 1.29, 1.30
    PostgreSQL14, 15, 16, 17, 1814, 15, 16, 17, 1812, 13, 14, 15, 16
    MySQL8.0, 8.4, Innovation8.0, 8.4, Innovation8.0, Innovation
    SQLite3.15.0+3.15.0+3.15.0+

    * Experimental

    Note: MariaDB is not tested/recommended.

    Note: SQLite is used in Airflow tests. Do not use it in production. We recommend using the latest stable version of SQLite for local development.

    Note: Airflow currently can be run on POSIX-compliant Operating Systems. For development, it is regularly tested on fairly modern Linux Distros and recent versions of macOS. On Windows you can run it via WSL2 (Windows Subsystem for Linux 2) or via Linux Containers. The work to add Windows support is tracked via #10388, but it is not a high priority. You should only use Linux-based distros as "Production" execution environment as this is the only environment that is supported. The only distro that is used in our CI tests and that is used in the Community managed DockerHub image is Debian Bookworm.

    Getting started

    Visit the official Airflow website documentation (latest stable release) for help with installing Airflow, getting started, or walking through a more complete tutorial.

    Note: If you're looking for documentation for the main branch (latest development branch): you can find it on s.apache.org/airflow-docs.

    For more information on Airflow Improvement Proposals (AIPs), visit the Airflow Wiki.

    Documentation for dependent projects like provider distributions, Docker image, Helm Chart, you'll find it in the documentation index.

    Installing from PyPI

    We publish Apache Airflow as apache-airflow package in PyPI. Installing it however might be sometimes tricky because Airflow is a bit of both a library and application. Libraries usually keep their dependencies open, and applications usually pin them, but we should do neither and both simultaneously. We decided to keep our dependencies as open as possible (in pyproject.toml) so users can install different versions of libraries if needed. This means that pip install apache-airflow will not work from time to time or will produce unusable Airflow installation.

    To have repeatable installation, however, we keep a set of "known-to-be-working" constraint files in the orphan constraints-main and constraints-2-0 branches. We keep those "known-to-be-working" constraints files separately per major/minor Python version. You can use them as constraint files when installing Airflow from PyPI. Note that you have to specify correct Airflow tag/version/branch and Python versions in the URL.

    1. Installing just Airflow:

    Note: Only pip and uv installation is currently officially supported.

    While it is possible to install Airflow with tools like Poetry or pip-tools, they do not share the same workflow as pip - especially when it comes to constraint vs. requirements management. Installing via Poetry or pip-tools is not currently supported.

    If you wish to install Airflow using those tools, you should use the constraint files and convert them to the appropriate format and workflow that your tool requires.

    pip install 'apache-airflow==3.4.0' \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.4.0/constraints-3.10.txt"
    1. Installing with extras (i.e., postgres, google)
    pip install 'apache-airflow[postgres,google]==3.4.0' \
 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.4.0/constraints-3.10.txt"

    For information on installing provider distributions, check providers.

    Installation

    For comprehensive instructions on setting up your local development environment and installing Apache Airflow, please refer to the INSTALLING.md file.

    Official source code

    Apache Airflow is an Apache Software Foundation (ASF) project, and our official source code releases:

    Following the ASF rules, the source packages released must be sufficient for a user to build and test the release provided they have access to the appropriate platform and tools.

    Convenience packages

    There are other ways of installing and using Airflow. Those are "convenience" methods - they are not "official releases" as stated by the ASF Release Policy, but they can be used by the users who do not want to build the software themselves.

    Those are - in the order of most common ways people install Airflow:

    • PyPI releases to install Airflow using standard pip tool
    • Docker Images to install airflow via docker tool, use them in Kubernetes, Helm Charts, docker-compose, docker swarm, etc. You can read more about using, customizing, and extending the images in the Latest docs, and learn details on the internals in the images document.
    • Tags in GitHub to retrieve the git project sources that were used to generate official source packages via git

    All those artifacts are not official releases, but they are prepared using officially released sources. Some of those artifacts are "development" or "pre-release" ones, and they are clearly marked as such following the ASF Policy.

    User Interface

    • Dags: Overview of all Dags in your environment.

      Dags

    • Assets: Overview of Assets with dependencies.

      Asset Dependencies

    • Grid: Grid representation of a Dag that spans across time.

      Grid

    • Graph: Visualization of a Dag's dependencies and their current status for a specific run.

      Graph

    • Home: Summary statistics of your Airflow environment.

      Home

    • Backfill: Backfilling a Dag for a specific date range.

      Backfill

    • Code: Quick way to view source code of a Dag.

      Code

    Semantic versioning

    As of Airflow 2.0.0, we support a strict SemVer approach for all packages released.

    There are few specific rules that we agreed to that define details of versioning of the different packages:

    • Airflow: SemVer rules apply to core airflow only (excludes any changes to providers). Changing limits for versions of Airflow dependencies is not a breaking change on its own.
    • Airflow Providers: SemVer rules apply to changes in the particular provider's code only. SemVer MAJOR and MINOR versions for the packages are independent of the Airflow version. For example, google 4.1.0 and amazon 3.1.1 providers can happily be installed with Airflow 2.1.2. If there are limits of cross-dependencies between providers and Airflow packages, they are present in providers as install_requires limitations. We aim to keep backwards compatibility of providers with all previously released Airflow 2 versions but there will sometimes be breaking changes that might make some, or all providers, have minimum Airflow version specified.
    • Airflow Helm Chart: SemVer rules apply to changes in the chart only. SemVer MAJOR and MINOR versions for the chart are independent of the Airflow version. We aim to keep backwards compatibility of the Helm Chart with all released Airflow 2 versions, but some new features might only work starting from specific Airflow releases. We might however limit the Helm Chart to depend on minimal Airflow version.
    • Airflow API clients: Their versioning is independent from Airflow versions. They follow their own SemVer rules for breaking changes and new features - which for example allows to change the way we generate the clients.

    Version Life Cycle

    Apache Airflow version life cycle:

    VersionCurrent Patch/MinorStateFirst ReleaseLimited MaintenanceEOL/Terminated
    33.4.0MaintenanceApr 22, 2025TBDTBD
    22.11.2EOLDec 17, 2020Oct 22, 2025Apr 22, 2026
    1.101.10.15EOLAug 27, 2018Dec 17, 2020June 17, 2021
    1.91.9.0EOLJan 03, 2018Aug 27, 2018Aug 27, 2018
    1.81.8.2EOLMar 19, 2017Jan 03, 2018Jan 03, 2018
    1.71.7.1.2EOLMar 28, 2016Mar 19, 2017Mar 19, 2017

    Limited support versions will be supported with security and critical bug fix only. EOL versions will not get any fixes nor support. We always recommend that all users run the latest available minor release for whatever major version is in use. We highly recommend upgrading to the latest Airflow major release at the earliest convenient time and before the EOL date.

    Support for Python and Kubernetes versions

    As of Airflow 2.0, we agreed to certain rules we follow for Python and Kubernetes support. They are based on the official release schedule of Python and Kubernetes, nicely summarized in the Python Developer's Guide and Kubernetes version skew policy.

    1. We drop support for Python and Kubernetes versions when they reach EOL. Except for Kubernetes, a version stays supported by Airflow if two major cloud providers still provide support for it. We drop support for those EOL versions in main right after EOL date, and it is effectively removed when we release the first new MINOR (Or MAJOR if there is no new MINOR version) of Airflow. For example, for Python 3.10 it means that we will drop support in main right after 27.06.2023, and the first MAJOR or MINOR version of Airflow released after will not have it.

    2. We support a new version of Python/Kubernetes in main after they are officially released, as soon as we make them work in our CI pipeline (which might not be immediate due to dependencies catching up with new versions of Python mostly) we release new images/support in Airflow based on the working CI setup.

    3. This policy is best-effort which means there may be situations where we might terminate support earlier if circumstances require it.

    Base OS support for reference Airflow images

    The Airflow Community provides conveniently packaged container images that are published whenever we publish an Apache Airflow release. Those images contain:

    • Base OS with necessary packages to install Airflow (stable Debian OS)
    • Base Python installation in versions supported at the time of release for the MINOR version of Airflow released (so there could be different versions for 2.3 and 2.2 line for example)
    • Libraries required to connect to supported Databases (again the set of databases supported depends on the MINOR version of Airflow)
    • Predefined set of popular providers (for details see the Dockerfile).
    • Possibility of building your own, custom image where the user can choose their own set of providers and libraries (see Building the image)
    • In the future Airflow might also support a "slim" version without providers nor database clients installed

    The version of the base OS image is the stable version of Debian. Airflow supports using all currently active stable versions - as soon as all Airflow dependencies support building, and we set up the CI pipeline for building and testing the OS version. Approximately 6 months before the end-of-regular support of a previous stable version of the OS, Airflow switches the images released to use the latest supported version of the OS.

    For example switch from Debian Bullseye to Debian Bookworm has been implemented before 2.8.0 release in October 2023 and Debian Bookworm will be the only option supported as of Airflow 2.10.0.

    Users will continue to be able to build their images using stable Debian releases until the end of regular support and building and verifying of the images happens in our CI but no unit tests were executed using this image in the main branch.

    Approach to dependencies of Airflow

    Airflow has a lot of dependencies - direct and transitive, also Airflow is both - library and application, therefore our policies to dependencies has to include both - stability of installation of application, but also ability to install newer version of dependencies for those users who develop Dags. We developed the approach where constraints are used to make sure airflow can be installed in a repeatable way, while we do not limit our users to upgrade most of the dependencies. As a result we decided not to upper-bound version of Airflow dependencies by default, unless we have good reasons to believe upper-bounding them is needed because of importance of the dependency as well as risk it involves to upgrade specific dependency. We also upper-bound the dependencies that we know cause problems.

    The constraint mechanism of ours takes care about finding and upgrading all the non-upper bound dependencies automatically (providing that all the tests pass). Our main build failures will indicate in case there are versions of dependencies that break our tests - indicating that we should either upper-bind them or that we should fix our code/tests to account for the upstream changes from those dependencies.

    Whenever we upper-bound such a dependency, we should always comment why we are doing it - i.e. we should have a good reason why dependency is upper-bound. And we should also mention what is the condition to remove the binding.

    Approach for dependencies for Airflow Core

    Those dependencies are maintained in pyproject.toml.

    There are few dependencies that we decided are important enough to upper-bound them by default, as they are known to follow predictable versioning scheme, and we know that new versions of those are very likely to bring breaking changes. We commit to regularly review and attempt to upgrade to the newer versions of the dependencies as they are released, but this is manual process.

    The important dependencies are:

    • SQLAlchemy: upper-bound to specific MINOR version (SQLAlchemy is known to remove deprecations and introduce breaking changes especially that support for different Databases varies and changes at various speed)
    • Alembic: it is important to handle our migrations in predictable and performant way. It is developed together with SQLAlchemy. Our experience with Alembic is that it very stable in MINOR version
    • Flask: We are using Flask as the back-bone of our web UI and API. We know major version of Flask are very likely to introduce breaking changes across those so limiting it to MAJOR version makes sense
    • werkzeug: the library is known to cause problems in new versions. It is tightly coupled with Flask libraries, and we should update them together
    • celery: Celery is a crucial component of Airflow as it used for CeleryExecutor (and similar). Celery follows SemVer, so we should upper-bound it to the next MAJOR version. Also, when we bump the upper version of the library, we should make sure Celery Provider minimum Airflow version is updated.
    • kubernetes: Kubernetes is a crucial component of Airflow as it is used for the KubernetesExecutor (and similar). Kubernetes Python library follows SemVer, so we should upper-bound it to the next MAJOR version. Also, when we bump the upper version of the library, we should make sure Kubernetes Provider minimum Airflow version is updated.

    Approach for dependencies in Airflow Providers and extras

    The main part of the Airflow is the Airflow Core, but the power of Airflow also comes from a number of providers that extend the core functionality and are released separately, even if we keep them (for now) in the same monorepo for convenience. You can read more about the providers in the Providers documentation. We also have set of policies implemented for maintaining and releasing community-managed providers as well as the approach for community vs. 3rd party providers in the providers document.

    Those extras and providers dependencies are maintained in provider.yaml of each provider.

    By default, we should not upper-bound dependencies for providers, however each provider's maintainer might decide to add additional limits (and justify them with comment).

    Contributing

    Want to help build Apache Airflow? Check out our contributors' guide for a comprehensive overview of how to contribute, including setup instructions, coding standards, and pull request guidelines.

    If you can't wait to contribute, and want to get started asap, check out the contribution quickstart here!

    Official Docker (container) images for Apache Airflow are described in images.

    Community standards

    Everyone interacting with the Apache Airflow project — on GitHub, the mailing lists, Slack, the CWiki, or anywhere else — is expected to follow the Code of Conduct.

    When repeated Code of Conduct breaches, spamming, abuse of project resources, or other sustained disruptive behaviour cannot be resolved through normal review and mentoring, the project applies the Community escalation process. It describes the steps maintainers and the PMC may take — from direct feedback, through closing PRs, up to PMC-level or ASF-Infrastructure-level blocks and reporting accounts to GitHub — and how affected contributors can appeal a decision by emailing the PMC at [email protected].

    Agent-assisted contribution (apache-magpie)

    This repo uses the apache/magpie framework, installed from its plugin marketplace. The framework provides maintainer-facing PR-management skills (pr-management-triage, pr-management-code-review, pr-management-stats, pr-management-mentor) that are exposed as agent skills in agent harnesses such as Claude Code.

    Nothing framework-related is committed to this repo, and a fresh clone needs no setup step — the plugin is installed per-user in your own agent harness. In Claude Code:

    /plugin marketplace add apache/magpie
/plugin install magpie-pr-management@apache-magpie

    Install magpie@apache-magpie instead to get every skill family at once, or add further families (magpie-security, magpie-release-management, …) one at a time. Pin a release rather than tracking main by adding the marketplace from a tag: /plugin marketplace add apache/[email protected].

    The project's recommended set is recorded in .apache-magpie.lock and includes the release-management family, which lets an agent verify a release candidate — including an optional check that runs your own changes against a providers wave installed in Breeze:

    /plugin install magpie-release-management@apache-magpie

    Other harnesses — Codex CLI, Gemini CLI, Copilot, Cursor — are covered in the framework's marketplace guide.

    Airflow-specific modifications to framework workflows live in .apache-magpie-overrides/ (committed) — the installed plugin reads them at run time, so they apply to every contributor without anyone editing the framework. Framework changes go via PR to apache/magpie.

    Voting Policy

    • Commits need a +1 vote from a committer who is not the author
    • When we do AIP voting, both PMC member's and committer's +1s are considered a binding vote.

    Who uses Apache Airflow?

    We know about around 500 organizations that are using Apache Airflow (but there are likely many more) in the wild.

    If you use Airflow - feel free to make a PR to add your organisation to the list.

    Who maintains Apache Airflow?

    Airflow is the work of the community, but the core committers/maintainers are responsible for reviewing and merging PRs as well as steering conversations around new feature requests. If you would like to become a maintainer, please review the Apache Airflow committer requirements.

    What goes into the next release?

    Often you will see an issue that is assigned to specific milestone with Airflow version, or a PR that gets merged to the main branch and you might wonder which release the merged PR(s) will be released in or which release the fixed issues will be in. The answer to this is as usual - it depends on various scenarios. The answer is different for PRs and Issues.

    To add a bit of context, we are following the Semver versioning scheme as described in Airflow release process. More details are explained in detail in this README under the Semantic versioning chapter, but in short, we have MAJOR.MINOR.PATCH versions of Airflow.

    • MAJOR version is incremented in case of breaking changes
    • MINOR version is incremented when there are new features added
    • PATCH version is incremented when there are only bug-fixes and doc-only changes

    Generally we release MINOR versions of Airflow from a branch that is named after the MINOR version. For example 2.7.* releases are released from v2-7-stable branch, 2.8.* releases are released from v2-8-stable branch, etc.

    1. Most of the time in our release cycle, when the branch for next MINOR branch is not yet created, all PRs merged to main (unless they get reverted), will find their way to the next MINOR release. For example if the last release is 2.7.3 and v2-8-stable branch is not created yet, the next MINOR release is 2.8.0 and all PRs merged to main will be released in 2.8.0. However, some PRs (bug-fixes and doc-only changes) when merged, can be cherry-picked to current MINOR branch and released in the next PATCHLEVEL release. For example, if 2.8.1 is already released and we are working on 2.9.0dev, then marking a PR with 2.8.2 milestone means that it will be cherry-picked to v2-8-test branch and released in 2.8.2rc1, and eventually in 2.8.2.

    2. When we prepare for the next MINOR release, we cut new v2-*-test and v2-*-stable branch and prepare alpha, beta releases for the next MINOR version, the PRs merged to main will still be released in the next MINOR release until rc version is cut. This is happening because the v2-*-test and v2-*-stable branches are rebased on top of main when next beta and rc releases are prepared. For example, when we cut 2.10.0beta1 version, anything merged to main before 2.10.0rc1 is released, will find its way to 2.10.0rc1.

    3. Then, once we prepare the first RC candidate for the MINOR release, we stop moving the v2-*-test and v2-*-stable branches and the PRs merged to main will be released in the next MINOR release. However, some PRs (bug-fixes and doc-only changes) when merged, can be cherry-picked to current MINOR branch and released in the next PATCHLEVEL release - for example when the last released version from v2-10-stable branch is 2.10.0rc1, some of the PRs from main can be marked as 2.10.0 milestone by committers, the release manager will try to cherry-pick them into the release branch. If successful, they will be released in 2.10.0rc2 and subsequently in 2.10.0. This also applies to subsequent PATCHLEVEL versions. When for example 2.10.1 is already released, marking a PR with 2.10.2 milestone will mean that it will be cherry-picked to v2-10-stable branch and released in 2.10.2rc1 and eventually in 2.10.2.

    The final decision about cherry-picking is made by the release manager.

    Marking issues with a milestone is a bit different. Maintainers do not mark issues with a milestone usually, normally they are only marked in PRs. If PR linked to the issue (and "fixing it") gets merged and released in a specific version following the process described above, the issue will be automatically closed, no milestone will be set for the issue, you need to check the PR that fixed the issue to see which version it was released in.

    However, sometimes maintainers mark issues with specific milestone, which means that the issue is important to become a candidate to take a look when the release is being prepared. Since this is an Open-Source project, where basically all contributors volunteer their time, there is no guarantee that specific issue will be fixed in specific version. We do not want to hold the release because some issue is not fixed, so in such case release manager will reassign such unfixed issues to the next milestone in case they are not fixed in time for the current release. Therefore, the milestone for issue is more of an intent that it should be looked at, than promise it will be fixed in the version.

    More context and FAQ about the patchlevel release can be found in the What goes into the next release document in the dev folder of the repository.

    Can I use the Apache Airflow logo in my presentation?

    Yes! Be sure to abide by the Apache Foundation trademark policies and the Apache Airflow Brandbook. The most up-to-date logos are found in this repo and on the Apache Software Foundation website.

    Links

    Sponsors

    The CI infrastructure for Apache Airflow has been sponsored by:

    The self-hosted CI runners on AWS are managed by RunsOn, which is free for open-source projects.

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    dify
    langgenius·Python

    可视化搭建 Agentic 工作流和 RAG 流水线，支持多种大模型与工具，可云端或私有化部署，帮团队从原型直接走到生产。

    Agent 编排+4Pipeline 编排
    160792+20325k1.19.02026-11-10 05:16 UTC
    ray
    ray-project·Python

    分布式 AI 计算引擎，提供核心分布式运行时和一组 AI 库，用来加速机器学习负载、大模型推理与超参搜索。

    分布式计算+4Pipeline 编排
    44245+178.3kray-2.60.02026-11-10 05:01 UTC
    compose
    docker·Go

    用 YAML 文件定义多容器应用，一条命令就能把整套服务编排起来跑，省去逐个 docker run 的麻烦。

    容器编排+4Pipeline 编排
    38669+385.9kv5.7.02026-11-09 13:38 UTC
    12-factor-agents
    humanlayer·TypeScript

    一套构建生产级 LLM 智能体应用的 12 条工程原则，帮团队把原型打磨成能真正交付给客户的可靠系统。

    智能体+4Pipeline 编排
    27067+322.0k2025-09-21 14:37 UTC