TriggerDagRunOperator defers even when wait_for_completion=False #60049
Description
Apache Airflow version
3.1.5
If "Other Airflow 3 version" selected, which one?
No response
What happened?
In Airflow 3.1.5 with apache-airflow-providers-standard==1.10.0, a TriggerDagRunOperator configured for “fire-and-forget” (wait_for_completion=False) enters the DEFERRED state (e.g. “queued deferred”) and does not progress, even though the target DAG run completes immediately or has no running tasks.
This occurs when deferrable=True is set on the operator (either explicitly or via default_args). Setting deferrable=False causes the task to behave as expected (trigger the target DAG and succeed).
What you think should happen instead?
When wait_for_completion=False, TriggerDagRunOperator should never defer. deferrable=True should be treated as a no-op (or overridden/validated) unless wait_for_completion=True.
Expected behaviour:
- Trigger target DagRun (optionally reset it, if configured)
- Finish the task in success
- Do not enter deferred state
How to reproduce
Create two DAG files:
dags/target_dag.py
from datetime import datetime
from airflow.sdk import DAG
from airflow.operators.empty import EmptyOperator
with DAG(
dag_id="repro_target_dag",
start_date=datetime(2024, 1, 1),
schedule=None,
catchup=False,
) as dag:
start = EmptyOperator(task_id="start")
end = EmptyOperator(task_id="end")
start >> end
dags/trigger_dag.py
from datetime import datetime
from airflow.sdk import DAG
from airflow.providers.standard.operators.trigger_dagrun import TriggerDagRunOperator
with DAG(
dag_id="repro_trigger_dag",
start_date=datetime(2024, 1, 1),
schedule=None,
catchup=False,
) as dag:
TriggerDagRunOperator(
task_id="trigger_target_fire_and_forget",
trigger_dag_id="repro_target_dag",
wait_for_completion=False, # fire-and-forget
deferrable=True, # triggers the issue
reset_dag_run=True, # optional (issue observed with this set)
)
Steps
- Start Airflow (ensure scheduler + triggerer are running).
- Trigger repro_trigger_dag from the UI.
- Observe repro_target_dag completes immediately.
- Observe repro_trigger_dag.trigger_target_fire_and_forget enters DEFERRED and remains stuck.
Workaround / control
- Change deferrable=True to deferrable=False and rerun. The trigger task completes successfully.
Operating System
[Breeze:3.12.12] root@eaed748ca5ce:/opt/airflow$ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
This issue appears related to previous reports where TriggerDagRunOperator becomes stuck in the DEFERRED state when running in deferrable mode, particularly around edge cases in task retries, restarts, or DagRun handling. While those issues often involve wait_for_completion=True or other specific conditions, this report highlights a narrower case where deferral occurs even when wait_for_completion=False.
Related issues (for context):
- TriggerDagRunOperator stuck in deferred state after restart/reset (Deferrable mode of TriggerDagRunOperator stays stuck if used with
reset_dag_run#57756) - Deferred trigger tasks stuck in Airflow 3.x (Deferred trigger tasks seem to get stuck in Airflow 3.0.2 #52247)
These prior issues suggest that TriggerDagRunOperator’s deferrable execution path may need additional guarding to ensure deferral is only used when explicitly waiting for downstream DagRun completion.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct