n8n

How to Automate Apache Airflow Run Management?

Kick off an Apache Airflow DAG from n8n, wait while it runs, and return a task result when it finishes. Great for data and IT teams that want a no code way to control pipelines and share results with other apps. Saves time and keeps people out of the Airflow UI.

The flow starts with input values for the DAG id, the task id, the config, and timing settings. n8n posts to the Airflow API to create a run with your config. If the run is queued, it pauses for a set number of seconds and tries again. A counter tracks attempts and stops with a clear error when the wait limit is reached. When the state turns to success, it reads the XCom value from the chosen task and passes it back. If the run fails, it stops with a short error so you know what happened. All calls use basic auth against your Airflow base URL.

Setup is simple. You need an Airflow user with API access, the base URL, and a Basic Auth credential in n8n Cloud. Provide dag_id, task_id, and conf in the input or from a parent workflow. Pick wait and wait_time to match your pipeline speed. Expect faster handoffs, less screen watching, and a clean result object you can route to Slack, databases, or other jobs.

What are the key features?

  • Starts an Airflow DAG run with your custom config payload
  • Polls the run state on a schedule you control with a wait interval
  • Routes by state to success, running, queued, or failed paths
  • Counts attempts and stops when a time limit is reached
  • Returns a task XCom value when the run finishes successfully
  • Uses Basic Auth against your Airflow base URL for all calls
  • Takes inputs for dag_id, task_id, conf, wait, and wait_time from a parent flow

What are the benefits?

  • Reduce manual Airflow checks from 10 minutes to under 1 minute per run
  • Automate up to 90% of polling and status tracking
  • Cut unknown failures by adding clear errors for queued and failed runs
  • Connect n8n processes to Apache Airflow without custom code
  • Scale to many concurrent runs with safe wait and timeout control

How do you set it up?

  1. Import the template into n8n: Create a new workflow in n8n > Click the three dots menu > Select 'Import from File' > Choose the downloaded JSON file.
  2. You'll need accounts with Apache Airflow. See the Tools Required section above for links to create accounts with these services.
  3. In your Airflow UI, confirm the REST API is enabled and you have a user with permissions to trigger DAG runs and read task data.
  4. In the n8n credentials manager, create a new HTTP Basic Auth credential. Use your Airflow username and password. Name it Airflow so it is easy to find later.
  5. Open the node that stores the Airflow base URL and set it to your Airflow address, for example https://airflow.example.com.
  6. Open each HTTP request node that calls Airflow. In the credential dropdown, pick the Airflow Basic Auth credential you created. If you do not see it, click Create new credential and follow the on screen steps.
  7. Check the input node named in data. It accepts dag_id, task_id, conf, wait, and wait_time. Set defaults or plan to pass them from a parent workflow.
  8. Pick a wait value in seconds and a wait_time limit for how many checks you will allow before a timeout. Start with wait 10 and wait_time 30 for slower pipelines.
  9. Run a test with a known DAG id and task id that pushes an XCom value. Watch the status change and confirm a result item is returned when the run completes.
  10. If you see unauthorized errors, verify the Airflow URL and Basic Auth user. If you see not found errors, confirm the dag_id and task_id names match your Airflow project.
  11. If runs queue for too long, increase wait_time or adjust Airflow resources. If the result is empty, ensure the target task pushes an XCom value in your DAG code.

Tools Required

$24 / mo or $20 / mo billed annually to use n8n in the cloud. However, the local or self-hosted n8n Community Edition is free.

Apache Airflow

Sign up

Open-source self-hosted: $0 (Apache License 2.0)

Similar Templates

Join Futurise to access 1,200+ automation templates

Get instant access to ready-made automation workflows for n8n, Make.com, AI agents, and more. Download, customise, and deploy in minutes.