The Airflow DAG succeeded and the table is three days stale

Someone asks why the revenue model is stale. You open Airflow. The DAG ran this morning, every task is green, the last five runs are green. The table it builds last changed on Tuesday. Nothing failed, and nothing is current.

A task in Airflow succeeds when its operator returns without raising. A table is fresh when new rows landed in it recently. Those are different facts, and a DAG can satisfy the first for days while missing the second. The PythonOperator that reads yesterday's partition still returns cleanly when yesterday's partition is empty. The dbt run still exits 0 when an incremental model's where clause matched nothing. A retry that eventually passed counts as a pass.

Green means the operator returned

The scheduler tracks task instances, not the data underneath them. A task with trigger_rule="all_done" runs and can succeed even when an upstream task failed. A sensor with soft_fail=True that times out is skipped, and a skip does not turn the DAG red. An on_failure_callback that itself throws leaves the run in a state nobody is watching. None of these are bugs. They are the scheduler doing what you told it, and none of them know whether the warehouse table moved.

So the DAG-level question, “did the run finish,” is answered, and the data-level question, “is the output current,” is not being asked anywhere in the pipeline.

Make freshness a task that can fail

The common.sql provider ships SQLColumnCheckOperator and SQLTableCheckOperator, which run an assertion against your warehouse and raise when it fails. Point one at the age of the newest row and give it a bound:

from airflow.providers.common.sql.operators.sql import SQLTableCheckOperator

freshness = SQLTableCheckOperator(
    task_id="revenue_is_fresh",
    conn_id="warehouse",
    table="analytics.revenue_daily",
    checks={
        "row_landed_today": {
            "check_statement":
                "max(loaded_at) >= current_date",
        },
    },
    trigger_rule="all_done",  # run even if an upstream task failed
)

build_revenue >> freshness

With trigger_rule="all_done" the check runs whatever happened upstream, so a partial failure that left the table stale still turns the DAG red. For dbt, the equivalent is a dbt_utils.recency test on the timestamp column, which fails dbt build when the newest row is older than a datepart and interval you set.

This catches the run that happened and produced stale output. It cannot catch the run that did not happen.

The run that never started

The most common version of a stale table is a DAG that stopped scheduling. Someone paused it during an incident and did not toggle it back. A deploy changed schedule or dropped catchup=False onto a DAG that then sat waiting. The scheduler fell behind and the run for today is still queued. A paused DAG has no runs, so it has no failed tasks and no freshness check firing, and it will not appear in a list you sort by last failure.

Airflow has had a task sla mechanism for the late-but-running case. It only evaluates on a DAG run that actually started, so a paused or unscheduled DAG never triggers it, and its future has been in flux across recent major versions. Either way, the check for “this DAG should have produced a run by now” has to live outside the DAG.

Report freshness to something off the Airflow box

Add one task at the end of the DAG that measures how stale the table is and sends that number, as a heartbeat, to a service that is not part of your Airflow deployment:

import requests
from airflow.decorators import task
from airflow.providers.common.sql.hooks.sql import BaseHook


@task(trigger_rule="all_done")
def report_freshness():
    hook = BaseHook.get_connection("warehouse").get_hook()
    age = hook.get_first(
        "select extract(epoch from now() - max(loaded_at)) "
        "from analytics.revenue_daily"
    )[0]
    requests.post(
        "https://illari.dev/ping/YOUR_KEY/0",
        data={"age_seconds": int(age)},
        timeout=10,
    )


build_revenue >> report_freshness()

Now two failures land in one place. If no heartbeat arrives on schedule, the DAG did not run, whatever the Airflow UI says about why. If a heartbeat arrives reporting age_seconds past a day, the DAG ran and the table is stale. On illari the second one is a metric rule on the monitor (age_seconds is greater than 86400); the alert names the metric, the value, and the rule, and the age shows on every check-in so the normal range is visible at a glance.

Where this stops being enough

A freshness bound you can name a number for is a single assertion and a schedule check. When the healthy lag varies by day of week, or you care about row-count drift, column-level nulls, or consistency across a hundred models at once, the profiling platforms are built for that and worth the adoption. “This table should never be more than a day behind, and this DAG should run every morning” does not need one.

A stale table should reach you the morning the DAG went quiet, not the afternoon someone downstream asks why the numbers stopped moving.