Temporal vs Airflow vs Step Functions

The one-line verdict

Temporal runs long-lived business processes in your own code and survives any crash mid-way. Airflow schedules and monitors batch data pipelines. Step Functions is AWS's managed state machine service: Standard for long, exactly-once workflows, Express for short, high-volume ones. They are all called "orchestrators", but they solve different problems.

The core difference: one process per thing, or one pipeline per schedule

The word "workflow" hides two very different shapes:

• A business process per thing. One workflow per order, per signup, per payment: charge, wait for a warehouse for up to three days, ship or refund. Millions run at once, each with its own state, and each must continue correctly after a crash. This is what Temporal and Step Functions are built for.
• A pipeline per schedule. One run per night (or per hour): extract, transform, load, refresh dashboards. Few runs, but each touches lots of data, and you sometimes re-run past dates. This is what Airflow is built for.

Picking a tool from the wrong shape is the most common mistake: scheduling a million per-order workflows in Airflow, or rebuilding a nightly ETL with backfills on top of Temporal.

Side by side

	Temporal	Airflow	Step Functions
Built for	Durable business processes	Scheduled batch pipelines	Managed workflows on AWS
You write	Ordinary code (Go, Java, Python, TypeScript, .NET and more)	Python DAGs	JSON state machines (Amazon States Language), or a visual designer
Long waits	Days to months, with durable timers	Not its purpose	Standard: up to one year
Delivery	Workflow steps recorded and replayed; activities retried by policy	Retries per task	Standard: exactly-once; Express: at-least-once (async)
Typical scale	Many concurrent workflows	Fewer, heavier runs	Standard: long runs; Express: very high volume, up to 5 minutes
Who runs it	You (MIT-licensed server) or Temporal Cloud	You, or a managed service	AWS

Temporal: durable execution in your own code

You write a workflow as a normal function: call this, wait for that, retry, branch. Temporal records every step in an event history. If the worker running it crashes, another worker replays the history and continues exactly where it stopped. Side effects (charging a card, sending an email) go in activities, which Temporal retries according to a retry policy.

The price of replay is determinism: given the same history, the workflow code must make the same decisions. Reading the clock, generating random numbers or calling the network directly inside a workflow breaks replay; use activities or the SDK's helpers instead. Each run's history is capped (51,200 events or 50 MB), so very long loops use Continue-As-New to start fresh with their current state.

Good fits: order and payment flows, sagas with compensations, onboarding sequences, long-running human approvals, and AI agents that run for minutes or hours and must not lose their progress.

Airflow: the scheduler for data pipelines

Airflow describes itself as a platform for orchestrating batch workflows: pipelines with a clear start and end that run on a schedule (or on an event). You define a DAG of tasks in Python, and Airflow schedules runs, retries failed tasks, shows every run in its UI, and backfills past dates when you need to recompute history.

Airflow 3 (released in April 2025) added DAG versioning, event-driven scheduling on data assets, scheduler-managed backfills, and a task execution interface that lets tasks run in other environments and languages.

Good fits: ETL and ELT, refreshing reports, training models on a schedule, data quality checks. Poor fits: a workflow per user request, or anything that needs millisecond-level responsiveness.

Step Functions: AWS's managed state machines

You describe a workflow as a state machine in JSON (Amazon States Language) or in a visual designer, with states that call AWS services directly, branch, wait, retry and run in parallel. There are two types:

• Standard: runs up to one year, exactly-once execution, full history available for 90 days, priced per state transition. For long, auditable processes like order fulfilment.
• Express: runs up to five minutes, at-least-once (asynchronous) or at-most-once (synchronous), priced by executions, duration and memory. For high-volume, short work like processing every upload or event.

Good fits: teams already on AWS who want no servers to run and tight integration with Lambda, SQS, DynamoDB and other services.

A worked example: one online store, three workflows

• Order fulfilment (charge, reserve stock, wait for the warehouse, ship or refund): a durable business process with compensations. Temporal, or Step Functions Standard if the team is all-in on AWS.
• Nightly sales reporting (load, transform, refresh dashboards, re-run last Tuesday): a scheduled batch pipeline. Airflow.
• Every product photo upload (resize, scan, tag, store, all within seconds): short, very high volume, rare repeats acceptable with idempotent steps. Step Functions Express, or a plain queue with workers.

Five questions that decide it

1. One workflow per thing, or one run per schedule? Per thing points to Temporal or Step Functions; per schedule points to Airflow.
2. How long can one run take? Minutes to months suits Temporal or Step Functions Standard; under five minutes allows Express.
3. Do you want to write code or configuration? Temporal is code-first; Step Functions is JSON or visual; Airflow is Python DAGs.
4. Are you committed to AWS? Step Functions removes all servers but ties you to AWS.
5. Who will run it? Self-hosting Temporal or Airflow is real operational work; Temporal Cloud, managed Airflow services and Step Functions take that on, at a price.

Where AI agents fit

An AI agent that runs for minutes, calls many tools and must not lose progress if a server restarts is a long-lived workflow, and teams increasingly run agents on durable execution engines like Temporal. The Agent Harnesses and Orchestration chapter explains how an agent harness differs from a workflow orchestrator, and when you need both. (Harness.io, a CI/CD product, is a different thing again.)

Honest limits

• Temporal's determinism rules and versioning of running workflows take time to learn, and self-hosting needs a database and careful operations.
• Airflow is not a low-latency, per-request engine, and passing large data between tasks belongs in storage, not in Airflow itself.
• Step Functions state machines in JSON get hard to read as they grow, Standard can get expensive at very high volume, and everything is tied to AWS.
• Similar tools exist in each space: Argo Workflows (Kubernetes-native), Prefect and Dagster (data pipelines), and Azure Durable Functions (durable execution on Azure), among others.

Remember this

• Per-thing durable processes (orders, payments, sagas, agents): Temporal or Step Functions Standard.
• Scheduled batch pipelines with backfills: Airflow.
• Short, high-volume event processing on AWS: Step Functions Express.
• Temporal workflow code must be deterministic; side effects go in activities.
• "Orchestrator" means different things; ask which shape of workflow you actually have.