Step 06 · Execution layer · Execution layer · Swarm: task orchestration ENPT
Alembic Complete Visual Course

Swarm: task orchestration

A depth-bounded, durable task queue that parks T4 work and resumes after crashes.

Read the plain version, or open the technical layer on any section.
1

Tasks with memory


@alembic/swarm takes a RunSpec (a goal plus tasks) and drains them. Tasks declare dependencies; the queue promotes them when ready. T4 tasks are parked in t4-parked.jsonl and never run autonomously.

Every state change is journaled to events.jsonl. If the process crashes, re-running the same spec resumes from the same directory because the run id is content-addressed.

Think of it like… a project manager with a whiteboard and a camera: the whiteboard shows current tasks, and the camera records every move so nothing is lost.

Under the hood

The orchestrator supports three tiers of depth: orchestrator → lead → worker (MAX_DEPTH = 2). Leads can fan out subtasks with ramp metadata. Workers write report files and rename them atomically to .complete.md or .failed.md. JsonlStore is the authoritative journal; SqliteIndex is an optional derived cache.

2

In one picture


Orchestrator Lead A Lead B Lead C worker worker worker worker
Orchestrator fans out leads; leads fan out workers; depth is bounded to two.
3

In the code


# run a swarm spec
alembic run tasks.json --offline

# watch it live
alembic tui <run-id>

Implementation notes

A TaskSpec carries id, title, tier, dependsOn, roleHint, and metadata. partitionByAutonomy splits tasks into autonomous and parked sets. The orchestrator drains ready tasks with a bounded worker pool and writes every event to the store.

4

Try it


Create a tasks.json with two dependent tasks and run it offline. Watch the events file grow.

Predict: What happens if a worker process is killed halfway through a task?
5

Quick check


Which file holds parked T4 tasks?