Progress Files — Agent Continuity Across Sessions

AI agents hit walls. Context windows fill up. Sessions time out. Processes crash. The work doesn't survive.

The Progress Files pattern solves this: an agent writes its current state to a persistent stash as it works, so any future session can read it and pick up where the last one left off. No orchestration framework, no checkpointing library. Just a named text file that persists.

The Problem

An agent starts a complex task — researching a topic, processing a dataset, building a report. Halfway through, the session ends. Maybe the context window filled up. Maybe the user closed the terminal. Maybe the cloud VM recycled.

The next session starts from zero. It re-reads the same files, re-discovers the same things, re-makes the same decisions. All the intermediate progress is gone.

This gets worse with long-running tasks. If a task takes 4 context windows to complete, the agent wastes significant time re-orienting on each restart.

The Pattern

At the start of a task, the agent reads its progress stash. If it exists, the agent is resuming — it reads the state and skips to where it left off. If it doesn't exist, this is a fresh start.
During the task, the agent periodically writes its progress to the stash. What's done, what's next, key decisions made, blockers encountered.
At the end of the task, the agent writes the final output to a separate stash and cleans up or archives the progress stash.

The progress stash is the agent's scratchpad. It's not the deliverable — it's the map that lets the agent navigate back to where it was.

Implementation

Create or resume

GET /stash/research-progress?persistent=true
X-API-KEY: sk_...

If the stash exists (200), parse the progress and resume. If it doesn't (404), start fresh:

PUT /stash/research-progress?persistent=true
X-API-KEY: sk_...
Content-Type: text/plain

{
  "task": "Research competitive landscape for agent memory products",
  "status": "in_progress",
  "started": "2026-04-13T10:00:00Z",
  "completed_steps": [],
  "next_step": "Identify top 5 competitors",
  "findings": [],
  "blockers": []
}

Update as you work

After completing each step, update the progress stash:

PATCH /stash/research-progress/json?persistent=true
X-API-KEY: sk_...
Content-Type: application/json

{
  "completed_steps": ["Identified 5 competitors: Mem0, Zep, LangGraph Memory, Letta, MemoryDB"],
  "next_step": "Analyze pricing and feature comparison for each competitor",
  "findings": [
    "Most competitors focus on single-agent memory, not multi-agent coordination",
    "Mem0 is closest competitor but tied to their embedding pipeline"
  ]
}

The JSON patch merges top-level fields, so you only send what changed.

Deliver and clean up

When the task is complete, write the final output to its own stash:

PUT /stash/competitive-research-final?persistent=true
X-API-KEY: sk_...
Content-Type: text/plain

# Competitive Landscape: Agent Memory Products
...full report...

Then update the progress stash to mark completion (or delete it):

PATCH /stash/research-progress/json?persistent=true
X-API-KEY: sk_...
Content-Type: application/json

{
  "status": "completed",
  "completed_at": "2026-04-13T14:30:00Z",
  "output_stash": "competitive-research-final"
}

What to Put in a Progress File

A good progress file answers one question: "If a brand new agent session reads this, can it continue the work without re-doing anything?"

Include:

What the task is — the objective, not just a label
What's been done — completed steps, in enough detail to not repeat them
What's next — the immediate next action
Key decisions — choices made and why, so the next session doesn't reconsider them
Findings so far — intermediate results that inform future steps
Blockers — anything that prevented progress

Don't include:

Raw data that's stored elsewhere (reference it by stash name instead)
Full conversation history (the progress file is a summary, not a transcript)
Implementation details of completed steps (only the outcomes matter)

Variations

Structured progress with a schema

For teams or repeatable workflows, define a schema for your progress files:

POST /schema
X-API-KEY: sk_...
Content-Type: application/json

{
  "name": "task-progress",
  "description": "Standard progress file for long-running agent tasks",
  "definition": {
    "type": "object",
    "properties": {
      "task": {"type": "string"},
      "status": {"enum": ["not_started", "in_progress", "blocked", "completed"]},
      "completed_steps": {"type": "array", "items": {"type": "string"}},
      "next_step": {"type": "string"},
      "findings": {"type": "array", "items": {"type": "string"}},
      "blockers": {"type": "array", "items": {"type": "string"}}
    },
    "required": ["task", "status", "next_step"]
  },
  "visibility": "public",
  "target_type": "stash"
}

Then bind the schema when creating the progress stash:

PUT /stash/research-progress?persistent=true&schema_id=sch_...

Every update gets validated, so a buggy agent can't corrupt the progress file with malformed data.

Progress with an execution stream

For tasks where you want a full audit trail, pair the progress stash with a stream:

POST /stream
X-API-KEY: sk_...
Content-Type: application/json

{
  "name": "research-progress-log",
  "ttl": 604800,
  "linked_stash": "research-progress"
}

The progress stash holds the current state (what's done, what's next). The stream holds the history (every update, in order). The stash is for resuming; the stream is for reviewing.

When to Use This Pattern

Long-running research — tasks that span multiple context windows
Multi-step pipelines — data processing, report generation, code migration
Recurring tasks — daily reports, monitoring sweeps, where each run builds on the last
Collaborative handoffs — one agent starts the work, another finishes it

When Not to Use This Pattern

Short tasks that fit in a single session. Don't add overhead for something that completes in one shot.
Idempotent tasks where re-running from scratch is cheap. If restarting costs nothing, a progress file adds complexity for no benefit.
Real-time coordination where multiple agents work simultaneously. Use streams for that — progress files are for sequential continuity, not parallel collaboration.