Durable runs

A durable run needs the agent's name and store:

const runId = await support.start('Sort the open tickets by urgency.', {
  thread: 'triage',
  user: 'team',
})

// Status, steps, credits, the messages so far, the answer or pending calls.
const run = await support.get(runId)
for await (const entry of support.events(runId)) console.log(entry.event.type)
await support.cancel(runId) // it ends cancelled at its next heartbeat

start() returns the run's id at once, and the run goes on without the caller. Its status is queued, running, paused, succeeded, failed or cancelled. A run that paused for approval goes on with support.resumeRun(runId, { approve: [callId] }). Of two decisions at once, one runs the calls and the other throws RunNotPausedError.

await support.recover() // when your app starts, then every half minute:
setInterval(() => {
  support.recover().catch((error) => console.error('recover', error))
}, 30_000)

• Every step is saved before its tools run, so a run continues from its last saved step.
• A running run renews its lease every 5 seconds while it works. After 20 seconds without a renewal, another process may take the run over.
• A run whose process died 3 times fails. durable: { leaseMs, heartbeatMs, maxAttempts } on the agent changes these.
• A tool call that was running when its process died runs again, with the same idempotencyKey. Key every write on it, and the second attempt finds the first.
• Each agent recovers, resumes and shows only the runs it started, by its name. Renaming an agent leaves its runs and threads behind.

By default a run executes in the process that started it. To spread runs over workers, give the agent an executor that hands the run's id to your queue, and let the worker call execute(runId):

import { Queue, Worker } from 'bullmq'

const queue = new Queue('ai-runs', { connection })

const support = playcode.ai.agent({
  name: 'support',
  store,
  model,
  system,
  tools,
  executor: (runId) => {
    const job = { jobId: runId, removeOnComplete: true, removeOnFail: true }
    // A run the queue lost is queued again by recover().
    queue.add('run', { runId }, job).catch((error) => console.error(error))
  },
})

new Worker('ai-runs', (job) => support.execute(job.data.runId), {
  connection,
})

Any process runs a queued run by its id. Two workers that take the same run at once run it once: one claim wins. The SDK queues a run again after an approval, a cancel of a paused run and a crash, so a finished job must free its id: hence removeOnComplete and removeOnFail.

A run's events reach its page through the agent's events bus. The default, memoryEventBus(), is one process's memory. When a page's request can reach another process than the run, pass a RunEventBus that every process shares: publish, subscribe and close. A new Cloud app passes one on Redis when REDIS_URL is set, with a BullMQ executor beside it.

A store is twelve reads and writes over your app's database:

Every write that can race is conditional. That is what keeps two processes from running one run, and an approved call from running twice. memoryStore() is the reference implementation, and agentStoreContract from @playcode/sdk/testing checks yours: see Testing with the SDK. A new Cloud app ships a store on Prisma, with the tables AiThread, AiRun and AiUserSpend.

• A thread is its agent's and its user's own: support.history('triage', { user: 'team' }) reads it.
• A thread keeps every message, and the brief of a long conversation.
• A thread takes one run at a time: start() throws ThreadBusyError while another is in progress.
• Once a run ends, no tool call in its thread is left without a result: a run cancelled or failed before a call ran answers that call "Not run". A run that waits for approval holds its calls open until the decision.

• The store is yours to keep: Playcode does not host conversations or runs.
• A run executes on your app's compute. A schedule, such as a nightly run, is your app's own timer or queue calling start().
• memoryStore() and memoryEventBus() lose everything on a restart: they are for tests and a first try.