Testing with the SDK

fakeModel(replies) answers each model call with the next reply of a script, in the model maker's real stream shape:

import { Playcode, tool } from '@playcode/sdk'
import { fakeModel } from '@playcode/sdk/testing'
import { expect, it } from 'vitest'
import { z } from 'zod'

it('looks the order up before it answers', async () => {
  const fake = fakeModel(
    [
      { toolCalls: [{ name: 'findOrder', input: { number: 'A-1042' } }] },
      { text: 'It shipped on Monday.' },
    ],
    { model: 'test-model' },
  )
  const playcode = new Playcode({ apiKey: 'pcsk_test', fetch: fake.fetch })
  const findOrder = tool({
    description: 'An order by its number.',
    input: z.object({ number: z.string() }),
    execute: () => ({ status: 'shipped', shippedOn: 'Monday' }),
  })

  const agent = playcode.ai.agent({
    model: 'test-model',
    tools: { findOrder },
  })
  const result = await agent.run('Where is A-1042?')

  expect(result.text).toBe('It shipped on Monday.')
  expect(result.toolCalls[0]?.name).toBe('findOrder')
  // fake.calls holds what each model call sent, with its run and its user.
  expect(fake.calls).toHaveLength(2)
})

• A reply is text, toolCalls, usage, or an error: a Playcode refusal by its code, or a model maker's error that the SDK retries into the next reply.
• usage.credits is priced from the fake's card. Change the card with card, and the wire with api.
• A call the script has no reply for is refused as fake_script_exhausted.

record() keeps a real run as a cassette, and replay(cassette) plays it back with no request to Playcode:

import { readFileSync, writeFileSync } from 'node:fs'
import { Playcode } from '@playcode/sdk'
import { record, replay } from '@playcode/sdk/testing'

// Once, with a real key: this run spends credits.
const tape = record()
await runYourAgent(new Playcode({ fetch: tape.fetch }))
writeFileSync('agent.cassette.json', JSON.stringify(tape.cassette()))

// In every test after: no network, no credits.
const cassette = JSON.parse(readFileSync('agent.cassette.json', 'utf8'))
const { fetch } = replay(cassette)
await runYourAgent(new Playcode({ apiKey: 'pcsk_test', fetch }))

A replay answers each request with the recording of the same body, so parallel runs replay too. A request the cassette does not hold is refused as cassette_mismatch, naming the field that differs: your prompt or tools changed, so record again if you meant it. A cassette holds no key, because request headers are never recorded. It holds every prompt, file and answer of the run, so record with test data, never a customer's.

Your own AgentStore, over your database, must keep the conditional writes that durable runs need. agentStoreContract checks it:

import { agentStoreContract } from '@playcode/sdk/testing'
import { describe, it } from 'vitest'

describe('my agent store', () => {
  for (const check of agentStoreContract(() => new MyStore(db))) {
    it(check.name, check.run)
  }
})

Each check uses new ids of its own, so the checks can share one database. A failure throws a StoreContractError that says what the store did instead.

A new Cloud app's backend has these tests already. booking-agent.test.ts replays the example agent from its cassette and checks that no request left the process. The backend's end-to-end tests run the real app against fakeModel behind a local server, and its browser tests drive the /chat page. pnpm ai:smoke then makes one real call, to prove the key, the address and a model: it spends credits.