教程整体目标
通过逐章可运行的增量,理解 Agent 的模型、上下文、任务状态、规划、工具、Skills 与 Agent Loop 等关键组件,并从零构建一个由 Harness 约束、不依赖 Agent 框架的可用 Agent。
实现 Agent Loop
把上下文、模型、工具和停止策略连接成真正可运行的循环。
预计 75 分钟本页目录
前面的章节已经完成仓库工具、权限策略、任务规划、用户交互、Skills 与 Trace。
src/agent-loop.tssrc/types.tssrc/index.tstests/agent-loop.test.ts
Agent Loop 是整个 Agent 的心脏。它不是一句“重复调用模型直到完成”,而是由确定性代码管理的一组状态转换:选择计划步骤 → 渐进加载相关 Skill → 组装上下文 → 请求模型 → 分类模型输出 → 执行工具 → 记录 observation → 更新或修订计划 → 进入下一轮或停止。
Step 1先定义循环不变量
写循环之前先固定任何一轮都不能破坏的规则:
- 只有 Harness 可以修改运行状态和决定停止;
- 每次模型请求前检查取消信号与模型轮次上限;
- 模型返回的每个工具调用都先验证,再执行;
- 同一批调用要么有足够预算全部处理,要么一个都不处理;
- 每个 observation 必须保留原始
call_id; - 有工具调用时不能把同时出现的文本误当最终答案;
- 没有工具调用时,只有非空最终文本才能完成;
- 最终文本只有在计划完成且验收条件全部通过时才能结束运行;
- 未激活 Skill 的正文和未请求资源永远不能进入本轮上下文;
- 每次模型调用、Skill 加载、工具调用、计划修订和停止原因都写入可审计状态。
Step 2定义 Loop 依赖与续轮输入
Loop 依赖领域接口,不直接依赖 OpenAI SDK。这样 Fake Model 可以完整测试控制流。
先把 harness_feedback 追加到 src/context.ts 的 ContextKindMap,把 blocked_plan 追加到 src/types.ts 的 StopReasonMap:
// src/context.tsexport interface ContextKindMap { harness_feedback: true;}// src/types.tsexport interface StopReasonMap { blocked_plan: true;}然后在 src/agent-loop.ts 定义 Loop 自身的契约:
import type { ModelRequest } from "./context.js";import type { ModelTurn } from "./model.js";import type { Planner } from "./planner.js";import type { TraceSink } from "./trace.js";import type { ToolRegistry } from "./tool-registry.js";import type { StopReason } from "./types.js";export type AgentLoopEvent = | { type: "run_started"; runId: string } | { type: "model_started"; step: number } | { type: "model_completed"; step: number } | { type: "plan_revised"; version: number; reason: string } | { type: "run_stopped"; reason: StopReason };export interface FunctionCallOutput { type: "function_call_output"; call_id: string; output: string;}export interface ModelDriver { start(options: { request: ModelRequest; tools: ReturnType<ToolRegistry["definitions"]>; }): Promise<ModelTurn>; continue(options: { previousResponseId: string; instructions: string; continuationContext: ModelRequest["input"]; outputs: FunctionCallOutput[]; tools: ReturnType<ToolRegistry["definitions"]>; }): Promise<ModelTurn>;}export interface AgentLoopOptions { cwd: string; skillsDirectory: string; maxSteps: number; maxToolCalls: number; toolTimeoutMs: number; model: ModelDriver; planner: Planner; tools: ToolRegistry; trace: TraceSink; signal?: AbortSignal; onEvent?: (event: AgentLoopEvent) => void;}这里选择用 previous_response_id 维护模型侧的历史,但它不能代替本轮 Assemble。根据 Responses API,续轮的 instructions 不会自动继承,所以 continue 明确再次传入它;同时把新组装的 continuationContext 作为新的 developer input,与 function_call_output 一起发送。
await client.responses.create({ previous_response_id: options.previousResponseId, instructions: options.instructions, input: [...options.outputs, ...options.continuationContext], tools: options.tools,});如果使用 store: false 或零数据保留模式,则改为在应用侧保存并回传所需的 response output items。无论采用哪种历史传输方式,当前计划、激活 Skill、上下文来源和 Harness feedback 都必须在每轮重新发送。
Step 3读懂单轮状态转换
先用伪代码理解每轮的唯一入口和三个出口:
WHILE status is running GUARD: cancelled? model budget exhausted? PLAN: select the next dependency-ready step; replan only after invalidating evidence ASSEMBLE: build goal + criteria + active step + selected context DECIDE: call model once IF model requested tools GUARD: enough tool-call budget for the whole batch? FOR EACH tool call ACT: validate, authorize, execute with timeout OBSERVE: store result and preserve call_id CONTINUE with all function_call_output items ELSE IF model returned non-empty final text COMPLETE only if the plan and every acceptance criterion are complete OTHERWISE reject early completion and continue ELSE FAIL with invalid_model_outputmaxSteps 统计模型轮次,maxToolCalls 统计工具调用次数。两者必须独立,否则模型可以在一次响应中生成大量工具调用绕过总预算。
Step 4实现完整 Agent Loop
创建 src/agent-loop.ts。下面是完整控制流,不再省略工具分支或停止分支。
import { buildModelRequest, summarizeSources } from "./context.js";import { checkCompletion } from "./completion.js";import { executeToolCall } from "./execute-tool.js";import { allCriteriaPassed, applyCriterionEvidence, assertEvidenceComesFromObservations, completeStep, createInitialPlan, planAsContextSource, reconcilePlan, selectNextStep, startStep,} from "./plan.js";import { createInitialState, transitionState } from "./state.js";import { waitForUserInput } from "./interaction.js";import { discoverSkills, skillCatalogAsContextSource } from "./skill-catalog.js";import type { AgentResult, AgentState, StopReason } from "./types.js";export async function runAgentLoop( task: string, options: AgentLoopOptions,): Promise<AgentResult> { let state = createInitialState(task, options.cwd, { maxSteps: options.maxSteps, maxToolCalls: options.maxToolCalls, }); state.skills.catalog = await discoverSkills(options.skillsDirectory); const toolDefinitions = options.tools.definitions(); state.plan = await createInitialPlan( task, summarizeSources(state.contextSources), options.tools.names(), options.planner, ); let previousResponseId: string | undefined; let pendingOutputs: FunctionCallOutput[] = []; emitLoopEvent(options, state, { type: "run_started", runId: state.runId }); while (state.status === "running") { if (options.signal?.aborted) return stop(state, "cancelled", options); if (state.budget.modelSteps >= state.budget.maxSteps) { return stop(state, "max_steps", options); } let activeStep = state.plan.steps.find((step) => step.id === state.activeStepId); if (!activeStep) { activeStep = selectNextStep(state.plan); if (activeStep) { state.plan = startStep(state.plan, activeStep.id); state.activeStepId = activeStep.id; activeStep = state.plan.steps.find((step) => step.id === activeStep?.id); } else if (state.plan.steps.some((step) => step.status !== "completed")) { return stop(state, "blocked_plan", options); } } const request = buildModelRequest({ task: state.task, cwd: state.cwd, toolNames: options.tools.names(), sources: [ ...state.contextSources, planAsContextSource(state.plan, activeStep), skillCatalogAsContextSource(state.skills.catalog), ...Object.values(state.skills.activeSkills).map((skill) => ({ id: `skill:${skill.name}`, kind: "skill_instructions" as const, label: skill.name, content: skill.instructions, priority: 96, })), ...Object.values(state.skills.activeSkills).flatMap((skill) => Object.entries(skill.loadedResources).map(([path, content]) => ({ id: `skill-resource:${skill.name}:${path}`, kind: "skill_resource" as const, label: `${skill.name}/${path}`, content, priority: 94, })), ), ], }); state.budget.modelSteps += 1; const modelStepCount = state.budget.modelSteps; emitLoopEvent(options, state, { type: "model_started", step: modelStepCount }); let turn: ModelTurn; try { turn = previousResponseId ? await options.model.continue({ previousResponseId, instructions: request.instructions, // Harness feedback and current plan are sent on every continuation. continuationContext: request.input, outputs: pendingOutputs, tools: toolDefinitions, }) : await options.model.start({ request, tools: toolDefinitions }); } catch (error) { state.steps.push({ number: state.steps.length + 1, kind: "model", summary: error instanceof Error ? error.message : String(error), }); return stop(state, "model_error", options); } state.steps.push({ number: state.steps.length + 1, kind: "model", summary: turn.toolCalls.length > 0 ? "模型请求工具" : "模型返回文本", }); emitLoopEvent(options, state, { type: "model_completed", step: modelStepCount }); if (turn.userInputRequest) { state = waitForUserInput(state, turn.userInputRequest); emitLoopEvent(options, state, { type: "run_stopped", reason: "user_input_required" }); return { status: "waiting", answer: "", stopReason: "user_input_required", state, }; } if (turn.toolCalls.length === 0) { const answer = turn.finalText.trim(); if (!answer) return stop(state, "invalid_model_output", options); const planCompleted = state.plan.steps.every((step) => step.status === "completed"); const completionError = await checkCompletion(state); if (!planCompleted || !allCriteriaPassed(state.plan) || completionError) { state.contextSources.push({ id: "completion-rejected-" + modelStepCount, kind: "harness_feedback", label: "completion rejected", content: completionError ?? "继续执行:计划或验收条件尚未完成。", priority: 100, }); previousResponseId = undefined; pendingOutputs = []; continue; } state.messages.push({ role: "assistant", content: answer }); state = transitionState(state, "completed", "final_answer"); emitLoopEvent(options, state, { type: "run_stopped", reason: "final_answer" }); return { status: "completed", answer, stopReason: "final_answer", state, }; } if (state.budget.toolCalls + turn.toolCalls.length > state.budget.maxToolCalls) { return stop(state, "max_tool_calls", options); } if (!activeStep) return stop(state, "blocked_plan", options); const outputs: FunctionCallOutput[] = []; const observations: string[] = []; for (const call of turn.toolCalls) { if (options.signal?.aborted) return stop(state, "cancelled", options); const result = await executeToolCall({ call, registry: options.tools, cwd: state.cwd, timeoutMs: options.toolTimeoutMs, }); state.budget.toolCalls += 1; if (result.type === "waiting") { state = transitionState(state, "waiting", result.reason); emitLoopEvent(options, state, { type: "run_stopped", reason: result.reason }); return { status: "waiting", answer: "", stopReason: result.reason, state, }; } state.steps.push({ number: state.steps.length + 1, kind: "tool", summary: call.name, }); state.contextSources.push({ id: call.callId, kind: "tool_observation", label: call.name, content: result.output, priority: 80, }); outputs.push({ type: "function_call_output", call_id: call.callId, output: result.output, }); observations.push(result.output); } const progress = await options.planner.evaluate({ step: activeStep, observations, }); assertEvidenceComesFromObservations(progress.evidence, observations); state.plan = applyCriterionEvidence(state.plan, progress.passedCriteria, observations); if (progress.completed) { state.plan = completeStep(state.plan, activeStep.id, progress.evidence.join("\n")); state.activeStepId = undefined; } if (progress.replanReason) { const draft = await options.planner.revise({ current: state.plan, reason: progress.replanReason, observations, }); state.plan = reconcilePlan(state.plan, draft, progress.replanReason); state.planHistory.push({ version: state.plan.version, reason: progress.replanReason, changedAt: new Date().toISOString(), }); emitLoopEvent(options, state, { type: "plan_revised", version: state.plan.version, reason: progress.replanReason, }); state.activeStepId = undefined; } previousResponseId = turn.responseId; pendingOutputs = outputs; } return stop(state, "invalid_model_output", options);}function stop( state: AgentState, reason: StopReason, options: AgentLoopOptions,): AgentResult { const status = reason === "cancelled" ? "cancelled" : reason === "blocked_plan" ? "blocked" : "failed"; const stopped = transitionState(state, status, reason); emitLoopEvent(options, stopped, { type: "run_stopped", reason }); return { status: stopped.status, answer: "", stopReason: reason, state: stopped };}function emitLoopEvent( options: AgentLoopOptions, state: AgentState, event: AgentLoopEvent,): void { const failedStop = event.type === "run_stopped" && event.reason !== "final_answer"; const blockedStop = event.type === "run_stopped" && ["approval_required", "user_input_required", "blocked_plan", "cancelled"].includes( event.reason, ); options.trace.record({ runId: state.runId, type: event.type, startedAt: new Date().toISOString(), outcome: event.type.endsWith("started") ? "started" : blockedStop ? "blocked" : failedStop ? "failed" : "passed", ...(failedStop ? { errorCode: event.reason } : {}), }); options.onEvent?.(event);}注意三个细节:buildModelRequest 必须在循环内执行,且续轮必须传入 request.input,才能真正带上新的计划状态、Skill、Harness feedback 与 observation;当前批次的大型工具输出只通过 function_call_output 发送一次,动态上下文保留其索引与摘要,避免重复占用预算;模型 API 自身失败属于 model_error,Loop 直接停止,重试策略应封装在 Model Adapter 内部。
Step 5追踪一次完整运行
以“读取 package.json 并解释 scripts”为例,外部可观察轨迹应类似:
- 初始化任务与计划:创建 Goal、验收标准和
inspect-packageStep; - 选择 Step:
inspect-package从pending进入in_progress; - Model 请求
read_file:保存responseId并增加模型计数; - Executor 读取文件:记录 observation、
call_id和 Trace; - 评估步骤:observation 满足完成证据,Step 进入
completed; - Model 收到
function_call_output:重新注入当前状态; - Model 返回解释文本:Harness 确认计划和验收条件全部通过;
- Harness 完成:记录 usage、
status=completed与stopReason=final_answer。
如果模型请求未知工具,Executor 不抛出未处理异常,而是生成 unknown_tool observation,模型可以换用已有工具。如果计划或验收条件没有完成,即使模型提前输出总结也会被 Harness 拒绝。综合工程任务留到最后的 Capstone,用任务矩阵而不是单条轨迹验收。
Step 6测试循环而不是测试模型
创建 tests/agent-loop.test.ts。Fake Model 固定返回两个 turn:先请求工具,再检查 observation 并返回答案。
import { describe, expect, it, vi } from "vitest";import { runAgentLoop } from "../src/agent-loop.js";import { InMemoryTraceSink } from "../src/trace.js";it("runs model → tool → observation → model → final", async () => { const start = vi.fn().mockResolvedValue({ responseId: "response-1", finalText: "", toolCalls: [ { callId: "call-1", name: "read_file", argumentsJson: '{"path":"package.json"}' }, ], }); const continueTurn = vi.fn().mockResolvedValue({ responseId: "response-2", finalText: "package.json defines the project scripts.", toolCalls: [], }); const result = await runAgentLoop("Explain package.json", { cwd: "/workspace", skillsDirectory: "/workspace/skills", maxSteps: 4, maxToolCalls: 4, toolTimeoutMs: 1_000, model: { start, continue: continueTurn }, planner: createFakePlanner(), tools: createFakeRegistry(), trace: new InMemoryTraceSink(), }); expect(result.stopReason).toBe("final_answer"); expect(result.state.steps.map((step) => step.kind)).toEqual(["model", "tool", "model"]); expect(continueTurn).toHaveBeenCalledWith( expect.objectContaining({ previousResponseId: "response-1", outputs: [ expect.objectContaining({ type: "function_call_output", call_id: "call-1", }), ], continuationContext: expect.anything(), }), );});再断言 continuationContext 包含工具执行后更新的计划、激活 Skill 与 Harness feedback。继续补充:纯文本提前完成被拒绝、计划无 ready Step、证据不足、计划修订、空输出、模型异常、取消、模型轮次耗尽、工具预算不足、工具错误后恢复,以及一轮多个工具调用。测试断言计划状态、事件和停止原因,不断言模型内部思考。