S7-08 · 基础 · 点名概念 S7-08 · Foundation · Core Concept

Harness · 把模型装进可控外壳 Agent Harness · Put the Model Inside a Controlled Shell

same model + state + validation + permission + retry + trace + stop

Harness 是包围模型的运行框架,负责状态、工具、校验、重试、权限、记录与停止。生产可靠性主要来自可控外壳,不能归功于某一次模型回答。 An Agent Harness is the runtime around a model, responsible for state, tools, validation, retries, permission, records, and stopping. Production reliability belongs largely to the controlled shell, not one model reply.

01 · 同一台发动机,两种整车结果 01 · Same engine, different vehicle

模型完全不换,只加校验、权限、重试和日志,故障任务成功率会变吗? If the model stays identical while validation, permission, retries, and logs change, does fault-task success change?

02 Harness 校验台:逐个拆掉外壳 Harness Test Rig: Remove the Shell One Switch at a Time

同模型 · 同测试集 same model · same batch
安全成功率 Safe success rate
100%
20/20 tasks
风险事件率 Risk event rate
0%
0/20 tasks
执行尝试 Execution attempts
24
含受控重试 includes controlled retries

03 四个开关,各拦一种故障 Four Switches, Four Fault Classes

局部控制 → 端到端结果 local control → outcome
SCHEMA · 4 TASKS

挡住畸形调用 Catch malformed calls

参数缺失或类型错误时不交给真实工具,并把错误送回修正。 Do not hand missing or mistyped arguments to the real tool; return the error for correction.

PERMISSION · 4 TASKS

挡住越权动作 Block unauthorized actions

模型提出的动作必须按主体、资源和范围重新授权。 Every proposed action is authorized again against subject, resource, and scope.

RETRY · 4 TASKS

恢复瞬时失败 Recover transient faults

只对可重试错误限次重放;不是对所有失败无限再来一次。 Retry only eligible transient faults with a limit—not every failure forever.

LOGS · 2 TASKS

让停止线看见重复 Make repetition visible

结构化日志保存动作与结果,停止规则才能识别“同错又来”。 Structured logs preserve actions and results so a stop rule can detect the same failure repeating.

04 最小测量:分母固定,才能归因于外壳 Minimal Measurement: Fix the Denominator to Attribute the Shell

safe success / 20
Rsafe = safe completions / 20
模型 θ、任务集 D 和任务顺序都锁定。只有 Harness 开关 H 变化,因此本页差异来自教学模拟中的 H,而不是换模型。 Model θ, task set D, and task order are locked. Only Harness configuration H changes, so this toy difference is attributable to H—not a model swap.
固定 θ 与 D Hold θ and D 同模型、同 20 题 same model, same 20 tasks
只切 H Change only H 四个外壳控制 four shell controls
比较成功与风险 Compare success and risk 不能只看“答得像” not just answer appearance

05 边界实验:只剩“多试几次” Boundary Test: Keep Only “Try Again”

主动制造失败 Create a failure

关闭校验、权限和日志,只开重试 Disable validation, permission, and logs; leave retry on

瞬时错误恢复了,但畸形调用、越权动作和重复循环都暴露。成功率与风险必须同时看。 Transient failures recover, while malformed calls, unauthorized actions, and loops remain exposed. Success and risk must be read together.

尚未执行。完整 Harness 当前开启。 Not run yet. The full Harness is enabled.
模型这次答对,系统就可靠。 One correct model answer makes the system reliable. → ✓ 要在固定故障集上测外壳后的安全成功率。 Measure safe success on a fixed fault set with the shell in place.
重试越多,成功率一定越高。 More retries always improve success. → ✓ 越权与结构错误重试只会重复风险。 Retrying authorization or schema failures merely repeats risk.
日志只是事后排错,不影响运行控制。 Logs are only for postmortems, not runtime control. → ✓ 结构化 trace 可供重复检测与停止线读取。 Structured traces can feed repetition detection and stop rules.
06 · 一句话带走 06 · One line to keep

Harness 是模型外的可控运行框架;固定同一模型与故障集后,Schema、权限、受控重试和日志/停止线的开关会直接改变安全成功率与风险。 An Agent Harness is the controlled runtime around a model. With the same model and fault set fixed, schema, permission, retry, and log/stop controls directly change safe success and risk.