S7-08 · 基础 · 点名概念S7-08 · Foundation · Core Concept
Harness · 把模型装进可控外壳Agent Harness · Put the Model Inside a Controlled Shell
same model + state + validation + permission + retry + trace + stop
Harness
是包围模型的运行框架,负责状态、工具、校验、重试、权限、记录与停止。生产可靠性主要来自可控外壳,不能归功于某一次模型回答。An Agent Harness is the runtime around a model, responsible for state,
tools, validation, retries, permission, records, and stopping.
Production reliability belongs largely to the controlled shell, not
one model reply.
01 · 同一台发动机,两种整车结果01 · Same engine, different vehicle
模型完全不换,只加校验、权限、重试和日志,故障任务成功率会变吗?If the model stays identical while validation, permission, retries,
and logs change, does fault-task success change?
[M]
02Harness 校验台:逐个拆掉外壳Harness Test Rig: Remove the Shell One Switch at a Time
同模型 · 同测试集same model · same batch
安全成功率Safe success rate
100%
20/20 tasks
风险事件率Risk event rate
0%
0/20 tasks
执行尝试Execution attempts
24
含受控重试includes controlled retries
03四个开关,各拦一种故障Four Switches, Four Fault Classes
局部控制 → 端到端结果local control → outcome
SCHEMA · 4 TASKS
挡住畸形调用Catch malformed calls
参数缺失或类型错误时不交给真实工具,并把错误送回修正。Do not hand missing or mistyped arguments to the real tool;
return the error for correction.
PERMISSION · 4 TASKS
挡住越权动作Block unauthorized actions
模型提出的动作必须按主体、资源和范围重新授权。Every proposed action is authorized again against subject,
resource, and scope.
RETRY · 4 TASKS
恢复瞬时失败Recover transient faults
只对可重试错误限次重放;不是对所有失败无限再来一次。Retry only eligible transient faults with a limit—not every
failure forever.
LOGS · 2 TASKS
让停止线看见重复Make repetition visible
结构化日志保存动作与结果,停止规则才能识别“同错又来”。Structured logs preserve actions and results so a stop rule can
detect the same failure repeating.
04最小测量:分母固定,才能归因于外壳Minimal Measurement: Fix the Denominator to Attribute the
Shell
safe success / 20
Rsafe = safe completions / 20
模型 θ、任务集 D 和任务顺序都锁定。只有 Harness
开关 H 变化,因此本页差异来自教学模拟中的
H,而不是换模型。Model θ, task set D, and task order are locked. Only Harness
configuration H changes, so this toy difference is attributable
to H—not a model swap.
固定 θ 与 DHold θ and D同模型、同 20 题same model, same 20 tasks
只切 HChange only H四个外壳控制four shell controls
比较成功与风险Compare success and risk不能只看“答得像”not just answer appearance
05边界实验:只剩“多试几次”Boundary Test: Keep Only “Try Again”
主动制造失败Create a failure
关闭校验、权限和日志,只开重试Disable validation, permission, and logs; leave retry on
瞬时错误恢复了,但畸形调用、越权动作和重复循环都暴露。成功率与风险必须同时看。Transient failures recover, while malformed calls, unauthorized
actions, and loops remain exposed. Success and risk must be read
together.
尚未执行。完整 Harness 当前开启。Not run yet. The full Harness is enabled.
✗ 模型这次答对,系统就可靠。One correct model answer makes the system reliable.
→ ✓ 要在固定故障集上测外壳后的安全成功率。Measure safe success on a fixed fault set with the shell in
place.
✗ 日志只是事后排错,不影响运行控制。Logs are only for postmortems, not runtime control.
→ ✓ 结构化 trace 可供重复检测与停止线读取。Structured traces can feed repetition detection and stop
rules.
06 · 一句话带走06 · One line to keep
Harness
是模型外的可控运行框架;固定同一模型与故障集后,Schema、权限、受控重试和日志/停止线的开关会直接改变安全成功率与风险。An Agent Harness is the controlled runtime around a model. With the
same model and fault set fixed, schema, permission, retry, and
log/stop controls directly change safe success and risk.