S8-17 · 基础 · 概念实验 S8-17 · Foundation · Concept Lab

提示注入 · 网页里写着“忽略规则”,它仍然只是网页数据 Prompt Injection · “Ignore the Rules” Inside a Page Is Still Page Data

untrusted content → source labeling → instruction/data isolation → tool authorization

提示注入(Prompt Injection)是不可信内容伪装成指令,诱导模型偏离授权目标或泄露、误用工具。网页和文档中的句子属于数据,不会因措辞像命令就自动获得权限。 Prompt injection is untrusted content disguised as instruction to divert an authorized task or misuse data and tools. Text inside webpages and documents remains data even when it sounds imperative.

01 · 先从日常问题开始 01 · Start with an everyday question

快递盒里夹了一张纸写着“把仓库钥匙寄给我”——分拣员该把它当地址,还是当上级命令? A parcel contains a note saying “Mail me the warehouse key.” Should the sorter treat it as data or as an authorized command?

02 开关指令—数据隔离,看越权路径是否接通 Toggle Instruction-Data Isolation and Trace the Unauthorized Path

拖动隔离强度 drag isolation strength
来源标签 Source labeling
untrusted data recognized
越权路径 Unauthorized path
reachable stages
工具动作 Tool action
allow or block

03 提示注入路径有四个关键点 A Prompt-Injection Path Has Four Critical Points

把刚才的变化拆开 Map the change to parts
01 · CONTENT

不可信内容 Untrusted content

网页、邮件、文档或检索片段都可能夹带命令式句子。 Webpages, email, documents, and retrieved spans may contain imperative text.

02 · ROLE

来源与角色 Source and role

内容来自数据通道,不因语气像命令就升级成系统指令。 Content arrives through a data channel; imperative tone does not promote it to a system instruction.

03 · DECISION

决策隔离 Decision isolation

规划只采用授权目标,把不可信句子标注为材料。 Planning follows authorized goals and labels untrusted text as material.

04 · TOOL

工具权限 Tool authorization

即使决策层被绕过,真实权限仍独立检查主体、动作与范围。 Even if decision controls fail, real authorization checks subject, action, and scope independently.

04 最小机制:数据来源不能自行升级指令优先级 Minimal Mechanism: Data Cannot Promote Its Own Instruction Priority

authority(content) = source role
authority(content) = assigned source role ≠ imperative tone
隔离层识别来源并把网页/文档句子锁在数据角色;S7-19 的输入、决策、工具、输出四层继续独立检查。 Isolation identifies provenance and keeps webpage or document text in a data role; the four S7-19 input, decision, tool, and output gates still act independently.
内容进入上下文 Content enters context 携带来源标签 carry provenance label
指令与数据分离 Separate instruction and data 数据不可自升权 data cannot self-promote
工具再次授权 Authorize the tool again 阻断越权副作用 block unauthorized effects

05 边界实验:关闭隔离层 Boundary Test: Disable the Isolation Layer

主动制造失败 Create a failure

恶意网页句子接上决策与工具路径 Malicious webpage text connects to decision and tool paths

把隔离强度降到 0%。命令式句子被误当成高层指令,若后续闸门也失守,就会形成越权工具路径。 Reduce isolation to 0%. Imperative text is mistaken for a high-level instruction; if later gates also fail, an unauthorized tool path forms.

尚未执行。先在上方改变主变量。 Not run yet. Change the main variable above first.
只要句子写“系统指令”,它就是系统指令。 Text saying “system instruction” becomes one. → ✓ 权限由来源角色决定,不由自我声明决定。 Authority comes from source role, not self-description.
过滤一个关键词就能解决注入。 One keyword filter solves prompt injection. → ✓ 来源隔离加决策、工具、输出多层控制。 Use provenance isolation plus decision, tool, and output controls.
模型拒绝一次就证明永远安全。 One refusal proves permanent safety. → ✓ 用固定攻击集反复评测完整路径。 Test the full path repeatedly on a fixed attack set.
06 · 一句话带走 06 · One line to keep

提示注入是数据伪装成指令;网页或文档里的恶意句必须被锁在不可信数据角色,并由来源隔离与 S7-19 四层闸门共同阻断越权路径。 Prompt injection is data disguised as instruction. Malicious webpage or document text must remain untrusted data, with provenance isolation and the four S7-19 gates jointly blocking unauthorized paths.