Stealth edits to large language models

We reveal the theoretical foundations of techniques for editing large language
models, and present new methods which can do so without requiring retraining. Our
theoretical insights show that a single metric (a measure of the intrinsic dimension
of the model’s features) can be used to assess a model’s editability and reveals its
previously unrecognised susceptibility to malicious stealth attacks. This metric
is fundamental to predicting the success of a variety of editing approaches, and
reveals new bridges between disparate families of editing methods. We collectively
refer to these as stealth editing methods, because they directly update a model’s
weights to specify its response to specific known hallucinating prompts without
affecting other model behaviour. By carefully applying our theoretical insights,
we are able to introduce a new jet-pack network block which is optimised for
highly selective model editing, uses only standard network operations, and can
be inserted into existing networks. We also reveal the vulnerability of language
models to stealth attacks: a small change to a model’s weights which fixes its
response to a single attacker-chosen prompt. Stealth attacks are computationally
simple, do not require access to or knowledge of the model’s training data, and
therefore represent a potent yet previously unrecognised threat to redistributed
foundation models. Extensive experimental results illustrate and support our
methods and their theoretical underpinnings. Demos and source code are available
at https://github.com/qinghua-zhou/stealth-edits.