Making use of Safety Engineering to Immediate Injection Safety
This looks like an necessary advance in LLM safety towards immediate injection:
Google DeepMind has unveiled CaMeL (CApabilities for MachinE Studying), a brand new method to stopping prompt-injection assaults that abandons the failed technique of getting AI fashions police themselves. As an alternative, CaMeL treats language fashions as essentially untrusted elements inside a safe software program framework, creating clear boundaries between consumer instructions and doubtlessly malicious content material.
[…]
To know CaMeL, you must perceive that immediate injections occur when AI methods can’t distinguish between professional consumer instructions and malicious directions hidden in content material they’re processing.
[…]
Whereas CaMeL does use a number of AI fashions (a privileged LLM and a quarantined LLM), what makes it progressive isn’t decreasing the variety of fashions however essentially altering the safety structure. Fairly than anticipating AI to detect assaults, CaMeL implements established safety engineering ideas like capability-based entry management and information movement monitoring to create boundaries that stay efficient even when an AI part is compromised.
Analysis paper. Good evaluation by Simon Willison.
I wrote about the issue of LLMs intermingling the info and management paths right here.
Posted on April 29, 2025 at 7:03 AM •
2 Feedback
Sidebar picture of Bruce Schneier by Joe MacInnis.
