INSPIRE: Idea-Guided Attacks on Policy-Adherent Agents

Large Language Model (LLM) agents are increasingly deployed in customer service, where they are given a natural-language policy and perform financial actions on customer accounts. Existing attacks rely only on the attacker LLM to find ideas for violating the policy, but it tends to stay close to its first ideas. We propose INSPIRE, which inspires the attacker LLM when it is stuck, by looking for an idea in a structured space of composable ideas and learning which to avoid. We evaluate INSPIRE on four domains, four attacker LLMs, and two state-of-the-art agents. It outperforms CRAFT, the state-of-the-art attacker, by 11.7 percentage points of pass@1 on average and up to 20.4. We also propose an idea-aware guard, which verifies the user’s claims in the agent’s records and lowers INSPIRE’s pass@4 from 79% to at most 17%.

M.Sc. student under the supervision of Associate Prof. Dana Drachsler Cohen.