AI Vocabulary · 07 When It Goes Wrong
What is Jailbreak?
A user's attempt to talk an AI system out of its own rules, for example to get content it is meant to refuse.
A jailbreak is when someone crafts inputs, such as role-play setups, elaborate hypotheticals or unusual encodings, to make a model ignore its safety rules or its system prompt. Providers keep improving defences, and attackers keep finding new tricks.
It is related to but different from prompt injection. In a jailbreak, the person typing is the attacker. In prompt injection, the attack hides in content the AI reads, and the user may be the victim.
Why it matters for you
If you put an AI assistant in front of customers, some users will try to make it say embarrassing things, reveal its instructions or promise discounts. Assume anything in the system prompt can eventually be extracted.
Own it
Don't put secrets in prompts, limit what a public-facing assistant can actually do, add guardrails outside the model, and test your own assistant with typical attacks before launch. Decide in advance how you handle screenshots of it misbehaving.
Where it comes from
The word was borrowed from phones: "jailbreaking" an iPhone meant removing Apple's restrictions so you could install unapproved software. With chatbots, it came to mean prompts that talk a model out of its safety rules. One famous early example is "DAN" ("Do Anything Now"), first posted on Reddit's r/ChatGPT in December 2022 (Know Your Meme). It is different from prompt injection, where the attack hides in content the AI reads.
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.