What is jailbreaking in AI?

Explained

Jailbreaking is the use of crafted inputs to make an AI model bypass its safety restrictions and produce output it would normally refuse, such as harmful instructions, confidential information or policy-breaking content. Typical techniques include role-play, hypothetical framing, encoding the request, splitting it into innocent-looking parts or overwhelming the model with examples. Research shows it works because safety training competes with the model's goal of being helpful and does not always generalise to unusual inputs. For companies, jailbreaking is a security and reputational risk, and defending against it is part of AI governance.

Common techniques

• Role-play: asking the model to act as a character without rules.
• Hypothetical or fictional framing: presenting the request as a story, game or research exercise.
• Obfuscation: encoding the request, translating it or using unusual formats the safety training did not cover.
• Decomposition: splitting a forbidden task into small steps that each look harmless.
• Pressure and repetition: long conversations or many examples that gradually shift the model's behaviour.

Jailbreaking vs prompt injection

The two are related and often confused. Jailbreaking aims at the model's own safety rules: the user tries to get it to say or do something it should refuse. Prompt injection aims at the application: instructions hidden in the input or in external content (a web page, an email, a document) override what the developer told the model to do.

OWASP treats jailbreaking as a form of prompt injection in which the attacker makes the model disregard its safety protocols. For agents that can call tools, both matter more, because a successful attack can trigger real actions.

How companies defend against it

• Layered guardrails: input and output filters in addition to the model's own safety training.
• Least privilege: the model and its tools can only reach the data and actions the task needs.
• Separation of instructions and untrusted content in prompts.
• Red teaming and adversarial testing before release and on a regular basis.
• Monitoring and logging to detect attempts and respond to incidents.
• Human approval for high-impact actions.

Frequently asked questions

How to put it to work

• How to red team your AI agents automatically

Further reading

• Automated Red-Teaming: The New Standard for Enterprise AI Safety
• AI Security's New Frontier: When the Model Is the Attacker

Related terms

• AI Governance The policies, processes and controls an organisation uses to make sure its AI is safe, lawful, ethical and worth what it costs.
• Training The process of teaching a model from data by adjusting its parameters, from general pre-training to task-specific fine-tuning.
• Bias Systematic distortion in the data, design or use of an AI system that produces unfair, discriminatory or inaccurate results for some people or cases.
• Constitutional AI A training approach in which a model learns to follow a written set of principles, a constitution, by critiquing and revising its own answers.
• Ethical AI AI designed, built and used according to ethical principles such as fairness, transparency, accountability, privacy and human oversight.
• EU AI Act The European Union regulation that sets rules for AI systems according to their level of risk, from banned uses to transparency duties.