Red Team Automation

Automate red teaming of generative AI to identify security and ethical risks efficiently.

Business impact

• Time to identify AI system vulnerabilities — Automation reduces manual effort, accelerating detection of security and ethical risks
• Coverage of risk scenarios tested — Automated prompt generation enables broader exploration of potential AI failure modes
• Expert resource utilization — Security professionals focus on strategy while automation handles repetitive tasks

Data requirements

• Harmful prompt datasets (Text) — Used as initial input to generate and explore additional risky prompts
• Generative AI model outputs (Text, Audio, Image, Video) — Analyzed to identify vulnerabilities and guide next attack inputs
• Security and responsible AI risk assessments (Structured) — Inform scoring and prioritization of identified risks during red teaming

AI methods and techniques

• Generative AI — Generate diverse harmful prompts to probe AI system vulnerabilities
• Predictive AI — Score and prioritize risks based on model outputs and threat likelihood

AI models and model families

GPT-4, Claude, Llama 2