# automation
50 articlestagged with “automation”
Automated AI Incident Triage
Building automated triage systems for AI security incidents using rule-based engines, anomaly detection, and LLM-assisted classification.
Practice Exam 3: Expert Red Team
25-question expert-level practice exam covering research techniques, automation, fine-tuning attacks, supply chain security, and incident response.
Tool Proficiency Assessment
Test your knowledge of AI red teaming tools, frameworks, automation platforms, and their appropriate application in security assessments with 9 intermediate-level questions.
Skill Verification: Red Team Automation
Practical verification of red team automation skills using Garak, PyRIT, and custom tooling.
Advanced Topics Study Guide
Study guide covering AI security research techniques, automation, forensics, emerging attack vectors, and tool development for advanced practitioners.
Capstone: Build a Complete AI Red Teaming Platform
Design and implement a comprehensive AI red teaming platform with automated attack orchestration, vulnerability tracking, and collaborative reporting.
Capstone: Build an AI Security Scanner
Design and implement an automated AI security testing tool that supports prompt injection detection, jailbreak testing, and output analysis.
Cloud AI Compliance Automation
Automating AI compliance checks and security assessments using cloud-native tools and policy-as-code approaches.
Secrets Rotation for Cloud AI Deployments
Implementing automated secrets rotation strategies for API keys, model endpoint credentials, and service accounts used in cloud AI/LLM deployments across AWS, Azure, and GCP.
CI/CD Pipeline AI Risks
Security implications of integrating AI into CI/CD pipelines — covering AI-powered code generation in builds, automated testing risks, deployment decision manipulation, and pipeline hardening.
LLM Security Testing Automation
Building automated security testing pipelines for LLM applications using CI/CD integration and continuous scanning.
Attack Automation Framework
Building end-to-end attack automation frameworks that orchestrate reconnaissance, payload generation, execution, and result analysis.
Red Teaming Automation
Frameworks and tools for automating AI red teaming at scale, including CART pipelines, jailbreak fuzzing, regression testing, and continuous monitoring.
Building Custom Red Team Tools
Guide to building custom AI red teaming tools, including target-specific harnesses, result analysis pipelines, and integration with existing security workflows.
Exploit Chain Builder
Building tools that automatically discover and chain multiple vulnerabilities into complete exploitation paths for complex LLM systems.
AI Exploit Development Overview
An introduction to developing exploits and tooling for AI red teaming, covering the unique challenges of building reliable attacks against probabilistic systems.
Red Team Reporting Automation
Automating report generation from red team testing data and findings.
Continuous Automated Red Teaming (CART)
Designing CART pipelines for ongoing AI security validation: architecture, test suites, telemetry, alerting, regression detection, and CI/CD integration.
Red Team Infrastructure & Tooling
AI red team C2 frameworks, automated attack pipelines, custom scanner development, and integration with Cobalt Strike, Mythic, and Sliver.
Reporting Tool Development
Building automated reporting tools that transform raw test results into professional assessment reports with reproducible findings.
Continuous Compliance Monitoring
Automated compliance monitoring for AI systems including continuous compliance checks, drift detection, regulatory change tracking, and integration with red team testing pipelines.
Building Evaluation Harnesses
Design and implement evaluation harnesses for AI red teaming: architecture patterns, judge model selection, prompt dataset management, scoring pipelines, and reproducible evaluation infrastructure.
Mining and Resource Extraction AI Security
AI security in mining operations including autonomous equipment, geological modeling, and safety systems.
Attacking ML CI/CD Pipelines
Advanced techniques for compromising ML continuous integration and deployment pipelines, including pipeline injection, artifact tampering, training job hijacking, and exploiting the unique trust boundaries in automated ML workflows.
Automated Jailbreak Pipelines
Building automated jailbreak systems with PAIR, TAP, AutoDAN, and custom pipeline architectures for systematic AI safety evaluation.
Injection Research
Advanced research in prompt injection, jailbreak automation, and multimodal attack vectors, covering cutting-edge techniques that push beyond standard injection approaches.
Jailbreak Research & Automation
Taxonomy of jailbreak primitives, crescendo attacks, many-shot jailbreaking, and automated jailbreak generation with TAP and PAIR.
Lab: Red Team Orchestration
Build an orchestration system that coordinates multiple attack strategies simultaneously, managing parallel attack campaigns and synthesizing results into comprehensive risk assessments.
Lab: Safety Regression Testing at Scale
Build automated pipelines that detect safety degradation across model versions, ensuring that updates and fine-tuning do not introduce new vulnerabilities or weaken existing protections.
Lab: Building a Simple Test Harness
Build a reusable Python test harness that automates sending test prompts, recording results, and calculating attack success metrics.
Lab: Build Jailbreak Automation
Build an automated jailbreak testing framework that generates, mutates, and evaluates attack prompts at scale. Covers prompt mutation engines, success classifiers, and campaign management for systematic red team testing.
Lab: Automated Red Team Pipeline
Hands-on lab for building a continuous AI red team testing pipeline using promptfoo, GitHub Actions, and automated attack generation to catch safety regressions before deployment.
Lab: Building an LLM Judge Evaluator
Hands-on lab for building an LLM-based evaluator to score red team attack outputs, compare model vulnerability, and lay the foundation for automated attack campaigns.
Simulation: Defense in Depth
Expert-level defense simulation implementing a full defense stack including input filter, output monitor, rate limiter, anomaly detector, and circuit breaker, then measuring effectiveness against automated attacks.
ML CI/CD Security
Security overview of ML continuous integration and deployment pipelines: how ML CI/CD differs from traditional CI/CD, unique attack surfaces in training workflows, and the security implications of automated model building and deployment.
Developing Custom AI Red Team Tools
Guide to designing, building, and maintaining custom tools for AI red team engagements.
Continuous Red Teaming for Production AI Systems
Implementing ongoing, automated red teaming programs for AI systems in production environments.
Red Team Automation Strategy
When and how to automate AI red teaming: tool selection, CI/CD integration, continuous automated red teaming (CART), human-in-the-loop design, and scaling assessment coverage through automation.
Injection Chain Automation
Automating the discovery and chaining of multiple injection techniques to create reliable multi-step attack sequences against hardened targets.
System Prompt Extraction Techniques
Catalog of system prompt extraction methods against LLM-powered applications: direct attacks, indirect techniques, multi-turn strategies, and defensive evasion.
Continuous Red Teaming Programs
Designing and operating ongoing AI red team programs with automated testing pipelines, metrics dashboards, KPI frameworks, alert-driven assessments, and integration with CI/CD and model deployment workflows.
Automated Defense Testing Pipeline
Build an automated pipeline that continuously tests defensive measures against evolving attack techniques.
Setting Up Continuous AI Red Teaming Pipelines
Walkthrough for building continuous AI red teaming pipelines that automatically test LLM applications on every deployment, covering automated scan configuration, CI/CD integration, alert thresholds, regression testing, and dashboard reporting.
Developing Comprehensive AI Security Test Plans
Step-by-step guide to developing structured test plans for AI red team engagements, covering test case design, automation strategy, coverage mapping, and execution scheduling.
Counterfit Walkthrough
Complete walkthrough of Microsoft's Counterfit adversarial ML testing framework: installation, target configuration, running attacks against ML models, interpreting results, and automating adversarial robustness assessments.
Integrating Garak into CI/CD Pipelines
Intermediate walkthrough on automating garak vulnerability scans within CI/CD pipelines, including GitHub Actions, GitLab CI, threshold-based gating, result caching, and cost management strategies.
Garak End-to-End Walkthrough
Complete walkthrough of NVIDIA's garak LLM vulnerability scanner: installation, configuration, running probes against local and hosted models, interpreting results, writing custom probes, and CI/CD integration.
Automating Red Team Evaluations with Promptfoo
Complete walkthrough for setting up automated red team evaluation pipelines using Promptfoo, covering configuration, custom evaluators, adversarial dataset generation, CI integration, and result analysis.
Python Red Team Automation
Building custom AI red team automation with Python: test harnesses with httpx and aiohttp, result collection and analysis, automated reporting, and integration with existing tools like promptfoo and garak.
Automated Red Team Report Generation
Build an automated system for generating structured red team reports from testing data and findings.