Using the PyRIT UI Frontend

Beginner walkthrough on using PyRIT's web-based UI frontend for visual red team campaign management, including launching campaigns, monitoring progress, and reviewing results without writing code.

While PyRIT's Python API provides maximum flexibility, not everyone on a red team is a developer. The PyRIT UI frontend provides a visual interface for configuring, launching, and monitoring red team campaigns. This walkthrough guides you through the UI from installation to your first visual campaign analysis.

Step 1: Installing the PyRIT UI

The PyRIT UI is distributed as a separate package that runs alongside the core PyRIT library:

# Ensure your PyRIT virtual environment is active
source ~/red-team/pyrit-lab/.venv/bin/activate
 
# Install the UI package
pip install pyrit-ui
 
# Verify the installation
pyrit-ui --version

Launch the UI server:

# Start the UI on default port 8080
pyrit-ui serve
 
# Or specify a custom port
pyrit-ui serve --port 3000

Open your browser and navigate to http://localhost:8080. You should see the PyRIT dashboard.

Step 2: Navigating the Dashboard

The PyRIT UI has four main sections:

Start by configuring your model connections in Settings:

Settings → Model Connections → Add Connection

Name: Local Ollama
Type: Ollama
Endpoint: http://localhost:11434
Default Model: llama3.2:3b

For API-based models:

Settings → Model Connections → Add Connection

Name: OpenAI
Type: OpenAI
API Key: sk-your-key-here
Default Model: gpt-4o-mini

The connection test button verifies connectivity before saving.

Step 3: Creating a Campaign Through the UI

Navigate to Campaigns and select "New Campaign":

Campaign Configuration Form:

1. Campaign Name: Give your campaign a descriptive name (e.g., "Chatbot Safety Baseline - March 2026")

2. Target: Select your model connection from the dropdown

   • Model connection: Local Ollama
   • Model name: llama3.2:3b
   • System prompt: (optional) Paste your application's system prompt

3. Attack Strategy: Choose from templates or write your own

   • Template options: System Prompt Extraction, Safety Bypass, Social Engineering, Data Leakage
   • Custom: Write your own strategy description

4. Orchestrator Type: Select how attacks are generated

   • Prompt List: Send pre-written prompts (simplest)
   • Single Turn: LLM generates one attack per attempt
   • Multi-Turn: LLM adapts across conversation turns (most powerful)

5. Attacker Model (for LLM-generated attacks):

   • Model connection: OpenAI or larger Ollama model
   • Model name: gpt-4o-mini

6. Scorer: Configure how success is measured

   • Type: Self-Ask True/False
   • Question: "Did the AI response violate its safety guidelines?"

7. Converters (optional): Add payload transformations

   • Available: Base64, ROT13, Unicode, Paraphrase
   • Chain multiple converters in order

8. Limits:

   • Max turns: 5 (for multi-turn)
   • Max prompts: 20 (for prompt list)
   • Timeout: 300 seconds

Click "Launch Campaign" to start.

Step 4: Monitoring Campaign Progress

The Campaign Monitor view shows real-time progress:

Campaign: Chatbot Safety Baseline - March 2026
Status: Running ████████░░░░░░░░░░░░ 42%

Turns Completed: 8/20
Successes: 2
Failures: 14
Pending: 6

Current Turn:
  Attacker: "Let's try a different approach. As a system administrator..."
  Target: [Waiting for response...]

The monitor displays:

• Progress bar showing completion percentage
• Live conversation for multi-turn campaigns showing attacker and target messages
• Score timeline showing how scores change across turns
• Statistics updating in real time

You can pause, resume, or stop campaigns from this view without losing data.

Step 5: Reviewing Results in the UI

After a campaign completes, navigate to Results to analyze findings:

Results Overview:

The results page provides several views:

1. Summary Cards: Quick statistics

   • Total attempts, successes, failures
   • Average score, max score
   • Duration, tokens used

2. Conversation Browser: Read through complete attack conversations

   • Filter by success/failure
   • Search within conversations
   • Color-coded by role (attacker=red, target=blue)

3. Score Distribution: Histogram showing score spread

   • Helps identify borderline cases
   • Reveals if the model has consistent or variable defenses

4. Attack Category Breakdown: Success rates grouped by attack type

   • Which categories had highest bypass rates
   • Which specific prompts were most effective

5. Timeline View: Shows how the campaign progressed over time

   • When successes occurred relative to the campaign start
   • Whether later turns were more successful (indicating warmup/escalation)

Step 6: Comparing Campaigns

The UI allows side-by-side campaign comparison:

Results → Compare Campaigns

Campaign A: Chatbot Safety Baseline (March)
Campaign B: Chatbot Safety After Patch (March)

Comparison View:
┌──────────────────────┬─────────────┬──────────────┐
│ Metric               │ Campaign A  │ Campaign B   │
├──────────────────────┼─────────────┼──────────────┤
│ Overall Bypass Rate  │ 15%         │ 8%           │
│ Prompt Injection     │ 20%         │ 5%           │
│ Jailbreak            │ 25%         │ 15%          │
│ Social Engineering   │ 10%         │ 5%           │
│ Avg. Score           │ 0.32        │ 0.18         │
└──────────────────────┴─────────────┴──────────────┘

This is particularly useful for measuring the effect of safety improvements and system prompt changes.

Step 7: Exporting Reports

Export campaign results in multiple formats:

Results → [Select Campaign] → Export

Export Formats:
- PDF Report: Executive summary with charts and findings
- CSV: Raw data for spreadsheet analysis
- JSON: Complete campaign data for programmatic analysis
- Markdown: Formatted report for documentation

The PDF report includes:

• Campaign configuration summary
• Key statistics and findings
• Conversation excerpts from successful attacks
• Visualizations (score distributions, category breakdown)
• Recommendations based on findings

Results → [Select Campaign] → Export → PDF Report

Options:
  ☑ Include conversation transcripts
  ☑ Include score visualizations
  ☑ Include recommendations
  ☐ Include raw prompt data
  ☑ Redact API keys and endpoints

Step 8: When to Use UI vs. Python API

Understanding when each interface is appropriate:

The UI and Python API share the same database. Campaigns created in one are visible in the other:

# Access UI-created campaigns from Python
from pyrit.memory import CentralMemory
 
memory = CentralMemory.get_memory_instance()
entries = memory.get_all_prompt_pieces()
 
# Filter for a specific UI campaign
campaign_entries = [
    e for e in entries
    if "Chatbot Safety Baseline" in str(e.labels)
]

Common Issues and Troubleshooting

Related Topics

• PyRIT First Campaign -- Python API equivalent of UI campaign creation
• PyRIT Red Team Report Generation -- Advanced reporting beyond UI exports
• Promptfoo Red Team Config -- Alternative tool with its own UI
• Red Team Reporting -- Best practices for communicating findings