What is Garak Walkthrough?

Complete walkthrough of NVIDIA's garak LLM vulnerability scanner: installation, configuration, running probes against local and hosted models, interpreting results, writing custom probes, and CI/CD integration.

What is PyRIT Walkthrough?

Complete walkthrough of Microsoft's Python Risk Identification Toolkit: setup, connecting to targets, running orchestrators, using converters, multi-turn attacks, and analyzing results with the web UI.

What is Promptfoo Walkthrough?

Complete walkthrough of promptfoo for AI red teaming: configuration files, provider setup, running evaluations, red team plugins, assertion-based scoring, reporting, and CI/CD integration.

What is Burp Suite for AI APIs?

Using Burp Suite to intercept, analyze, and fuzz LLM API calls: proxy setup, intercepting streaming responses, parameter fuzzing with Intruder, and building custom extensions for AI-specific testing.

What is Inspect AI Walkthrough?

Complete walkthrough of UK AISI's Inspect AI framework: installation, writing evaluations, running against models, custom scorers, benchmark suites, and producing compliance-ready reports.

What is Ollama for Red Teaming?

Using Ollama as a local red teaming environment: model selection, running uncensored models, API-based testing, comparing safety across model families, and building a cost-free testing lab.

What is Python Automation?

Building custom AI red team automation with Python: test harnesses with httpx and aiohttp, result collection and analysis, automated reporting, and integration with existing tools like promptfoo and garak.

What is Counterfit Walkthrough?

Complete walkthrough of Microsoft's Counterfit adversarial ML testing framework: installation, target configuration, running attacks against ML models, interpreting results, and automating adversarial robustness assessments.

What is HarmBench Walkthrough?

Complete walkthrough of the HarmBench evaluation framework: installation, running standardized benchmarks against models, interpreting results, creating custom behavior evaluations, and comparing model safety across versions.

What is NeMo Guardrails Walkthrough?

End-to-end walkthrough of NVIDIA NeMo Guardrails: installation, Colang configuration, dialog flow design, integration with LLM applications, and red team bypass testing techniques.

工具導覽

入門2 分鐘閱讀更新於 2026-03-15

必備 AI 紅隊演練工具的端對端實務導覽，涵蓋安裝、設定、執行與結果詮釋。

tools walkthroughs garak pyrit promptfoo burp-suite inspect-ai ollama python

AI 紅隊演練生態系已顯著成熟。本節提供你在專業案件中最常使用之工具的動手、逐步導覽。

為何工具熟練度重要

有效 AI 紅隊演練不是執行單一掃描器並交出報告。它需要分層多個工具、理解每個測試什麼，以及知道何時從自動化探測切換到手動探索。

工具選擇矩陣

工具	主要用途	優勢	限制
Garak	自動化漏洞掃描	廣泛探測庫、可擴展	未調整可能產生誤報
PyRIT	編排攻擊活動	多輪編排器、轉換器	學習曲線較陡
Promptfoo	評估驅動紅隊	CI/CD 整合、宣告式設定	聚焦提示詞層級測試
Burp Suite	API 層級攔截	深度 HTTP 檢查、模糊測試	需要代理設定
Inspect AI	結構化評估	基準套件、自訂評分器	評估導向，非攻擊導向
Ollama	本地模型測試	無 API 成本、未審查模型	限於本地硬體可容納模型
Python 自動化	自訂測試 harness	完全彈性、API 整合	需要開發努力

建議工具進程

從 Ollama 開始 — 設置無 API 成本或速率限制的本地測試環境
學習 Promptfoo — 其宣告式 YAML 設定是最容易的系統測試入口
轉向 Garak — garak 探測庫讓你大規模掃描已知漏洞模式
加入 PyRIT — 對精密多輪攻擊與編排活動提供自動化框架
疊加 Burp Suite — 需要檢視客戶端與 API 間實際線上傳輸時使用
使用 Inspect AI — 對照基準套件的正式評估，特別是與治理團隊合作時
建構自訂自動化 — 當現成工具未涵蓋特定目標時，Python 自動化填補落差

案件階段對應

偵察階段

Burp Suite — 攔截 API 呼叫以理解端點與認證
Python 自動化 — 腳本化模型能力與 API 參數發現
Ollama — 在對生產目標執行前本地測試攻擊假設

主動測試階段

Garak — 廣泛自動化掃描已知漏洞模式
PyRIT — 具自動越獄升級的編排多輪攻擊活動
Promptfoo — 具斷言的特定攻擊向量系統評估

驗證與報告階段

Inspect AI — 合規文件的正式基準評估
Promptfoo — 驗證修復有效性的迴歸測試
Python 自動化 — 自訂報告與證據收集

環境設置

所有導覽假設 Linux 或 macOS 環境。你需要：Python 3.10+、Node.js 18+、Docker（選用但建議）、8GB+ RAM（本地模型最低）、API 金鑰。

# Verify your environment
python3 --version   # 3.10+
node --version      # 18+
docker --version    # Optional
ollama --version    # Install from ollama.com if needed