Evals

Workhorse Articles Home

Pricing AI risk starts with knowing what can already be measured. This is the index of open-source AI risk and cyber evals, benchmarks, and red-team tools we track to answer that question.

297 catalog entries across 20 categories, mapped against 11 insurance exposures — 159 are evals or test environments we would score a model against, and the other 138 are tools, harnesses, taxonomies, and reference lists we track because they shape how AI risk gets measured, not because they are evals themselves. Licenses are resolved directly from each repository, never guessed — an entry we could not verify is marked “unknown,” not left blank or assumed permissive.

Coverage map: category × insurance exposure
Category \ Exposure	Unmapped	Agent actions	All	Breach/cyber	Correlated/systemic	Data poisoning	Hallucination	Model failure	Model/weight theft	Privacy	Prompt injection	Regulatory/bias
Adversarial ML (classic)
Agent safety & harmful actions
AI control & monitoring
Autonomy, R&D & self-replication
Bias, fairness & societal harm
CBRN & dual-use knowledge
Cyber defense & threat intel
Cyber offense capability
Deception, scheming & alignment auditing
Eval frameworks & harnesses
Guardrails & safety classifiers (adjacent)
Hallucination & factuality
Jailbreak & misuse
Meta-lists & catalogs
Model-internals auditing
Prompt injection & agent security
Red-team & scanning tools
Risk taxonomies, incident data & standards
Secure code & vuln detection
Vulnerable labs & CTF targets

Shaded by count. Click a cell to see the evals in it and filter the table below — click again to clear.

SearchCategoryExposureType297 of 297

Name	Category ▲	Type	Insurance exposure	License	Stars
Adversarial Robustness Toolbox (ART)(Tool)	Adversarial ML (classic)	Tool	Model/weight theft	MIT	6279
CleverHans(Tool)	Adversarial ML (classic)	Tool	Model failure	MIT	6458
Counterfit(Tool)	Adversarial ML (classic)	Tool	Model failure	MIT	945
Foolbox(Tool)	Adversarial ML (classic)	Tool	Model failure	MIT	2984
Scaling LLM Robustness	Adversarial ML (classic)	Benchmark / dataset	Model failure	MIT	2
TextAttack(Tool)	Adversarial ML (classic)	Tool	Model failure	MIT	3477
Agent-SafetyBench	Agent safety & harmful actions	Benchmark / dataset	Agent actions	MIT	165
AgentDAM	Agent safety & harmful actions	Benchmark / dataset	Privacy	unknown	49
AgentHarm	Agent safety & harmful actions	Benchmark / dataset	Agent actions	MIT	704^†
AgentLeak	Agent safety & harmful actions	Benchmark / dataset	Privacy	MIT	34
MACHIAVELLI	Agent safety & harmful actions	Benchmark / dataset	Agent actions	MIT	156
OpenAgentSafety	Agent safety & harmful actions	Benchmark / dataset	Agent actions	MIT	37
OS-Harm	Agent safety & harmful actions	Benchmark / dataset	Agent actions	Apache-2.0	78
PersistBench	Agent safety & harmful actions	Benchmark / dataset	Privacy	MIT	704^†
R-Judge	Agent safety & harmful actions	Benchmark / dataset	Agent actions	unknown	116
SafeAgentBench	Agent safety & harmful actions	Benchmark / dataset	Agent actions	unknown	79
SafeArena	Agent safety & harmful actions	Benchmark / dataset	Agent actions	unknown	26
ST-WebAgentBench	Agent safety & harmful actions	Benchmark / dataset	Agent actions	Apache-2.0	29
ToolEmu	Agent safety & harmful actions	Benchmark / dataset	Agent actions	Apache-2.0	226
Agent Governance Toolkit(Tool)	AI control & monitoring	Tool	Agent actions	MIT	6430
Arize Phoenix(Tool)	AI control & monitoring	Tool	Hallucination	unknown	11859
BashArena	AI control & monitoring	Benchmark / dataset	Agent actions	unknown	16
ControlArena(Framework / harness)	AI control & monitoring	Framework / harness	Agent actions	MIT	253
Docent(Tool)	AI control & monitoring	Tool	Model failure	Apache-2.0	141
Inspect Scout(Tool)	AI control & monitoring	Tool	Model failure	MIT	82
LangKit(Tool)	AI control & monitoring	Tool	Privacy	Apache-2.0	1001
SHADE-Arena	AI control & monitoring	Benchmark / dataset	Agent actions	MIT	60
SLEIGHT-Bench	AI control & monitoring	Benchmark / dataset	Agent actions	MIT	16
GDM Self-Proliferation	Autonomy, R&D & self-replication	Benchmark / dataset	Agent actions	MIT	704^†
Humanity's Last Exam	Autonomy, R&D & self-replication	Benchmark / dataset	Unmapped	MIT	1715
METR Public Tasks	Autonomy, R&D & self-replication	Benchmark / dataset	Agent actions	MIT	136
METR Time Horizons	Autonomy, R&D & self-replication	Benchmark / dataset	Correlated/systemic	unknown	338
MLE-bench	Autonomy, R&D & self-replication	Benchmark / dataset	Unmapped	MIT	1775
OpenAI Frontier Evals (PaperBench, SWE-Lancer, MLE-bench...)	Autonomy, R&D & self-replication	Benchmark / dataset	Unmapped	MIT	1317
RE-Bench	Autonomy, R&D & self-replication	Benchmark / dataset	Unmapped	MIT	171
RE-Bench (Inspect port)	Autonomy, R&D & self-replication	Benchmark / dataset	Unmapped	MIT	4
BBQ	Bias, fairness & societal harm	Benchmark / dataset	Regulatory/bias	CC-BY-4.0	153
DecodingTrust	Bias, fairness & societal harm	Benchmark / dataset	Regulatory/bias	CC-BY-SA-4.0	317
Discrim-Eval	Bias, fairness & societal harm	Benchmark / dataset	Regulatory/bias	unknown	—
Political Even-handedness Eval	Bias, fairness & societal harm	Benchmark / dataset	Regulatory/bias	CC-BY-4.0	139
RealToxicityPrompts	Bias, fairness & societal harm	Benchmark / dataset	Regulatory/bias	Apache-2.0	235
ToxiGen	Bias, fairness & societal harm	Benchmark / dataset	Regulatory/bias	MIT	349
TrustLLM	Bias, fairness & societal harm	Benchmark / dataset	Regulatory/bias	MIT	632
LAB-Bench	CBRN & dual-use knowledge	Benchmark / dataset	Unmapped	CC-BY-SA-4.0	133
SOSBench	CBRN & dual-use knowledge	Benchmark / dataset	Regulatory/bias	MIT	704^†
Virology Capabilities Test (VCT)	CBRN & dual-use knowledge	Benchmark / dataset	Unmapped	unknown	—
WMDP	CBRN & dual-use knowledge	Benchmark / dataset	Breach/cyber	MIT	187
ACES (Agent Capability Evaluation Suite)(Framework / harness)	Cyber defense & threat intel	Framework / harness	Breach/cyber	MIT	18
CS-Eval	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	MIT	67
CTI-REALM	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	MIT	704^†
CTIBench	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	unknown	94
CyberMetric	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	unknown	126
ExCyTIn-Bench	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	MIT	160
Purple Llama (CyberSecEval 1-4, AutoPatchBench, CyberSOCEval, LlamaFirewall, Llama Guard)	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	unknown	4433
SecEval	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	unknown	116
SecQA (Inspect)	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	MIT	704^†
SECURE	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	unknown	17
SEvenLLM	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	unknown	39
VLoc Bench	Cyber defense & threat intel	Benchmark / dataset	Breach/cyber	Apache-2.0	31
3CB (Catastrophic Cyber Capabilities Benchmark)	Cyber offense capability	Benchmark / dataset	Breach/cyber	unknown	17
3CB (Inspect port)	Cyber offense capability	Benchmark / dataset	Breach/cyber	MIT	704^†
AI-Pentest-Benchmark	Cyber offense capability	Benchmark / dataset	Breach/cyber	MIT	25
AIRTBench	Cyber offense capability	Benchmark / dataset	Prompt injection	Apache-2.0	111
Atlantis (AIxCC winner)(Tool)	Cyber offense capability	Tool	Breach/cyber	GPL	654
AutoPenBench	Cyber offense capability	Benchmark / dataset	Breach/cyber	MIT	104
BountyBench	Cyber offense capability	Benchmark / dataset	Breach/cyber	Apache-2.0	112
Buttercup (AIxCC CRS)(Tool)	Cyber offense capability	Tool	Breach/cyber	AGPL-3.0	1725
CAI (Cybersecurity AI)(Framework / harness)	Cyber offense capability	Framework / harness	Breach/cyber	MIT	9835
CVE-Bench	Cyber offense capability	Benchmark / dataset	Breach/cyber	Apache-2.0	298
Cybench	Cyber offense capability	Benchmark / dataset	Breach/cyber	Apache-2.0	342
CyberBattleSim	Cyber offense capability	Environment / sim	Breach/cyber	MIT	1803
CyberGym	Cyber offense capability	Benchmark / dataset	Breach/cyber	Apache-2.0	956
CybORG	Cyber offense capability	Environment / sim	Breach/cyber	MIT	133
GDM Dangerous Capability Evaluations	Cyber offense capability	Benchmark / dataset	Breach/cyber	Apache-2.0	75
GDM In-House CTF	Cyber offense capability	Benchmark / dataset	Breach/cyber	MIT	704^†
GDM InterCode-CTF	Cyber offense capability	Benchmark / dataset	Breach/cyber	MIT	704^†
hackingBuddyGPT(Framework / harness)	Cyber offense capability	Framework / harness	Breach/cyber	MIT	1273
HackSynth(Framework / harness)	Cyber offense capability	Framework / harness	Breach/cyber	AGPL-3.0	319
Incalmo(Tool)	Cyber offense capability	Tool	Breach/cyber	MIT	144
InterCode-CTF	Cyber offense capability	Environment / sim	Breach/cyber	MIT	258
Linux PrivEsc Benchmark	Cyber offense capability	Benchmark / dataset	Breach/cyber	unknown	52
NASim	Cyber offense capability	Environment / sim	Breach/cyber	MIT	208
NYU CTF Agents(Tool)	Cyber offense capability	Tool	Breach/cyber	MIT	170
NYU CTF Bench	Cyber offense capability	Benchmark / dataset	Breach/cyber	GPL-2.0	182
Palisade InterCode-CTF agents(Tool)	Cyber offense capability	Tool	Breach/cyber	MIT	71
PentAGI(Tool)	Cyber offense capability	Tool	Breach/cyber	MIT	30465
PentestGPT(Tool)	Cyber offense capability	Tool	Breach/cyber	MIT	15962
SEC-bench	Cyber offense capability	Benchmark / dataset	Breach/cyber	MIT	101
Shannon(Tool)	Cyber offense capability	Tool	Breach/cyber	AGPL-3.0	48926
Strix(Tool)	Cyber offense capability	Tool	Breach/cyber	Apache-2.0	68822
XBOW Validation Benchmarks	Cyber offense capability	Benchmark / dataset	Breach/cyber	Apache-2.0	727
Agentic Misalignment(Framework / harness)	Deception, scheming & alignment auditing	Framework / harness	Agent actions	MIT	661
Alignment Faking (open replication)(Framework / harness)	Deception, scheming & alignment auditing	Framework / harness	Model failure	MIT	60
AuditBench	Deception, scheming & alignment auditing	Benchmark / dataset	Model failure	unknown	34
Bloom (Petri Bloom)(Tool)	Deception, scheming & alignment auditing	Tool	Model failure	MIT	38
Capture the Fish (spec gaming)	Deception, scheming & alignment auditing	Environment / sim	Agent actions	unknown	25
DarkBench	Deception, scheming & alignment auditing	Benchmark / dataset	Regulatory/bias	MIT	19
Deception Detection Probes(Tool)	Deception, scheming & alignment auditing	Tool	Model failure	unknown	58
Find the Trojan	Deception, scheming & alignment auditing	Benchmark / dataset	Data poisoning	Apache-2.0	119
GDM Self-Reasoning	Deception, scheming & alignment auditing	Benchmark / dataset	Agent actions	MIT	704^†
GDM Stealth	Deception, scheming & alignment auditing	Benchmark / dataset	Agent actions	MIT	704^†
Insider Trading (deception under pressure)	Deception, scheming & alignment auditing	Benchmark / dataset	Agent actions	CC-BY-4.0	64
InstrumentalEval	Deception, scheming & alignment auditing	Benchmark / dataset	Agent actions	unknown	11
MASK	Deception, scheming & alignment auditing	Benchmark / dataset	Hallucination	MIT	27
Model-Written Evaluations	Deception, scheming & alignment auditing	Benchmark / dataset	Model failure	CC-BY-4.0	438
Petri(Tool)	Deception, scheming & alignment auditing	Tool	Model failure	MIT	1378
SAD (Situational Awareness Dataset)	Deception, scheming & alignment auditing	Benchmark / dataset	Model failure	CC-BY-4.0	56
Shutdown Resistance	Deception, scheming & alignment auditing	Environment / sim	Agent actions	unknown	11
SycophancyEval	Deception, scheming & alignment auditing	Benchmark / dataset	Hallucination	unknown	136
Synthetic Document Finetuning(Tool)	Deception, scheming & alignment auditing	Tool	Data poisoning	unknown	65
Utility Engineering(Tool)	Deception, scheming & alignment auditing	Tool	Model failure	MIT	99
AI Verify(Tool)	Eval frameworks & harnesses	Tool	Regulatory/bias	Apache-2.0	103
AISI Evaluation Environment (aisev)(Tool)	Eval frameworks & harnesses	Tool	Regulatory/bias	Apache-2.0	81
COMPL-AI(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Regulatory/bias	Apache-2.0	213
DeepEval(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Hallucination	Apache-2.0	18892
Dioptra(Tool)	Eval frameworks & harnesses	Tool	Model failure	unknown	301
Giskard(Tool)	Eval frameworks & harnesses	Tool	Hallucination	Apache-2.0	5905
Hawk(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Unmapped	MIT	94
HELM (incl. HELM Safety)(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Regulatory/bias	Apache-2.0	2946
Inspect(Framework / harness)	Eval frameworks & harnesses	Framework / harness	All	MIT	3014
Inspect Cyber(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Breach/cyber	MIT	43
Inspect K8s Sandbox(Tool)	Eval frameworks & harnesses	Tool	Unmapped	MIT	42
inspect_evals(Framework / harness)	Eval frameworks & harnesses	Framework / harness	All	MIT	704^†
LightEval(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Unmapped	MIT	2561
LM Evaluation Harness(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Hallucination	MIT	14236
METR Task Standard(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Unmapped	MIT	198
ModelBench (AILuminate runner)(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Regulatory/bias	Apache-2.0	136
OpenAI Evals(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Unmapped	MIT	19609
Prime Environments (safety envs)	Eval frameworks & harnesses	Environment / sim	Agent actions	Apache-2.0	263
Project Moonshot(Tool)	Eval frameworks & harnesses	Tool	Regulatory/bias	Apache-2.0	358
safety-tooling(Tool)	Eval frameworks & harnesses	Tool	Unmapped	MIT	141
Triframe (Inspect)(Tool)	Eval frameworks & harnesses	Tool	Unmapped	unknown	4
verifiers(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Unmapped	MIT	4710
Vivaria(Framework / harness)	Eval frameworks & harnesses	Framework / harness	Unmapped	MIT	144
Aegis 2.0 / Nemotron Content Safety V2	Guardrails & safety classifiers (adjacent)	Benchmark / dataset	Regulatory/bias	unknown	—
Ai2 Safety Eval Suite(Framework / harness)	Guardrails & safety classifiers (adjacent)	Framework / harness	Regulatory/bias	Apache-2.0	111
CircleGuardBench	Guardrails & safety classifiers (adjacent)	Benchmark / dataset	Regulatory/bias	Apache-2.0	81
Circuit Breakers(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Regulatory/bias	MIT	270
gpt-oss-safeguard(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Regulatory/bias	Apache-2.0	61
Granite Guardian(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Hallucination	Apache-2.0	185
Guard Eval Harness(Framework / harness)	Guardrails & safety classifiers (adjacent)	Framework / harness	Regulatory/bias	MIT	19
GuardBench	Guardrails & safety classifiers (adjacent)	Benchmark / dataset	Regulatory/bias	EUPL-1.2	39
Guardrails AI(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Hallucination	Apache-2.0	7527
Invariant Guardrails(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Agent actions	Apache-2.0	472
LLM Guard(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Prompt injection	MIT	3216
NeMo Guardrails(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Prompt injection	MIT	7305
PIGuard / InjecGuard(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Prompt injection	MIT	88
Rebuff(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Prompt injection	Apache-2.0	1529
Superagent(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Prompt injection	MIT	6767
Vigil(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Prompt injection	Apache-2.0	506
WildGuard(Tool)	Guardrails & safety classifiers (adjacent)	Tool	Regulatory/bias	Apache-2.0	141
AbstentionBench	Hallucination & factuality	Benchmark / dataset	Hallucination	MIT	704^†
Confabulations Leaderboard	Hallucination & factuality	Benchmark / dataset	Hallucination	unknown	249
FActScore	Hallucination & factuality	Benchmark / dataset	Hallucination	MIT	460
HalluLens	Hallucination & factuality	Benchmark / dataset	Hallucination	unknown	86
HaluEval	Hallucination & factuality	Benchmark / dataset	Hallucination	MIT	603
Lynx(Tool)	Hallucination & factuality	Tool	Hallucination	unknown	47
Phare	Hallucination & factuality	Benchmark / dataset	Hallucination	unknown	13
RAGTruth	Hallucination & factuality	Benchmark / dataset	Hallucination	MIT	278
simple-evals (SimpleQA, HealthBench)	Hallucination & factuality	Benchmark / dataset	Hallucination	MIT	4666
TruthfulQA	Hallucination & factuality	Benchmark / dataset	Hallucination	Apache-2.0	952
Vectara Hallucination Leaderboard (HHEM)	Hallucination & factuality	Benchmark / dataset	Hallucination	Apache-2.0	3320
AILuminate (demo prompts)	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	Apache-2.0	84
AIR-Bench 2024	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	Apache-2.0	31
BeaverTails	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	Apache-2.0	185
CySecBench	Jailbreak & misuse	Benchmark / dataset	Breach/cyber	MIT	50
Do-Not-Answer	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	Apache-2.0	347
FORTRESS	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	704^†
GCG / AdvBench	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	4831
HarmBench	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	1071
HH-RLHF red-team data	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	1856
In-the-Wild Jailbreak Prompts	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	3846
ISC-Bench (Internal Safety Collapse)	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	Creative Commons	1371
jailbreak-evaluation(Tool)	Jailbreak & misuse	Tool	Regulatory/bias	Apache-2.0	29
JailbreakBench	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	687
JailbreakEval(Tool)	Jailbreak & misuse	Tool	Regulatory/bias	MIT	200
JailBreakV-28K	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	unknown	99
JailJudge	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	67
MakeMePay	Jailbreak & misuse	Benchmark / dataset	Agent actions	MIT	704^†
MakeMeSay	Jailbreak & misuse	Benchmark / dataset	Agent actions	MIT	704^†
Multi-Turn Human Jailbreaks (MHJ)	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	unknown	—
OR-Bench	Jailbreak & misuse	Benchmark / dataset	Model failure	Apache-2.0	30
PandaGuard(Framework / harness)	Jailbreak & misuse	Framework / harness	Regulatory/bias	MIT	69
RapidResponseBench	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	unknown	35
Redteaming Resistance Benchmark	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	unknown	50
SafetyBench	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	300
SALAD-Bench	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	Apache-2.0	175
SORRY-Bench	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	87
StrongREJECT	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	MIT	160
VLLM Safety Benchmark	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	unknown	91
WildTeaming / WildJailbreak	Jailbreak & misuse	Benchmark / dataset	Regulatory/bias	unknown	43
X-Teaming(Tool)	Jailbreak & misuse	Tool	Regulatory/bias	unknown	69
XSTest	Jailbreak & misuse	Benchmark / dataset	Model failure	CC-BY-4.0	149
awesome-ai-agent-incidents(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	MIT	57
awesome-ai-security(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	MIT	1552
Awesome-AI-Security(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	MIT	883
awesome-auditable-ai(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	CC0-1.0	152
awesome-cybersecurity-agentic-ai(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	MIT	603
awesome-evals(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	Creative Commons	984
Awesome-Jailbreak-on-LLMs(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	MIT	1691
Awesome-LLM-Safety(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	unknown	1913
awesome-llm-security(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	unknown	1712
Awesome-LLM4Cybersecurity(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	unknown	1787
Awesome-LLMSecOps(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	unknown	165
LLM-Agent-Benchmark-List(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	Apache-2.0	172
SafetyPrompts.com(List / catalog)	Meta-lists & catalogs	List / catalog	Unmapped	unknown	—
circuit-tracer(Tool)	Model-internals auditing	Tool	Model failure	MIT	2920
EleutherAI SAE(Tool)	Model-internals auditing	Tool	Model failure	MIT	742
Neuronpedia(Tool)	Model-internals auditing	Tool	Model failure	Apache-2.0	1169
nnsight(Tool)	Model-internals auditing	Tool	Model failure	MIT	1124
Observatory(Tool)	Model-internals auditing	Tool	Model failure	MIT	267
Persona Vectors(Tool)	Model-internals auditing	Tool	Model failure	Apache-2.0	466
Representation Engineering(Tool)	Model-internals auditing	Tool	Model failure	MIT	1050
TransformerLens(Tool)	Model-internals auditing	Tool	Model failure	MIT	3970
Agent Security Bench (ASB)	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	315
Agent Smith	Prompt injection & agent security	Benchmark / dataset	Correlated/systemic	MIT	130
Agent-Attack (multimodal)	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	146
AgentDojo	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	925
AgentPoison	Prompt injection & agent security	Benchmark / dataset	Data poisoning	MIT	248
AgentThreatBench	Prompt injection & agent security	Benchmark / dataset	Agent actions	MIT	704^†
b3 (Breaking Agent Backbones)	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	704^†
BIPIA	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	160
CodeIPI	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	704^†
InjecAgent	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	176
LLMail-Inject	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	27
MCP Security Bench (MSB)	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	51
Meta SecAlign(Tool)	Prompt injection & agent security	Tool	Prompt injection	unknown	71
Open-Prompt-Injection(Tool)	Prompt injection & agent security	Tool	Prompt injection	MIT	506
PINT	Prompt injection & agent security	Benchmark / dataset	Prompt injection	MIT	200
PoisonedRAG	Prompt injection & agent security	Benchmark / dataset	Data poisoning	MIT	304
RedTeamCUA	Prompt injection & agent security	Benchmark / dataset	Agent actions	Apache-2.0	65
TaskTracker(Tool)	Prompt injection & agent security	Tool	Prompt injection	MIT	97
WASP	Prompt injection & agent security	Benchmark / dataset	Prompt injection	unknown	97
0DIN AI Scanner(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	675
A.I.G (AI-Infra-Guard)(Tool)	Red-team & scanning tools	Tool	Breach/cyber	Apache-2.0	6882
A2A Scanner(Tool)	Red-team & scanning tools	Tool	Agent actions	Apache-2.0	166
Agent-Wiz(Tool)	Red-team & scanning tools	Tool	Agent actions	Apache-2.0	393
Agentic Radar(Tool)	Red-team & scanning tools	Tool	Agent actions	Apache-2.0	1060
Agentic Security(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	2021
ARES (AI Robustness Evaluation System)(Framework / harness)	Red-team & scanning tools	Framework / harness	Prompt injection	Apache-2.0	73
Arsenal (CALDERA plugin)(Tool)	Red-team & scanning tools	Tool	Breach/cyber	Apache-2.0	128
AutoDAN(Tool)	Red-team & scanning tools	Tool	Regulatory/bias	MIT	468
CALDERA(Framework / harness)	Red-team & scanning tools	Framework / harness	Breach/cyber	Apache-2.0	7437
DeepTeam(Framework / harness)	Red-team & scanning tools	Framework / harness	Prompt injection	Apache-2.0	3019
Dyana(Tool)	Red-team & scanning tools	Tool	Model/weight theft	MIT	368
EasyJailbreak(Framework / harness)	Red-team & scanning tools	Framework / harness	Regulatory/bias	GPL-3.0	923
FuzzyAI(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	1587
garak(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	9583
GPTFuzzer(Tool)	Red-team & scanning tools	Tool	Regulatory/bias	MIT	608
h4rm3l(Tool)	Red-team & scanning tools	Tool	Regulatory/bias	MIT	28
Jailbreak Cookbook (General Analysis)(Tool)	Red-team & scanning tools	Tool	Regulatory/bias	GPL	593
MCP Scanner(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	1091
MCP-Scan (Snyk Agent Scan)(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	3139
ModelScan(Tool)	Red-team & scanning tools	Tool	Model/weight theft	Apache-2.0	784
PAIR(Tool)	Red-team & scanning tools	Tool	Regulatory/bias	MIT	799
Prompt Fuzzer(Tool)	Red-team & scanning tools	Tool	Prompt injection	MIT	717
promptfoo(Tool)	Red-team & scanning tools	Tool	Prompt injection	MIT	26022
promptmap(Tool)	Red-team & scanning tools	Tool	Prompt injection	GPL-3.0	1281
PyRIT(Framework / harness)	Red-team & scanning tools	Framework / harness	Prompt injection	MIT	4630
Rigging(Framework / harness)	Red-team & scanning tools	Framework / harness	Unmapped	MIT	420
Skill Scanner(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	2604
SkillSpector(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	20300
spikee(Tool)	Red-team & scanning tools	Tool	Prompt injection	Apache-2.0	266
TAP (Tree of Attacks)(Tool)	Red-team & scanning tools	Tool	Regulatory/bias	MIT	250
AI Atlas Nexus (Risk Atlas)(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Regulatory/bias	Apache-2.0	142
AI Incident Database(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Correlated/systemic	Apache-2.0	273
AVID (AI Vulnerability Database)(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Breach/cyber	Apache-2.0	12
CoSAI Risk Map / tooling(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Breach/cyber	Apache-2.0	100
MIT AI Risk Repository(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Unmapped	unknown	—
MITRE ATLAS(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Breach/cyber	Apache-2.0	199
OWASP AI Testing Guide(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Unmapped	Creative Commons	822
OWASP AIVSS(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Breach/cyber	unknown	80
OWASP Top 10 for LLM Apps (2026)(Database / taxonomy)	Risk taxonomies, incident data & standards	Database / taxonomy	Prompt injection	unknown	128
A.S.E (AICGSecEval)	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	Apache-2.0	664
ARVO	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	BSD-2-Clause	119
BaxBench	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	MIT	105
CASTLE	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	MIT	8
CVEfixes	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	MIT	370
CWEval	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	Apache-2.0	49
DiverseVul	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	unknown	195
eyeballvul	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	MIT	72
LLMSecEval	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	unknown	61
LSCBench (Last Secure Code Benchmark)	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	Apache-2.0	30
OSS-Fuzz-Gen(Framework / harness)	Secure code & vuln detection	Framework / harness	Breach/cyber	Apache-2.0	1443
PrimeVul	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	MIT	279
RedCode	Secure code & vuln detection	Benchmark / dataset	Agent actions	MIT	91
SecCodePLT	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	unknown	18
SecLLMHolmes	Secure code & vuln detection	Benchmark / dataset	Breach/cyber	GPL-3.0	68
Vulnhuntr(Tool)	Secure code & vuln detection	Tool	Breach/cyber	AGPL-3.0	2800
AI Red Teaming Playground Labs	Vulnerable labs & CTF targets	Environment / sim	Prompt injection	MIT	2095
Damn Vulnerable LLM Agent	Vulnerable labs & CTF targets	Environment / sim	Prompt injection	Apache-2.0	530
Damn Vulnerable MCP Server	Vulnerable labs & CTF targets	Environment / sim	Prompt injection	unknown	1355
Evil MCP Server	Vulnerable labs & CTF targets	Environment / sim	Prompt injection	MIT	34
LLM Agent Honeypot	Vulnerable labs & CTF targets	Environment / sim	Breach/cyber	unknown	65
OWASP FinBot CTF	Vulnerable labs & CTF targets	Environment / sim	Agent actions	Apache-2.0	105
SaTML LLM CTF	Vulnerable labs & CTF targets	Environment / sim	Prompt injection	MIT	30
Vulnerable MCP Servers Lab	Vulnerable labs & CTF targets	Environment / sim	Prompt injection	MIT	279

† Stars reflect a shared parent repository (e.g. several evals bundled inside one harness like inspect_evals). Identical counts across those rows are expected, not a data error.

Refreshed weekly from our internal catalog. Unknowns stay unknown — nothing here is estimated or interpolated.

Wanich Risk Lab, 2026.