Garak

Garak is a command-line tool that probes language models for weaknesses, much like a network scanner does for servers. It fires hundreds of attacks, including jailbreaks, prompt injection, data leakage and toxicity, and reports which ones succeed.

Security researchers use it for red teaming and for comparing models. It works with hosted APIs and local models.

It does not fix problems for you, but it shows where to look.