Promptfoo

Promptfoo runs automated test suites across prompt variations and catches regressions when a prompt or model changes, with custom scoring functions that fit into a CI/CD pipeline. Developers use it to test prompts the same rigorous way they'd test any other piece of software, rather than eyeballing outputs in a chat window.