Continuous Alignment Testing (CAT) Framework

A framework for implementing Continuous Alignment Testing (CAT) for LLM-based applications.

Overview

CAT provides the infrastructure to:

• Run and track automated tests against an AI application.
• Store and analyze AI test results over iterations.
• Monitor reliability changes in an AI application as prompts, models, code, and data evolve.
• Integrate AI tests into the CI pipeline.
• Build the AI application as an API service.
• Ensure AI applications are reliable while maintaining creativity.
• Iterate on prompts and code while measuring improvements.

Example Apps

• team_recommender - Team Recommender
• you next idea?

Documentation

Github Pages

Pages Source in docs

Wiki

Wiki Pages