Building and testing software with LLMs — evals, AI testing, and CI for AI apps

CI for AI agents: testing non-deterministic systems

Agents are the hardest software to test. A practical blueprint for CI for AI agents: trajectory assertions, tool mocking, eval harnesses, and what 'green' actually attests to.

Keeping AI-generated code from breaking your test suite

AI coding assistants raised your PR throughput — and quietly raised your flaky-test rate with it. Here's how to keep your CI signal trustworthy at the new volume.

Testing RAG pipelines: what actually breaks

RAG systems break in two very different places — retrieval and generation — and treating them as one testable unit is how you end up debugging blindly in production.

LLM-as-judge: how to use it without fooling yourself

LLM-as-judge can replace hours of human annotation — or silently validate garbage. Here's how to tell which one you're getting.

Setting up CI for prompt and eval regression testing

A prompt change that tanks quality is the silent killer of LLM features. Here's how to run evals in CI so every PR gets a score — and failing scores block the merge.

Flaky evals: when your LLM tests are non-deterministic

Your LLM eval passes on Monday, fails on Wednesday, and you changed nothing. That's not a model problem — that's a flaky eval. Here's how to fix it.

How to test LLM-powered applications

Traditional assertions fall apart when your function returns something different every time. Here's how to build a test suite that actually catches regressions in LLM apps.

How to test LLM-powered applications

Testing LLM-powered applications is genuinely hard — but most teams are making it harder than it needs to be. Here's what actually works.