来源：https://academy.claude.com/courses/ai-native-sdlc-playbook/continuous-evals-in-ci 读这篇之前：05 CLAUDE.md + 08 反馈环 不确定：ZH 是 Clint 译官方英文课体，不是 Anthropic 中文。
本课词汇
| English | 中文 | 怎么记 |
|---|---|---|
| eval | 评测任务 | 一条 prompt + 何谓可接受的检查 |
| stage-gate QA | 阶段门质量关 | 课把 evals 称为它的 AI-native 对应物 |
| pass rate | 通过率 | 改配置时用来挡回归 |
| non-interactively | 非交互 | CI 里跑，没人坐在旁边 |
对照正文
EN Evals are the AI-native equivalent of stage-gate QA. In practice that means a suite that runs whenever the agent’s configuration changes. When a new model is swapped in or a prompt is rewritten, the eval suite says whether the agent still does the work to the same standard.
ZH Evals 是阶段门 QA 的 AI-native 对应物。实践上就是：agent 配置一变就跑的一套。换了新模型或重写了 prompt，eval 套件说 agent 是否还按同一标准干活。
EN The evals should be seen as a live suite. As models improve, cases that once discriminated stop doing so, and new ones must be added that arise from ongoing monitoring.
ZH Evals 应看成活的套件。模型变好，曾经能区分好坏的用例不再能区分，就必须从持续监控里补新的。
EN Depending on the use case, some teams may prefer to run these evals offline on a set cadence rather than on every change. The steps below are for continuous evaluations.
ZH 看用途，有的团队更愿意按固定节奏离线跑，而不是每次改动都跑。下面步骤针对持续评测。
怎么起步 / Getting started
EN - Prerequisites: The CLAUDE.md and feedback loop (Stage 4: Test).
- Infrastructure: CI that can run Claude Code non-interactively, and an API key with budget for eval runs.
ZH 先决条件：CLAUDE.md 和反馈环（Stage 4: Test）。
- 基础设施：能非交互跑 Claude Code 的 CI，以及带 eval 预算的 API key。
怎么做 / How to execute it
EN 1. The platform engineer collects 20 to 50 real tasks from recent work, each with its expected or accepted outcome.
2. Write each task as an eval, meaning the prompt plus the checks that define acceptable (tests pass, lint clean, behavior unchanged, policy followed).
3. The suite runs non-interactively in CI on a schedule and on any change to CLAUDE.md, skills, or hooks, since that configuration steers the agent and deserves the regression testing that code gets.
4. Gate configuration changes on the results. A skill change that drops the pass rate gets reviewed before it merges.
5. Each production incident gets an eval, written by the team that owned the incident, and stays in the suite as a regression test.
ZH 1. 平台工程师从近期工作收 20 到 50 个真任务，每个带预期或已接受的结果。
2. 每个任务写成一条 eval：prompt + 何谓可接受的检查（测试过、lint 干净、行为没变、政策遵守）。
3. 套件在 CI 里非交互跑：定时，以及任何对 CLAUDE.md、skills、hooks 的改动。这些配置在给 agent 指路，配得上代码那种回归测试。
4. 用结果卡住配置改动。让通过率掉下去的 skill 改动，合并前要审。
5. 每起生产事故变成一条 eval，由负责那起事故的团队写，留在套件里当回归。
长什么样 / What it looks like
EN .github/workflows/agent-evals.yml:
ZH .github/workflows/agent-evals.yml：
name: Agent evals
on:
pull_request:
paths: ['CLAUDE.md', '.claude/**']
schedule:
- cron: '0 2 * * *'
jobs:
evals:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm install -g @anthropic-ai/claude-code
- name: Run eval suite
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
for eval in evals/*.json; do
claude -p "$(jq -r '.prompt' $eval)" \
--allowedTools "Read,Edit,Bash(make test)" \
--output-format json > result.json
./evals/check.sh "$eval" result.json
doneZH 官方示例，保持英文。
治理 / Governance considerations
EN Evals give QA a gate that keeps up with agent output. The pass-rate threshold is enforced as a merge check, runs are logged so results can be compared over time, and the team that owns the configuration change approves it.
ZH Evals 给 QA 一扇跟得上 agent 产出的闸。通过率阈值当成合并检查来强制；跑次记下来，结果可跨时间比；配置改动由拥有它的团队批准。
怎么衡量 / How to measure it
EN - Leading indicator: The eval pass rate over time, reported by the suite on every run, and how long a production incident takes to become a permanent eval.
- Lagging indicator: Regressions caught in CI compared with regressions found in production derived from the incident tracker.
ZH 领先指标：eval 通过率随时间的变化，每次跑套件都会报；以及生产事故变成永久 eval 要多久。
- 滞后指标：CI 抓住的回归，对照事故追踪里生产才发现的回归。
读完能记住的三句话
EN Evals regression-test the agent’s configuration: CLAUDE.md, skills, hooks, and model swaps.
ZH Evals 给 agent 的配置做回归：CLAUDE.md、skills、hooks、换模型。
EN Use 20–50 real tasks. A skill change that drops the pass rate should not merge blindly. ZH 用 20 到 50 个真任务。让通过率掉下去的 skill 改动，不要盲目合并。
EN Every production incident becomes an eval and stays in the suite. ZH 每起生产事故变成一条 eval，留在套件里。