Task and baseline
Define observable success and compare against the current process or a simpler method. Include typical, rare, ambiguous and adversarial cases.
学完后你可以
- Build an evaluation set tied to real users and failure costs.
- Measure quality separately from safety, latency and cost.
- Plan limited rollout, monitoring, fallback and rollback.
可信评估的五个部分
Quality and calibration
Measure task utility, factual support, citation validity, consistency and appropriate abstention. A single average can hide severe failures.
Security and privacy
Test direct and indirect prompt injection, malicious retrieved content, tool abuse, data disclosure and privilege boundaries. Treat all model output as untrusted input.
Operational measures
Track end-to-end latency, tail latency, token and tool cost, availability and recovery. Version models, prompts, tools, policies, data and evals together.
Rollout and governance
Start with limited permissions and users, observe real outcomes, keep a deterministic or human fallback, and define circuit breakers and rollback before promotion.
The evidence loop
- 01
Define risk
- 02
Build representative set
- 03
Run baseline and candidate
- 04
Review failures
- 05
Gate and monitor
Separate construction from evaluation when possible. For variable outputs, repeat tests and examine distributions and worst cases—not only the best example.
常见混淆
- An LLM judge can scale review, but it must be calibrated against blinded human judgments for the target task.
- Safety refusal rate alone is incomplete: measure both harmful compliance and excessive refusal of legitimate requests.
- Offline success does not prove production success when users, data, tools or model versions differ.
Practice: create a 12-case eval
Choose one LLM feature and build a small predeployment gate.
- Include normal, edge, ambiguous, multilingual, injection and tool-failure cases.
- Score factual support, task completion, appropriate refusal, latency and cost separately.
- Set a blocking safety threshold plus a rollback trigger before seeing candidate results.
Quick check
Why compare with a baseline?
To learn whether added model complexity materially improves the current or simpler process under the same criteria.
Why test prompt injection if the model has no database write access?
Injection can still disclose data, corrupt answers or abuse any available read tools; least privilege limits but does not erase risk.
What must be versioned for reproducible evaluation?
At minimum the model, parameters, prompts, tool contracts, policies, evaluation data and scoring method.
关键词
- evaluation
- baseline
- benchmark
- calibration
- abstention
- prompt injection
- least privilege
- latency
- observability
- rollback
评估框架与指南
HELMHolistic framework for transparent multi-metric language-model evaluation.NIST Generative AI ProfileCross-sector risk-management profile for generative AI.OWASP Top 10 for LLM applicationsCommunity risk catalogue for common LLM application vulnerabilities.OpenAI evaluation best practicesCurrent guidance on eval design, criteria, datasets and continuous evaluation.