Visual
Contract Tests
Snapshot atau deterministic hash untuk tools/list. Review perubahan pada tool name, description, schema, annotations, scope, dan output schema.
Test unique names, deterministic order, no collision, bounded schema, no identity fields, list-time + call-time scope parity, dan result fixtures.
Golden Prompt Matrix
| Type | Prompt | Expected |
|---|---|---|
| Direct | Tampilkan semua project aktif. | projects_list |
| Indirect | Sekarang kita lagi ngerjain apa saja? | projects_list |
| Follow-up | Buka project kedua tadi. | pakai ID hasil sebelumnya |
| Negative | Apa itu project management? | no tool call |
Behavioral Evals
Buat scenario end-to-end nyata dan multi-step. Verifikasi outcome secara independen dan simpan model/version/catalog digest agar hasil eval bisa dibandingkan dari waktu ke waktu.
Attack / Refusal Matrix
- auth bypass dan tenant impersonation
- read → write escalation
- expired/revoked token
- wrong issuer/audience
- OAuth code replay / redirect manipulation / PKCE downgrade
- SSRF, path traversal, command injection
- tool collision
- approval replay / changed args after approval
- rate-limit dan secret leakage
Local Verification Gate
Run all applicable gates before deploy:
- format / lint
- typecheck
- unit tests
- MCP contract tests
- list-time + call-time scope tests
- auth / OAuth tests
- approval / policy tests
- file security tests
- golden prompt tests
- behavioral evals
- plugin/package checks
- dependency audit
- secret scan
- production build
- MCP Inspector / official client wire testAmbil Test Pattern
Buat MCP kamu sendiri
Gunakan MCP Builder Kit sebagai starter/reference production-grade.