This section is very important. provious we focused on how to write a Skill. we need answer a very simple question: is the Skill you wrote actually good?
Now, we focuse on how to prove whether the Skill works well and how to iteratively improve it. This is where we start moving from Prompt/Skill authoring into AI Engineering quality engineering.
For example, the SOP (Standard Operating Procedure) says:
Data Quality Report Skill
- Check row count.
- Check NULL values.
- Check duplicate records.
- Check data quality rules.
- Generate a report.
How do you know this SOP is actually good? You should test it.
A Test Case is basically an “exam question.” we have customer_id
1001
1002
1003
1003 <– duplicated
1004
NULL <– null
Now let agent use the Skill to Test.
Actual Result:
Row Count = 6
NULL = YES
Duplicate = YES
so result PASS.
if Agent return
Row Count = 6
NULL = NO
Duplicate = YES
so result FAIL
We should perpare multiple test case, because a Skill can encounter many different situations.
Key Takeaways
| 要点 | EN | CN |
|---|---|---|
| Skill Testing | Test whether a Skill behaves correctly | 测试 Skill 是否按照要求正确工作 |
| Behavioral Testing | Validate what the Skill actually does | 验证 Skill 实际执行了什么 |
| Quality Evaluation | Evaluate output quality, not just existence | 评价输出质量，而不仅仅是有没有输出 |
| Golden Test Cases | Use stable cases to validate behavior | 使用稳定的标准案例验证行为 |
| Regression Testing | Make sure old behavior does not break | 确保修改后原来的功能没有坏 |
| Root Cause Analysis | Identify which layer caused the failure | 判断失败到底是哪一层造成的 |
| Iteration | Test → diagnose → modify → retest | 测试 → 定位 → 修改 → 再测试 |
| AI Engineering | Treat Skill quality as an engineering problem | 把 Skill 质量当成工程问题处理 |
| E07 vs G02 | E07 tests a Skill; G02 systematizes Golden Dataset evaluation | E07 测试具体 Skill；G02 系统化构建 Golden Dataset |