Tag: Skill Testing

This section is very important. provious we focused on how to write a Skill. we need answer a very simple question: is the Skill you wrote actually good?

Now, we focuse on how to prove whether the Skill works well and how to iteratively improve it. This is where we start moving from Prompt/Skill authoring into AI Engineering quality engineering.

For example, the SOP (Standard Operating Procedure) says:

Data Quality Report Skill

1. Check row count.
2. Check NULL values.
3. Check duplicate records.
4. Check data quality rules.
5. Generate a report.

How do you know this SOP is actually good? You should test it.

A Test Case is basically an “exam question.” we have customer_id
1001
1002
1003
1003 <– duplicated
1004
NULL <– null

Now let agent use the Skill to Test.

Actual Result:
Row Count = 6
NULL = YES
Duplicate = YES

so result PASS.

if Agent return
Row Count = 6
NULL = NO
Duplicate = YES

so result FAIL

We should perpare multiple test case, because a Skill can encounter many different situations.

Key Takeaways

要点	EN	CN
Skill Testing	Test whether a Skill behaves correctly	测试 Skill 是否按照要求正确工作
Behavioral Testing	Validate what the Skill actually does	验证 Skill 实际执行了什么
Quality Evaluation	Evaluate output quality, not just existence	评价输出质量，而不仅仅是有没有输出
Golden Test Cases	Use stable cases to validate behavior	使用稳定的标准案例验证行为
Regression Testing	Make sure old behavior does not break	确保修改后原来的功能没有坏
Root Cause Analysis	Identify which layer caused the failure	判断失败到底是哪一层造成的
Iteration	Test → diagnose → modify → retest	测试 → 定位 → 修改 → 再测试
AI Engineering	Treat Skill quality as an engineering problem	把 Skill 质量当成工程问题处理
E07 vs G02	E07 tests a Skill; G02 systematizes Golden Dataset evaluation	E07 测试具体 Skill；G02 系统化构建 Golden Dataset