Pattern Matrix/White Paper/F2

Problem

An agent may complete the same kind of task successfully and still start from zero on the next run: it reads the same references, repeats the same trial and error, and rediscovers the same path. Tokens and time are spent on work the system has already learned, because nothing callable was retained after the task finished.

A Skill Package records a verified procedure as a structured asset, usually YAML frontmatter (name, description, triggers), a Markdown body (steps, gotchas, examples), and bundled scripts. When a similar task arrives, the agent routes to the corresponding skill and loads the established path. Generator-Critic revises one output; a Skill Package retains a capability across tasks. The ACT-R cognitive architecture calls this proceduralization: compiling declarative knowledge into procedural skill. Contemporary agent systems often implement the idea with a SKILL.md-style package.

Classification: Reflection × Route

• Vertical axis · Reflection: After repeated successful runs, the system extracts a procedure and prepares it for reuse across tasks.
• Horizontal axis · Route: Runtime matching selects a skill by task features and triggers. Discovery exposes metadata, Activation loads SKILL.md, and Execution loads bundled scripts on demand.

Solution and mechanics

A Skill Package system consists of two pipelines:

1. Loading pipeline: When a task arrives, the agent routes from the skill library to the right skill. Three-stage loading keeps context use under control: startup exposes only a name and short description, activation loads the full SKILL.md after a match, and execution loads bundled scripts only when needed. The exact context cost depends on the catalogue and model tokenizer and should be measured locally.
2. Release pipeline: A skill may be written by a person or distilled with agent assistance. Both paths move through candidate → safety review → isolated evaluation → coexistence evaluation → canary → release → monitoring → refine, merge, or retire.

Useful operating discipline includes putting critical constraints where they are hard to miss, preferring deterministic scripts when the work can be encoded, adding worked examples, testing each skill, and reviewing usage evidence so stale or low-value packages do not accumulate.

Isolated evaluation asks whether a skill can complete its own task. Coexistence evaluation asks whether it still behaves correctly when other skills are available. A skill may pass alone and then steal triggers, lose routing decisions, or interfere with a neighbouring skill. When an outcome degrades, the trace must distinguish the main agent, skill content, trigger description, and host harness.

Applicability

• Recurring tasks with a relatively stable flow: operations runbooks (batch cluster restarts, configuration changes), customer-service triage SOPs, standard sales processes—tasks of the same kind that recur frequently with flows that do not change daily.
• Enterprise processes that need to be observable and reusable: structuring the tacit flow in a veteran employee's head into a SKILL.md so that new employees and the agent read and use the same thing, and a sales leader can review and version it.
• Critical processes where mistakes are costly and worth solidifying: financial approval flows, incident-response steps, and similar processes where a mistake is expensive—solidifying them into a skill is steadier than relying on the agent's improvisation each time.

Known failure modes

• Forcing a skill onto a task that should not be solidified: tasks that differ every time (open-domain research), run only once, still iterate rapidly, or are judgment-intensive—these four categories carry more retention risk than reuse benefit. A Skill Package pays off when "recurring + stable flow + costly mistakes" hold together.
• Skill library pollution: low-quality self-distilled skills enter the production library without curation and mislead later runs. A growing catalogue can therefore reduce, rather than improve, task success. The defense is evidence-based distillation, a probation period, replay tests, and regular curation.
• Stale skills: the infrastructure changed but the skill did not follow, so an agent executing a stale skill makes mistakes instead, especially common in operations scenarios. The defense is success-rate monitoring plus automatic alerts plus version binding (mark tested_with in SKILL.md).
• Description mismatch: a description written too broad or too narrow makes the agent pick the wrong skill on recall, or fail to use one it should have. The description must contain concrete scenarios, and the triggers list must be fine-grained enough.
• Passes in isolation, regresses in combination: A new skill works alone but conflicts with existing triggers in the production catalogue. Run conflict cases and mixed-skill tasks before release, and retain a known rollback version.
• Weak attribution: A failed task is labelled as a “skill failure” without saving route candidates, selected content, model version, and harness version. The team then cannot identify which layer needs repair.

Verification and metrics

• Skill hit rate: the proportion of eligible tasks that recall the correct skill. Calibrate descriptions and triggers against a labelled task set.
• Skill success rate: the proportion of tasks that succeed after calling a given skill, compared with the general-flow baseline. A sustained decline should trigger review or rollback.
• Library health: track whether task quality, precision of activation, and maintenance burden change as the catalogue grows.
• Loading token share: measure catalogue, activation, and execution context separately. Set a local budget that leaves enough room for the task itself.
• Coexistence regression rate: Compare false activation, missed activation, task success, and context cost before and after a skill joins the active catalogue.
• Release and rollback evidence: Record the evaluations a candidate passed, its canary period, approval, and whether rollback returns the system to a known state.

Reference implementation

# Stage 1 Discovery: load only name + description at startup
            catalog = [{"name": s.name, "desc": s.description} for s in library]

            # Stage 2 Activation: load the full SKILL.md after a task matches
            matched = top_k(task, library, k=activation_k)   # tune on labelled tasks

            # Stage 3 Execution: load bundled scripts on demand, track success rate
            result = run(matched_skill, task)
            mark_used(matched_skill, success=result.ok)

            # Retention: Hermes-style automatic distillation (multiple filters)
            if distillation_policy.accepts(task, trace, outcome):
                skill = distill(task, tool_calls)   # enters probation, not direct production

            # Release: evaluate alone and with the active catalogue
            ISOLATED_EVAL(skill, held_out_tasks)
            COEXISTENCE_EVAL(skill, active_library, conflict_cases)
            PROMOTE(replay_passed and coexistence_passed and approval_granted)

            # Runtime attribution
            record_route(candidates, selected_skill, skill_version, harness_version, outcome)

            # Lifecycle
            REFINE(sufficient_usage_evidence and quality_declining)
            ROLLBACK(blocking_regression)
            EVICT(no_effective_use and review_approved)
            

Use staged loading to control context. A self-distilled skill stays in probation until replay and review pass. Isolated and coexistence evaluations are both release gates. Every activation records routing and version evidence; a blocking regression rolls back to the previous stable version.

Illustrative scenario

Consider a B2B sales team whose strongest salesperson follows a repeatable but undocumented process: researching the account before contact, using a consistent discovery structure, selecting objection-handling material, and sending a decision checklist before signature. The team can package this process as a human-authored SKILL.md, with scripts for deterministic checks and examples for judgment-heavy steps. Agent-distilled variations stay in a lower-trust namespace, customer-facing changes require review, and promotion depends on replay against historical opportunities. The example demonstrates lifecycle and trust controls. Any claim about improved conversion or onboarding speed would require the company's own records and publication approval.

Related patterns

• Procedural Memory (memory module M5): nearly isomorphic in deployment—both are skill libraries, both store procedural knowledge in SKILL.md form, and the difference is design intent. A Skill Package emphasizes "package after reflection"—the agent proactively distills and solidifies the successful path after repeated success (post-reflection); M5 emphasizes "store what is learned"—writing procedural knowledge in as memory. The former is the reflection view, the latter the memory view, with a high overlap in engineering substrate.
• Generator-Critic (F1): a sequential link. Generator-Critic is reflection within a single task, while a Skill Package is reflection across tasks, solidifying "the thing done right."
• Experience Replay (F3): a paired sibling pattern. A Skill Package holds verified callable units (done many times, all successful, packaged into a skill to call directly), while Experience Replay holds broader reference assets (useful but not necessarily verified). The agent calls a skill first and falls back to experience retrieval when nothing matches.
• RAG (memory module): a complementary ensemble. RAG handles "what is known" (declarative, fact retrieval), and a Skill Package handles "how to do the work" (procedural, flow reuse); in enterprise agent deployments the two are used together.

Engineering judgment

A Skill Package turns a verified procedure, its tools, and its boundary conditions into a versioned runtime asset. Its value depends on routing quality and lifecycle discipline as much as on the instructions inside the package.

Further reading

• Reflection module: Make feedback change the system
• First Reflection workshop, 12 August 2026
• Claude Enterprise Skills: testing and lifecycle guidance