🌐 Language: English Version | 中文版
When using Agents on engineering tasks, some requirements need to be explained repeatedly even after they have been written into instructions: check the existing implementation before writing a plan, state what a test actually executed, and rerun the original failure scenario after a fix. Clarifying a requirement within one conversation lets that task continue. In a new project or session, the same gap may appear again. The question is what the instructions are missing and how to improve them from actual execution.
I started collecting Agent session logs to answer that question. At the time, I also found that the local logs from the Agent tools I used were retained for only about three months; older records would no longer be available. I backed them up so I could keep reviewing and analyzing them. The initial goal was to revise the instructions and make them work better in real tasks. Only later, as reviews accumulated, did some stable task practices grow into skills.
Summary: I backed up Agent session logs with limited local retention to find where instructions repeatedly failed in practice and revise them. As the reviews accumulated, stable task practices became skills; after putting them to use, I continued adjusting them based on hands-on experience and the execution recorded in the logs.
Why Back Up the Logs
To improve instructions, first preserve the records that show where they fail.
At the end of a task, the final code, plan, and summary are usually available. What may be missing is the decision that changed along the way. For example, a plan initially classified a query capability as new work, then changed it to reuse after checking the existing implementation. The final plan shows only “reuse.” The conversation also records why the capability was first considered new and where the correction happened.
These records are useful for review, but an Agent’s explanation is not evidence by itself. The archive needs to preserve message order, source session, project context, and a path back to the original record. For tool use, the call and its response must be checked to distinguish “about to run,” “ran,” and “succeeded.” Logs capture observable conversations and tool events; they do not recover the model’s complete internal reasoning.
Records from different tools can be converted into a common format for reading and search, while retaining a path to the originals. The readable view helps locate clues. Whether a command ran or a piece of code existed still has to be checked against original events and actual code. A common format does not guarantee that every source has extracted every tool event.
flowchart LR
A["Local session logs have a retention limit"] --> B["Back up and search them"]
B --> C["Review corrections and actual results"]
C --> D["Revise shared instructions"]
D --> E["Extract stable task practices as skills"]
Backing up logs, then, is more than preserving history. It keeps records available after the local copies expire and provides material for asking why an instruction did not work and how to change it for the next task.
Revising Instructions from Corrections
Instructions need checkable decision criteria, not just a desired attitude.
A review does not need to read every line of every conversation equally. The useful points are often the corrections: why a plan was rewritten, why a test could not be counted as passed, or why an operation changed the conditions under test. The next step is to determine whether the issue came from one-off business information, a particular environment, or a collaboration gap that recurs across tasks.
Using the query example, the table below is an anonymized summary of the decision, not a verbatim session excerpt:
| Initial judgment | What to check | Reusable instruction |
|---|---|---|
| Add a new query capability | Whether the existing entry point, callers, response contract, and filters already meet the requirement | Record current behavior and code evidence in the implementation plan before deciding to reuse, modify, or add; explain why existing behavior is insufficient when adding something new |
“Read the code carefully” is difficult to verify. If the plan names the actual code location, the required difference, caller impact, and corresponding tests, the next person can check the decision. In this example, the test expectation must change with the plan: verify that the reused entry point meets the new requirement and run regression checks for affected callers, rather than test an interface that was never added.
A correction to test reporting has the same structure. “A test file exists,” “a test process started,” and “the business behavior passed” are three different facts. They cannot all be reported as “testing completed.” The resulting instruction asks for each case’s actual result, version, environment, and observations, with unrun and blocked cases marked separately. For page flows and business data, it also calls for checking the actions, requests, persistence, and related results.
These findings first revise instructions; they do not automatically justify a new skill. A business field’s meaning belongs in its project. “Compare a plan with the current implementation” and “distinguish test execution states” may belong in shared instructions. The historical review used keyword searches, sample reading, and follow-up on specific sessions. It did not cover every archived log and cannot establish a model failure rate or quantify the benefit of the rules.
When Instructions Become Skills
Shared boundaries and task procedures have different scopes; putting them in one entry point adds reading and maintenance cost.
At first, the constraints found in reviews went into AGENTS.md. As the instructions became more detailed, general requirements such as protecting existing changes, respecting authorization, and reporting verification honestly became mixed with procedures for investigating failures, recording bugs, and running business acceptance checks. The former apply to many tasks. The latter have their own inputs, steps, evidence, and completion conditions. That is when extracting task procedures into skills began to make sense.
| Location | What belongs there |
|---|---|
| Project rules | Business semantics, technical constraints, and environment assumptions |
Shared AGENTS.md |
Authorization boundaries, protection of existing changes, and evidence and delivery requirements |
| Task skill | Inputs, sequence of actions, handoff information, and completion conditions for a type of task |
For example, “testing completed” being unclear leads to a shared rule to distinguish executed, unrun, and blocked cases. A business acceptance skill then defines how to check pages, APIs, data, and related totals against confirmed cases. “No one reran the failure after a fix” leads to a rule to preserve the original failure scenario. The corresponding delivery workflow defines the sequence of developer fix, Review, independent retest, and issue closure.
A skill is therefore neither a saved conversation template nor a new entry point for every small problem. It organizes a stable way of handling a class of tasks after the instructions have matured: when to enter, what inputs are needed, what evidence to return, and when to finish or return a decision to a person. Where an existing tool or workflow already covers the job, it should be reused or merged to avoid competing expectations.
An Example That Grew into a Skill: Multi-Agent Handoffs
Handoffs often fail because the next Agent receives inconsistent grounds for judgment, not because there are too few Agents.
In reviewed sessions, a developer might receive a revised design while a tester still uses old cases. Or the code might change during testing while the report says only “passed.” Either situation makes the result hard to judge. The resulting instruction is more specific than “communicate more”: keep the detailed design and the single authoritative test-case file in the same confirmed version, and hand off the code baseline, current diff, environment assumptions, and scope for the round.
In the query example, development follows the revised decision to reuse the existing entry point. Review checks for duplicate implementation and effects on callers. Testing uses the same version of the cases to verify query results and related regressions. A clear deviation from a confirmed expectation can go through fix, review, and retest. If the business meaning of a filter is still undecided, the work must return to a person; the design and cases are updated before it continues.
flowchart TD
A["Person confirms design, cases, and implementation authorization"] --> B["Development and self-check"]
B --> C["Lead Agent Review"]
C --> D{"Matches confirmed expectations?"}
D -- "Clear deviation" --> B
D -- "Yes" --> E["Independent testing"]
E --> F{"Test result?"}
F -- "Clear defect" --> B
F -- "Passed" --> G["Check evidence and hand over for human acceptance"]
D -- "Business expectation undecided" --> H["Human decision; update the contract"]
F -- "Business expectation undecided" --> H
H --> B
The lead Agent coordinates handoffs, performs Review, and checks evidence. The “handoff contract” consists of the design and cases in the same version, along with authorization and baseline assumptions. The Agent can advance a cycle within that confirmed scope. It cannot decide a new business rule on its own or treat implementation authorization as permission to commit, push, or deploy. This process costs more version and evidence tracking, but it gives each fix round the same basis for judgment.
What Instructions Can and Cannot Enforce
A skill expresses rules; it does not replace tool permissions or real verification.
Instructions can tell an Agent to check authorization before acting and stop when shared state conflicts. Actual permission limits, resource locks, and process isolation have to come from tools and the environment. Format and link checks can find structural problems in the skill documentation. Whether the behavior follows the rules must be observed in specific tasks: did a read-only task stay read-only, were unrun tests reported accurately, and was an existing confirmation reused correctly?
The review continues after a skill starts running. After using it for a while, there is usually an intuitive sense of where it feels awkward or repeatedly needs human correction. The backed-up logs let that impression be checked against specific tasks: did the skill enter the intended workflow, how far did execution get, what artifacts and verification results were produced, and at which decision did it stop or drift? Comparing those records with actual code and results helps distinguish unclear instructions, a missing skill step, a task prerequisite, and a one-off business case.
Those observations feed the next revision. Missing steps call for clearer inputs or completion conditions. A rule that only applies to one project moves back into that project. Overlapping workflows are merged. Logs from later runs then show whether the same problem recurs. The logs that first helped improve instructions remain a feedback source after a skill is in use.
Scope and Cost
Recurring collaboration gaps across projects can first inform shared instructions. A task type is worth turning into a skill only when its inputs, steps, and completion conditions have become stable. A one-off small fix or a single article edit may need only its key evidence and actual verification, without the full multi-Agent handoff.
This approach has a cost: logs must be backed up and reviewed, instructions corrected against actual deviations, and skills kept free of duplicate entry points. My choice is to use logs to improve instructions first, then extract a skill once the practice for a task type has proved stable.