Post

Fine-tuning a model to fix a specific agent error often breaks three other things that were working.

This is called catastrophic forgetting, where updating internal parameters erases previous knowledge. SkillOpt solves this by treating a markdown file as the model's trainable state. It optimizes a text document instead of the model's internal weights.

What it does: It treats a SKILL.md file as the "weights" and an optimizer LLM as the "gradient" to find the best instructions. Why it's interesting: It uses bounded edits, which are restricted add, delete, or replace operations that prevent wild semantic jumps. The mechanism: A validation gate tests every edit against a held-out set of tasks to ensure no regressions occur before the change is committed. The advantage: Since the model remains frozen, there is zero additional inference overhead or cost per call. The result: You get a portable, version-controlled text artifact that works across different models like GPT-4 or Claude.

The evolution cycle follows a strict loop. First, the agent runs a task and the system reflects on the failure. The optimizer proposes a surgical edit to the markdown file. This edit must pass the validation gate on a separate dataset. If it passes, the SKILL.md is updated; if it fails, the edit is rejected.

This approach replaces the "hope" of prompt engineering with a systematic search for the best instructions. It turns prompt optimization into a repeatable engineering process rather than a guessing game.

Download the full Field Guide PDF to see the SkillOpt feedback loop in detail.

#LLMOps #AgenticWorkflows #PromptEngineering #ModelAgnostic