Skill-Synthesizer: Teaching AI Agents to Learn from Experience

Scientific discovery is rarely about solving a problem from scratch. Researchers constantly reuse ideas, adapt previous solutions, and combine concepts they learned in their previous experience. If we want AI agents to become trusted collaborators in science and engineering, they need to do the same.

Our recent paper, Skill-Synthesizer: Query-Aware Skill Synthesis for Scientific Reasoning, focuses on this, as it introduces a new approach to long-term memory for AI agents. Instead of simply retrieving previous experiences, our system learns how to compose them into new, query-specific knowledge. With this ability, agents become capable of synthesizing accumulated experience into novel solutions, instead of simply retrieving previous ones.

Current agent memory systems work like a search engine: after completing a task, an agent stores what it learned as a skill. Then, when facing a new problem, the agent retrieves the most relevant skill (or n concatenated skills) among the existing ones and adds it to its context.

The standard approach to skill retrieval.

This approach works well when one skill contains all the information the agent needs to solve a new task. In scientific reasoning, however, a new problem typically requires combining information from multiple sources — a mathematical derivation from one experiment, a physical principle from another, and a numerical technique learned somewhere else. Simply retrieving complete skills does not load and synthesize all the relevant information, and it introduces extra, unnecessary information in the agent context. Furthermore, simply concatenating the retrieved skills fails to create connections that gather complementary pieces of knowledge into a coherent solution strategy.

Our Skill-Synthesizer addresses these limitations in two ways. First, instead of retrieving entire skills, it retrieves only the relevant passages from across the set of available skills. Second, a dedicated synthesis agent reasons over these passages, identifies how they complement one another, verifies intermediate derivations through Python code execution, and produces a new skill tailored specifically to the current problem.

Skill-Synthesizer: ① relevant passages are extracted from the existing skills; ② a specialized agent reasons over them, and ③ synthesizes the information into a new, query-specific skill.

This process has two important advantages. First, it reduces noise by retrieving only relevant passages and ignoring irrelevant parts of previously learned skills. Second, it enables knowledge composition: the agent is no longer limited to reusing existing solutions, but can assemble new procedures by combining complementary ideas that were previously stored independently.

We evaluated Skill-Synthesizer on two challenging university-level physics benchmarks, UGPhysics and PHYSICS. Across both datasets, it consistently outperformed a broad range of baselines, including memoryless agents, LLM-based skill retrieval (LLM in the results table below), and embedding-based retrieval methods (Emb). It improved both answer accuracy and the quality of the reasoning process, with particularly large gains on graduate-level physics problems, where solving a question often requires integrating multiple concepts rather than recalling a single formula.

Our analysis also revealed why this works. Fine-grained retrieval successfully recovers information that would otherwise remain buried inside large skills, while the synthesis stage reconstructs complete solution procedures by combining information spread across multiple sources. An ablation study confirmed the crucial role played by the synthesis step: simply concatenating retrieved passages (passage-only in the table below) performs substantially worse than actively synthesizing them into a coherent procedure.

Another important component is code verification. During synthesis, the agent can execute Python code to manipulate symbolic expressions, check algebraic derivations, verify intermediate results, and validate numerical computations before producing the final skill. This makes the synthesized knowledge considerably more reliable, particularly for the derivation-heavy workflows common in scientific reasoning.

Our work builds on the idea that simply retrieving procedural knowledge is not sufficient to deal with new, complex scientific tasks. Rather, agents must be able to reorganize, combine, and verify their accumulated knowledge in order to solve new problems. Skill-Synthesizer is a step toward that future: AI systems that do not merely remember what they have learned, but continuously synthesize new expertise from their past experience.