MemoHarness Explained: Agent Harnesses That Learn from Experience

The July 14, 2026 preprint “MemoHarness: Agent Harnesses That Learn from Experience” proposes an alternative to a single fixed harness. It searches over a small set of labeled tasks, retains what it learns, and then builds a better-suited harness for a new case without test labels, gradient updates, or another search loop.

MemoHarness is not an official DeepSeek Harness feature or a DSH plugin. It is independent research, but its treatment of context, tools, generation, orchestration, memory, and output handling as optimizable harness surfaces is relevant to composable systems such as DSH.

The problem it addresses

A fixed harness typically applies the same prompt, tool policy, and context management to every task. That is easier to deploy, but it may not suit terminal operations, code generation, and financial analysis equally well.

MemoHarness defines the harness as the control layer outside the model and identifies six editable surfaces:

• Context: how context is selected, compressed, and organized.
• Tool: which tools are available and how they are described and constrained.
• Generation: sampling, budget, and related generation settings.
• Orchestration: planning, loops, retries, and decomposition.
• Memory: how prior experience is written, retrieved, and used.
• Output: how results are parsed, checked, and delivered.

The central idea is not simply to rewrite a system prompt. It is to turn decisions across these surfaces into experience that can be retained, retrieved, and transferred.

How the dual-layer experience bank works

MemoHarness keeps both case-specific entries and global patterns distilled across cases.

During search, it tries harness edits on labeled examples. Candidates are ranked by correctness first, with token use as a tie-breaker. Successful and unsuccessful edits both become experience that can inform later decisions.

At test time, the system retrieves relevant experience and adapts a global harness into a case-specific one. The paper emphasizes that this step uses no test labels, execution feedback, gradient update, or extra search. It is therefore different from repeatedly trying solutions on the test case until one passes.

What the paper reports

The following are author-reported results under the paper's protocol, not independent replications or universal product claims:

On Terminal-Bench, the paper says 0.806 exceeds its strongest fixed baseline, Codex at 0.722. In cross-model testing, the authors report a mean gain of 0.098 over six held-out models; DeepSeek V3.2 rises from 0.333 to 0.444, while GLM-5 rises from 0.500 to 0.733.

These figures do not mean that MemoHarness universally beats Codex, nor do they establish a general model ranking. Benchmark versions, task samples, execution environments, fixed-baseline configurations, and scoring protocols all affect the result.

MemoHarness versus JIT-Agent

Both projects treat the harness as a first-order performance variable, but they adapt it at different moments:

JIT-Agent can be understood as constructing for the present task, while MemoHarness brings lessons from earlier executions into the present task. They are not direct substitutes, and the papers do not provide a comprehensive comparison under one common protocol.

What DSH developers can take from it

Treat harness configuration as an evaluated artifact

Version prompts, tool sets, permissions, context compression, loop limits, and output checks. Recording only a model name cannot explain why the same model behaves differently across two evaluations.

Preserve failed experience as well as success

A failed trajectory can reveal which tools, context structures, or recovery policies do not suit a task class. Such records must remove secrets, personal data, and code that should not cross project boundaries.

Measure cost alongside correctness

The paper uses token count to break ties between equally correct candidates. A production evaluation should also include latency, tool calls, human approvals, and recovery cost.

Keep learned adaptations inside security ceilings

An experience-derived harness should not grant itself broader file, network, or process permissions. Adaptation belongs inside fixed permission boundaries and tool allowlists; one successful prior run is not permanent authorization.

Open questions

• This is a preprint, and the authors call for larger studies and finer-grained ablations.
• Cross-dataset transfer is selective rather than uniform; some saturated suites do not change, and results on legal tasks are mixed.
• An experience bank can become stale, poisoned, or sensitive, requiring provenance, versioning, expiration, and deletion controls.
• Search requires labeled cases and compute; practical ROI must include that acquisition cost.

Related reading

• JIT-Agent explained: can AI generate a better agent harness?
• Why DSH, deepagents, and Pi converge on a similar harness architecture
• DSH Agent presets versus plugins

FAQ

Is MemoHarness an official DeepSeek project?

No. It is independent research with no official relationship to DeepSeek Harness; its cross-model evaluation includes DeepSeek V3.2.

Does “no test-time feedback” mean it never executes the task?

No. It still executes the task with the generated harness. The phrase means adaptation does not use test labels or execution results to run another search or parameter update.

Does DSH already learn harnesses automatically?

The cited paper does not show that DSH implements MemoHarness. For DSH, it is better understood as a research direction for evaluation, preset versioning, and security-aware experience stores.