What Makes Structured Episodic Retrieval Robust to Noisy Query Cues?

A Controlled Study

Under review  ·  ACL submission
Anonymous authors
Retrieval rules on one EPBench question: cue extraction with calibrated confidence, Tier-1 index lookup, and Hard-AND, Literal and Graded combination of per-axis matches.

Figure 1: Retrieval rules on one EPBench question. (A) An LLM extracts a cue value for each queried axis with a self-reported confidence, which isotonic calibration turns into Confk. (B) Tier-1 index maps look up exact keys and list candidate cells; Tier 2 stores the cells. (C) Each candidate gets a graded embedding match mk per axis, and three rules combine them. The gold cell 2 misses the content threshold (0.93 < 0.98), so Hard-AND returns nothing, while Literal and the graded sum both retrieve it.

63.2%of EPBench questions get an empty context under hard intersection (Hard-AND)
+0.240 F1for the graded, confidence-weighted sum over Hard-AND ([0.194, 0.285])
+7.9 ptsLoCoMo answer accuracy from matching the speaker on top of dense retrieval ([6.0, 9.9])

Abstract

Structured memories for LLM agents index events by time, place, entity and topic, and answer a query by intersecting the matches on each axis. Intersection fails when an LLM extracts the events and the query cues, because one weak axis removes the right memory. We run a controlled study of retrieval rules on EPBench and LoCoMo. Hard intersection returns nothing for 63% of EPBench questions, and adding up the per-axis matches instead recovers most of the loss. How best to add them up depends on what the answer does with the retrieved set. A ranked top-K set helps questions that list events but hurts questions that ask for the most recent event, and routing those questions to a selective rule recovers the retrieval loss. Cue confidence is informative, yet as a single weight per axis it cannot change the result in most cases and leaves retrieval and answer accuracy essentially unchanged.

On LoCoMo, matching the speaker, which is metadata, raises answer accuracy over dense retrieval by 7.9 points ([6.0, 9.9]; 7.0 for the full score), while turning the topic score into a yes-or-no indicator, or choosing the weights on other conversations, makes no detectable difference. All results carry paired bootstrap intervals, and our conclusions are limited to these two benchmarks.

Four findings

  1. Hard intersection breaks down. It returns nothing for 63.2% of EPBench questions and for 49.6% of those that have an answer. Adding up the matches instead (Literal) gains +0.217 F1, and lowering the content threshold to 0.90 does not close the gap.
  2. The combination rule has to fit the question. A ranked top-K set raises gold-chapter recall on list and order questions (+0.116), but drops it on “latest” questions (−0.479), because later non-answer events slip in. Routing “latest” questions to Literal recovers the loss.
  3. Confidence tracks correctness but changes almost nothing. Wrong time cues average 0.14 against 0.99 for right ones after calibration, yet calibrated, raw and no confidence are indistinguishable: the retrieved set changes for only 41 of 668 questions. A single weight per axis provably cannot reorder single-axis queries or change the Literal set (Proposition 1).
  4. Efficiency, measured again. The index cuts the cells to score by 90.3% on EPBench, and scoring only the candidates is about 3× faster than a flat scan — both small next to the 7.7 ms needed to embed a new cue string.

Reference

Anonymous authors. What Makes Structured Episodic Retrieval Robust to Noisy Query Cues? A Controlled Study. Under review, 2026.

BibTeX citation

@misc{anonymous2026structured,
  title  = {What Makes Structured Episodic Retrieval Robust to Noisy
            Query Cues? A Controlled Study},
  author = {Anonymous},
  note   = {Under review},
  year   = {2026}
}

Method

A query constrains some axes — time, place, entity, topic or persona (the speaker). For each such axis k an LLM extracts a cue Ck and a self-reported confidence, which per-axis isotonic regression turns into Confk, the estimated probability that the cue is right. A two-tier memory stores the cells (events on EPBench, dialogue turns on LoCoMo) in Tier 2, and flat inverted index maps from cue values to cells in Tier 1. Every candidate cell e gets a graded match mk(e) ∈ [0, 1] per axis, and a rule combines the matches. We compare four rules on the same candidates, cues and judge:

RuleKeeps a cell when…Uses confidence?
Hard-AND (STEM / ARTEM)mk(e) ≥ ρk on every queried axisno
Gated-ANDsame, but only over axes with Confk ≥ γas a gate
LiteralS(e) = Σk Confk · 1[mk(e) ≥ ρk] > 0as a weight
Graded (CIRCUIT)top-K by S(e) = Σk Confk · mk(e)as a weight

Thresholds follow the published values ρ = (1.0, 0.99, 0.99, 0.98) for time, place, entity and content; γ = 0.75 and K = 8. For “latest” questions the protocol answers with the time-latest kept cell; otherwise all kept cells are returned in time order.

Results

RuleEPBench Book 1LoCoMo
Empty (%)F1Recall list/orderRecall latestGold in ctx. (%)QA acc. (%)
Dense retrieval0.0–0.5180.04277.472.5
Hard-AND63.20.5080.4860.271––
Gated-AND55.20.5740.5490.667––
Literal31.30.7250.8000.792––
Graded (CIRCUIT)1.00.7480.9160.31289.679.5
Router: latest → Literal, else Graded3.1–0.9160.792––

Table 1 (excerpt): controlled comparison of retrieval rules. F1 on 611 judge-valid EPBench questions; recall is gold-chapter recall without a judge (440 list/order, 48 latest answerable questions); LoCoMo on 1,536 questions. Dashes: not run. Full table with confidence variants and intervals in the paper.

EPBench Book 1: F1 with 95% intervals, empty-context rate and the effect of loosening the content threshold.
Figure 2: EPBench Book 1. (a) Lenient F1 on 611 judge-valid questions with 95% bootstrap intervals. (b) Share of the 488 answerable questions that receive an empty context. (c) Loosening Hard-AND's content threshold from 0.98 to 0.90 raises its F1 to 0.628, still 0.120 below the graded sum (CIRCUIT).
Gold-chapter recall of the graded sum as the context size K varies, for list/order and latest questions.
Figure 3: Gold-chapter recall of the graded sum as the context size K varies (440 list/order and 48 “latest” answerable questions). The two question types want opposite settings of K; the orange line is Literal, which has no K.
Paired differences between the graded sum and Hard-AND or Literal, with 95% intervals.
Figure 4: Paired differences of the graded sum (CIRCUIT) against Hard-AND and Literal. It wins clearly on answerable questions, but loses on the 142 questions whose gold answer is empty, because a soft rule always returns K cells and cannot abstain.