Skip to main content
accepted REALM Workshop @ EMNLP 2026

AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue

Adib Sakhawat, Fardeen Sadab, Rakin Shahriar

Across 439 games, containment is tightly clustered (σ = 1.9 Elo) while extraction varies widely (σ = 53.3 Elo); confirmation framing raises extraction odds 7.75×.

BibTeX

@inproceedings{sakhawat2026aidg,
  title     = {{AIDG}: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue},
  author    = {Sakhawat, Adib and Sadab, Fardeen and Shahriar, Rakin},
  booktitle = {Proceedings of the REALM Workshop at EMNLP 2026},
  year      = {2026},
  eprint    = {2602.17443},
  archivePrefix = {arXiv}
}

Description

Multi-turn LLM evaluations usually report one number, a win rate. But winning a conversational game takes two different skills: extracting information you don't have, and containing information you do. AIDG separates them.

The framework

AIDG (Adversarial Information Deduction Game) formalises multi-turn adversarial dialogue as a two-player partially observable stochastic game with two roles:

  • Seeker: tries to extract a hidden secret through questioning.
  • Holder: tries to keep it while still answering.

The decomposition isolates three failure modes: cooperative-prior leakage (giving information away out of helpfulness), constraint–reasoning interference (losing track of the rules while reasoning), and inefficient hypothesis-space traversal (asking questions that don't narrow the search).

Results

439 games across six frontier LLMs, rated with Bradley–Terry Elo:

Model Extraction (Seeker Elo) Containment (Holder Elo) Gap
Qwen3-235B 1443.6 1627.1 183.5
GPT-5 1409.9 1626.1 216.2
Gemini-2.5-Flash 1372.1 1626.3 254.2
DeepSeek-V3.1 1360.0 1629.7 269.7
Llama-4-Maverick 1349.5 1630.9 281.4
Granite-3.3-8B 1297.9 1626.6 328.7
Standard deviation 53.3 1.9

Key findings

  • Containment is uniform; extraction is not. Holder ratings sit within a 1.9-point spread, while Seeker ratings vary 28 times as much.
  • Confirmation beats guessing. Confirmation-style questions raise extraction odds 7.75× over uninformed deduction (p below 0.00001).
  • Rule-following breaks under pressure. Constraint violations cause 41.3% of deductive failures, and the rate does not improve with scale (ρ = 0.0).

Why it matters

The containment-over-extraction gap is not a surprise to be reported but a predictable consequence: defensive decisions can be made locally, turn by turn, while offensive planning needs a global view of the conversation. The decomposition lets you attribute that gap model by model, with every design choice derived from explicit assumptions.

Resources