Skip to main content
preprint arXiv preprint 2026

Aligned in Form, Not in Meaning: The Comprehension–Containment Decoupling of LLM Safety in Low-Resource Bangla Derogatory Speech

Shadab Bin Habib, A K M Ferdous Reza Habib, Subarno Neel, Adib Sakhawat

Safety alignment in a low-resource language can hold in form while failing in meaning.

BibTeX

@misc{habib2026aligned,
  title         = {Aligned in Form, Not in Meaning: The Comprehension-Containment Decoupling of LLM Safety in Low-Resource Bangla Derogatory Speech},
  author        = {Habib, Shadab Bin and Habib, A K M Ferdous Reza and Neel, Subarno and Sakhawat, Adib},
  year          = {2026},
  eprint        = {2608.02941},
  archivePrefix = {arXiv}
}

Description

The question

When a frontier LLM refuses to repeat a slur in English, does it refuse because it understands the harm, or because it recognises the surface form? This paper tests that in Bangla derogatory speech (gali), where the surface forms are rarely in safety training data.

The hypothesis: comprehension–containment decoupling

Safety alignment may be bound to high-resource surface form rather than harmful meaning. If so, a model's ability to comprehend a low-resource slur and its ability to contain it should operate independently. We tested this with five frontier LLMs across six protocols, against a human-calibrated baseline (κ = 0.84).

Results

Comprehension (Pass %) against containment failure (Use %, the rate at which the model reproduces the slur):

Condition Comprehension Containment failure
Bangla, native script 89.06% 92.83%
English reference 96.98% 92.83%
Romanised Bangla 83.77% 95.47%
Orthographically perturbed 76.23% 68.68%*
Chain-of-thought 94.72% 96.23%
Expert persona 63.98% 85.46%

* An apparent improvement that the paper shows is a containment mirage: leakage falls only because tokenisation breaks, not because safety engages.

Key findings

  • Same leakage, different understanding. Models understand Bangla slurs 7.92 points less well than English ones, yet leak them at an identical 92.83%.
  • Severity tracks anatomy, not harm. Severity judgements follow surface anatomical cues rather than compositional harm: +4.00 error on mild slang, −2.00 on threats.
  • Reasoning helps understanding and hurts safety. Chain-of-thought rescues comprehension to 94.72% while pushing containment failure to 96.23%.
  • Personas break refusal. Expert-persona framing collapses refusal to 6.57%, and keyword-style filters miss dehumanising communal slurs entirely.

Why it matters

High-resource safety benchmarks cannot certify safety in low-resource languages. Containment has to be grounded in meaning, not in the forms a model happened to see in training.

Resources