The Chunk Stability Index: Measuring Extraction Consistency
A chunk that holds together under one chunking strategy but falls apart under another is an unstable chunk. The Chunk Stability Index measures how consistently your content maintains its semantic integrity across different extraction approaches.
Different AI systems use different chunking strategies. Some split on fixed token counts. Some split on paragraph boundaries. Some use heading tags as chunk boundaries. The same page, processed by different systems, can produce radically different chunks — which means the same content can be highly retrievable in one system and poorly retrievable in another. The Chunk Stability Index measures this variance.
How the Index Is Computed
The Chunk Stability Index requires a minimum of three independent chunking passes with different boundary strategies. For each pass, the system extracts the chunks and measures: how many chunks contain all three required properties (entity clarity, claim completeness, self-containment). The stability index is the proportion of semantic units that maintain all three properties across all three chunking strategies. A score of 1.0 means every semantic unit is stable across all strategies. A score of 0.4 means 60% of semantic units fall apart under at least one chunking strategy.
What Low Stability Looks Like
Low chunk stability is typically caused by: entity references in headings that do not repeat in the paragraph body, multi-paragraph claims where the conclusion is in a separate chunk from the premise, context-dependent language ("As mentioned above," "Following the previous section,"), and page structures where the most important information is concentrated in the first paragraph under each heading, with the rest of the heading section providing context.
◆The single highest-impact fix for low chunk stability: ensure every paragraph begins with an explicit subject (entity or concept name) rather than a pronoun or reference to a previous section. "The machine trust score is computed from five sub-scores" is stable. "It is computed from five sub-scores" is not.
Stability vs Quality
A page can have high chunk stability and low chunk quality, or high chunk quality and low stability. Stability measures consistency across extraction strategies. It does not measure whether the chunks are good. A page full of complete but shallow, context-free sentences might have a high stability index (every chunk is self-contained) but a low AI Extractability Score (the chunks have nothing meaningful to extract). Both dimensions are necessary.
AI Visibility Engineering — Part 5 of 10