RAG chunking: preserve source positions and test retrieval units

Splitting a document into equal lengths does not automatically produce usable evidence. If “disconnect power before opening the cover” is divided between its condition and action, retrieval may return only the action. The goal is to preserve answerable context together with its source position.

This article builds a character-based baseline with Python 3. Its size and overlap values are experimental choices, not RAGO-X production defaults.

Inspect extraction before splitting

Broken reading order in multi-column PDFs, repeated headers, and misplaced table cells cannot be repaired by adjusting chunk size. Check headings, paragraphs, tables, and footnotes against the original. Prefer sections and paragraphs as initial units and split long paragraphs further when necessary.

A table row often needs column names and units. For long tables, associate each piece with its caption and headers and retain page boundaries. If you add headers to a piece, store the original source span separately from context added for retrieval.

A baseline that preserves offsets

Save the following as chunk_demo.py and run python3 chunk_demo.py. Half-open intervals [start, end) ensure that text[start:end] reproduces the chunk.

def split_text(text, size=80, overlap=15):
    if size <= 0 or not 0 <= overlap < size:
        raise ValueError("require size > 0 and 0 <= overlap < size")
    start = 0
    while start < len(text):
        end = min(start + size, len(text))
        yield {"start": start, "end": end, "text": text[start:end]}
        if end == len(text):
            break
        start = end - overlap


text = "Pump X17: disconnect power. Wait 30 seconds. Check the inlet filter."
chunks = list(split_text(text, size=40, overlap=10))
print([(c["start"], c["end"]) for c in chunks])
assert [(c["start"], c["end"]) for c in chunks] == [(0, 40), (30, 68)]
assert all(text[c["start"]:c["end"]] == c["text"] for c in chunks)
assert set().union(*(set(range(c["start"], c["end"])) for c in chunks)) == set(range(len(text)))
assert list(split_text("")) == []

The output is [(0, 40), (30, 68)]: two spans overlap by 10 characters and cover the full input. Empty input yields no chunks; invalid overlap raises an error. The function does not strip or normalize the input, preserving offset comparisons.

Python string length is not byte length, a count of visual grapheme clusters, or a model token count. Check the actual model limit with its tokenizer. This baseline also ignores sentence and heading boundaries; it is not a complete production splitter.

Preserve enough metadata

Offsets must identify the version of extracted text they refer to. OCR or whitespace changes can move the same passage. When replacing a document, switch chunks and indexes consistently and verify that obsolete revisions are no longer retrieved. These are design concepts, not a public API schema.

Label evidence before tuning size

Mark the source passages that answer real questions, then compare two splitting configurations. Check whether top results contain the answer and its exceptions, and whether opening the source shows the matching revision. More chunks or longer answers are not evidence of improvement.

Overlap may reduce boundary loss but also increases storage and redundant candidates. Record the question set, input tokens, latency, and citation accuracy instead of assuming a universally optimal size. Continue with hybrid retrieval evaluation and the evidence-to-answer workflow.