What Broke My First Retrieval Pipeline

A contract-search prototype flagged half of a clause as a major risk. The condition that changed its meaning had landed in another chunk, so the model was working from an incomplete passage.

Clean the PDF text before indexing

The extraction also contained repeated headers, footers, page numbers, and broken line wraps. Those fragments entered the index alongside the contract text.

I added a cleanup step before creating embeddings. It filtered repeated line noise, repaired line wraps, normalized whitespace, and skipped empty blocks. It also retained page numbers and recognized headings as metadata rather than leaving them mixed into the passage.

Try paragraph and sentence breaks first

The original character-count cuts could split a clause from its condition. I used paragraph and sentence breaks as the first options, with character cuts as a fallback. A short overlap carried the tail of the previous chunk into the next one.

Keep the source with each result

The chunk records included the document, heading, page range, and index. Those details traveled with the retrieved text into the prompt, giving me a place to look when an answer needed checking.

For example, a passage could carry this source record:

source: Vendor Agreement
section: Limitation of Liability
pages: 14–15
chunk: 08

A fixed overlap could still cut through a sentence or leave out a longer condition. With the source record attached, I could compare the retrieved passage with the surrounding contract text and see what the model had been given.