Validating GPT-2 Output Before Database Writes

A GPT-2 extraction prototype returned prose where the application expected JSON. The parser failed before the result could be written.

The prototype took contract text, proposed a clause record, and prepared that record for review. The model could change field names, add commentary, or produce a value unsupported by the source.

Generated text stayed outside the write path

I separated extraction from approval and storage:

contract text
→ GPT-2 completion
→ raw output capture
→ JSON parser
→ schema validation
→ review
→ database write

The extraction wrapper kept the latest raw output while it tried to build a typed record. A successful parse made the result eligible for review, without deciding whether the clause had been interpreted correctly.

The prompt showed the required record shape

The prompt paired contract snippets with JSON examples. It named the required fields and asked for an extracted value alongside the source text.

A contract qualifier can change the value even when the JSON is valid. In the payment example, the source allows different terms agreed in writing, while the extracted value states only fifteen days. A reviewer needs to check that omission before accepting the record.

The schema controlled what could continue

The wrapper extracted a JSON block, supplied the document ID from the request, and constructed a typed record. Invalid JSON and values rejected by that record's schema entered the retry path. The document ID came from the application rather than a model guess.

Attempts stopped at a fixed limit. If every attempt failed, the wrapper returned a failure record containing the input, last raw output, validation error, model name, prompt version, and attempt count.

The function named send_to_dead_letter_queue constructs and returns a record in this excerpt. It doesn't show a queue write or durable storage. A caller still needs to handle that result, and any retained contract text needs defined access and retention rules.

Check the value against the contract

The model also generates the source_text field. Carrying a quotation helps a reviewer find the passage, but the wrapper doesn't verify that it occurs in the supplied contract. The reviewer has to compare both the quotation and the extracted value with the original text.

The shown code ends with a typed result or a failure record. Approval and the database write belong to the calling application; a valid schema alone shouldn't authorize either one.