CI/CD for APIs Handling Sensitive Data

A sensitive-data API cannot depend on whatever happens to build that day.

That was the problem I wanted to remove.

The service handled document classification with private input data. Some payloads had names, company details, financial terms, and internal notes. The risk wasn’t only that the API could crash. The bigger risk was shipping a container where validation, preprocessing, or dependency behavior changed without anyone noticing.

The early deployment flow was too loose:

local update
→ Docker build
→ staging deploy
→ manual smoke test
→ production push

That worked while the service was small. It became weak once the API started handling sensitive inputs, because the deployment path had too many places where behavior could drift.

One dependency update could change preprocessing.
One schema mistake could allow bad payloads through.
One missing environment variable could break startup.
One manual test could miss the actual failure path.

I made builds reproducible first

I rebuilt the pipeline around reproducibility and release gates.

First, I locked the Python environment. I moved the service from a loose requirements.txt setup to Poetry, with the full dependency tree pinned in poetry.lock.

The goal was simple: local, CI, staging, and production should build from the same package graph. No silent sub-dependency movement. No “latest compatible version” surprise during Docker build. No deployment changing just because time passed.

The dependency graph became part of the release artifact instead of a moving target during Docker build.

GitHub Actions became the required path to staging

Then I made GitHub Actions the required path to staging.

The pipeline became:

commit pushed
→ locked dependencies installed
→ validation tests run
→ Docker image builds
→ container starts inside CI
→ mock sensitive-data payload sent to the API
→ response schema checked
→ image pushed to registry
→ staging deploy allowed

The image had to pass dependency install, validation tests, container startup, and endpoint checks before staging.

The most useful step was booting the actual container inside CI.

Unit tests are helpful, but they don’t prove the service can start from a clean image. A bad entrypoint, missing system package, broken import, or wrong runtime variable can still kill the API before it receives traffic.

So the pipeline ran the built image, waited for the port to open, then sent a mock request through the same endpoint used in staging. The request used fake data shaped like the real sensitive payloads: long text fields, optional metadata, missing values, and fields that had to be rejected.

A passing build had to prove four things:

• the locked environment installs cleanly
• the validation layer rejects unsafe request shapes
• the container starts from scratch
• the API returns the expected response contract

Validation moved risk out of runtime

The validation tests mattered because sensitive-data systems fail badly when they accept vague input.

For this service, the pipeline checked:

• required fields
• maximum payload size
• empty or malformed text
• unexpected metadata keys
• preprocessing assumptions
• response shape
• error handling for rejected payloads

That moved a lot of risk out of runtime and into CI.

The deployment artifact also became easier to trust. If the image reached the registry, it had already passed dependency resolution, schema tests, startup, and a real endpoint check. Staging stopped being the first place where basic container health was discovered.

The service still used a classification model, but the useful engineering work sat around it: reproducible builds, strict request contracts, container validation, and controlled promotion between environments.

That’s the part that matters when private data enters the system. You need fewer surprises. You need the same build path every time. You need bad inputs rejected before they touch deeper processing. You need broken containers stopped before deployment.

Result

After this change, deployment became slower in the right place and faster everywhere else. CI spent extra time proving the image was valid. I spent less time checking whether staging had drifted. More importantly, sensitive payload handling became part of the release process instead of something manually remembered at the end.

A small API becomes a real backend system when its deployment path is part of the design.

Onto the next one. Let’s keep sharpening that edge.

First written on September 08, 2021.