I honestly thought I’d seen every way a metadata extractor could break. Then I opened a Slack alert last Friday at 4:45 PM and saw our nightly batch job had crashed—again. My SLM for CI/CD Reliability journey started right there, in the middle of my seventh attempt to convince GPT-4 that “valid JSON” doesn’t include Markdown code fences.
I had written “MUST” in all-caps in the system prompt. To a language model. As if emphasis would work on something that doesn’t have feelings or, apparently, a consistent definition of what a snake_case key looks like. It didn’t work. Consequently, I had to refactor the entire approach.
The Problem with “Mostly Consistent”
For a nightly batch pipeline feeding a data warehouse, non-determinism is a bottleneck you can’t ignore. At temperature=0, GPT-4 gives you mostly consistent outputs. However, in a CI/CD context, “mostly” is just another way of saying “it will break when you’re on vacation.”
The failures were subtle. One night the model returned dataset_source, the next night datasetSource, and occasionally source_dataset. I spent longer than I’d like to admit staring at JSON nulls returned as Python “None” strings. After 23 pipeline failures in six weeks with zero actual bugs in my code, I stopped defending the setup.
Implementing a Local SLM for CI/CD Reliability
I went in expecting local models to fail on quality. Instead, I realized document extraction into a fixed schema isn’t actually a “frontier model” task. It’s structured reading comprehension. Specifically, I found that a well-tuned 7B model like Qwen2.5 handles this perfectly when seeded for determinism.
Using Ollama, I was able to enforce a seed: 42. This is the hack—or rather, the standard—that actually delivers reliability. Locally, same input plus same seed equals same output. Every single time. Furthermore, the latency dropped from 5.8s per doc to about 1.5s after the initial model load.
# The REST API call to Ollama ensuring determinism
def call_local_slm(doc_text):
payload = {
"model": "qwen2.5:7b-instruct",
"options": {
"temperature": 0,
"seed": 42 # This is the whole point
},
"stream": False,
"messages": [{"role": "user", "content": doc_text}]
}
# ... rest of the logicGitHub Actions and the Caching Gotcha
Moving this to a CI/CD pipeline introduced two non-obvious problems. First, service containers are network-isolated from the runner. You can’t just docker exec into them; you have to use the REST API for model pulls. Second, a cold pull is nearly 5GB. If you don’t cache the model, your SLM for CI/CD Reliability will add 4 minutes to every single run.
For more on automating your dev workflow, check out my guide on Agentic AI for Repositories.
The Pydantic Validation Fix
Even with a seeded model, Qwen occasionally gets excited and returns “Experimental” with a capital ‘E’ despite instructions. This workaround using a Pydantic field_validator with mode="before" catches those quirks before the type check fails the build.
from pydantic import BaseModel, field_validator
class DocMetadata(BaseModel):
methodology: Literal["experimental", "review", "simulation"]
@field_validator("methodology", mode="before")
@classmethod
def normalize_input(cls, v):
return v.lower().strip() if isinstance(v, str) else vLook, if this SLM for CI/CD Reliability stuff is eating up your dev hours, let me handle it. I’ve been wrestling with WordPress and backend automation since the 4.x days.
The Silent Pipeline
The morning the pipeline first ran clean—no notifications, no manual re-runs—I checked the logs twice. It felt strange. I’d spent months bracing for impact every time I pushed code. If you’re tired of playing whack-a-mole with probabilistic outputs, it might be time to ship it locally.
A local model on hardware you control is a different class of tool. It’s not better at everything, but it is better at being the same thing twice. And for a nightly pipeline, that’s exactly what matters. For technical reference, see the official Ollama API documentation or the Pydantic docs.
