The standard advice for messy, unstructured data has not changed in years: write a regex, or throw some K-means clustering at it. After 14 years of building WordPress and WooCommerce systems, I can tell you that is the quickest way to wreck your data accuracy, and the maintenance never ends.
I recently had to sort several thousand rows of free-text annotations explaining security findings. Every developer phrased things their own way. One wrote “test code,” the next wrote “non-production env,” someone else wrote “CI/CD fixture.” Same meaning, three strings with almost nothing in common, and no traditional tool could see the connection. Zero-shot classification can.
Where traditional clustering falls down
Unsupervised clustering groups text by mathematical proximity, and that assumption breaks on short entries where every word carries weight. Embedding similarity ends up merging things that do not mean the same thing. “This key is for development” and “This API key is hardcoded” produce similar vectors because the tokens overlap, but one describes a safe environment and the other describes a security tradeoff. K-means and DBSCAN have no way to separate them.
Anyone who read my earlier argument for why you should not over-engineer your RAG vector database yet knows I lean toward the simpler option. Keyword matching is simple. It also cannot handle paraphrasing, so the rule file collects special case after special case until it reads like the legacy spaghetti nobody wants to open.
Zero-shot classification with a local LLM
The idea is straightforward. You define the categories yourself from what you know about the domain, and the language model does the mapping. Modern LLMs read for intent rather than token overlap, which is exactly where BERT and the smaller classifiers struggle. That is why a local model like Gemma 2 can beat a fine-tuned classifier on a small or medium dataset with no training data at all.
Here is how I run it locally with Ollama. Nothing leaves the machine, and there is no OpenAI bill waiting at the end of the month.
import ollama
# The template that forces the model into a classification role
CLASSIFICATION_PROMPT = """
Classify this text into one of these themes:
{themes}
Text: "{content}"
Respond with ONLY the theme number and name.
Format: THEME_NUMBER. THEME_NAME | Reason
Classification:
"""
def bbioon_classify_entry(content, themes):
prompt = CLASSIFICATION_PROMPT.format(themes=themes, content=content)
response = ollama.generate(
model="gemma2",
prompt=prompt,
options={
"temperature": 0.1, # Low temp for deterministic output
"num_predict": 100, # Prevent the model from writing an essay
}
)
return response['response']Gemma 2 (9B) is what I settled on. I ran the same set through Llama 3.2 (3B), which was faster, but it made worse calls on the ambiguous entries, and those are the ones you care about. On a MacBook Pro, roughly 7,000 entries took about 45 minutes. I will trade 45 minutes for clean, categorized data any day.
Running it over tens of thousands of rows
At tens of thousands of rows, a plain loop over the data will not hold up. Preprocess first to strip URLs and boilerplate phrases, then deduplicate by content hash, which can cut your token budget by around 30%. Save a checkpoint every 100 classifications too. I have lost enough long runs to a memory leak or a power flicker to stop trusting one.
In WordPress work, this is most useful for automating taxonomies, something I got into in my post on the WordPress AI Plugin 0.7.0 updates. The same setup handles bug triage and support ticket sorting, and it will file blog posts by what they actually say rather than which keywords happen to appear in them.
If zero-shot classification is eating your dev hours, I can take it off your plate. I have been wrestling with WordPress since the 4.x days.
When to use it, and when not to
Zero-shot classification fits the case where the text is semantically messy and you have no labeled training data. It does not fit every case. With 100,000+ entries and a sub-second latency requirement, embeddings and nearest-neighbor lookups are still the better answer. For cleaning up messy business data or internal logs, though, a local model will get you further than one more regex.
