Prompt optimization is worth doing, but in WordPress work it happens at build time rather than on the server. The tool doing it, DSPy, runs offline on your machine against a test set, and what you ship is still a prompt string inside your plugin’s PHP.
The source is Automate Writing Your LLM Prompts by W. Brett Kennedy. His argument is that hand-tuning prompts is slow and unreliable for software that runs unattended. You test a few rewrites on a few inputs, eyeball the output, keep the one that felt best. I agree. He doesn’t cover where the generated prompt should live once a plugin has to carry it, and that matters nearly as much as the tuning.
What the framework gives you
You don’t write the prompt. You write a signature like document -> plausibility_score, a test set of maybe 20 real inputs with ground truth, and a small metric function. abs(ground_truth – prediction) is enough for a numeric task, the kind Kennedy uses when he scores documents for plausibility on a 0 to 10 scale. DSPy then loops: candidates get generated by meta-prompting (one LLM writing prompts for another), scored against the test set, dropped early if they look weak, and the best one survives. You end with a prompt and a number, so prompts and models compare on a scale instead of on instinct. The Stanford NLP repo frames this as programming rather than prompting, and the optimizer list on dspy.ai runs from BootstrapFewShot to GEPA.
That output is plain text. Nothing on your server needs Python. You run the optimization locally, copy the winning prompt into the plugin, and commit it.
Where the prompt lives in a plugin
// Compiled offline against a test set. Version it like any other asset.
define( 'BBIOON_REVIEW_PROMPT_V3',
'You moderate product reviews for an outdoor gear store. Judge tone, spam '
. 'signals and relevance. Return only one word: approve, hold or reject.'
);
function bbioon_moderate_review( $review_text ) {
$prompt = str_replace( '{review}', $review_text, BBIOON_REVIEW_PROMPT_V3 );
// send $prompt, parse the one-word verdict
}I’ve seen three homes for a prompt in a plugin. The default is a hardcoded string you tuned by hand. The one that gets requested is an admin settings field the site owner can edit. The one I ship is a constant that holds a prompt compiled offline, with the version in the name.
The settings field is the tempting one and the wrong one. It takes the prompt out of version control, so behavior changes without a deploy and nobody can diff what changed. It also breaks in ways the client can’t see: one extra sentence about being friendly can turn a one-word verdict into a paragraph. If the owner wants control, add a knob instead. A strictness setting that appends one of a few pre-written lines buys them the feeling of control while the format contract stays fixed.
The constant is boring on purpose. It diffs in git, the score it earned can sit in a comment next to it, and swapping models means recompiling rather than guessing. Prompts tuned on one model don’t transfer cleanly, and providers retire models on their own schedule.
Where prompt optimization pays off
For a small site with an occasional feature, alt text generation or a monthly summary, write the prompt by hand, test it on ten real examples, and ship it as a constant anyway. If you’re still at the stage of moving pasted prompts into real workflows, a hand-written prompt is a reasonable step one.
A store is different once the prompt runs unattended on input nobody reviews first: review moderation, support email triage, extracting structured data from order notes. Those inputs are long, weird and occasionally hostile, and a prompt tested on five polite samples will miss most of it. Build the test set first, let the optimizer do the iterating.
The part you still have to build
The framework removes the iteration. The test set of 20 to 30 labeled inputs is still yours to produce, and for a store it usually exists already: past reviews with the moderator’s decision, past tickets with where they ended up. The owner’s opinion on the edge cases is the ground truth you want, so collecting it is closer to a conversation than a research project.
The metric is the other half. Classification and numeric scores are easy, an equality check or abs(a – b). Long output needs an LLM-as-judge, which Kennedy flags as imperfect, and it is the wobbly part: the judge’s tastes become your quality bar and every optimization run pays for judge calls. Keep a holdout slice, maybe a third of the data, that the optimizer never sees. The score that matters is the one on data it didn’t train against.
If you have an AI feature on a store that keeps drifting, or you want a prompt compiled against your own content instead of guessed at, that’s work I take on: build the test set with you, run the optimization, ship the plugin side of it.
When the provider retires the model your prompt was tuned on, re-score against the holdout before treating the replacement as a drop-in. A prompt that agreed with the moderator on nearly everything can drift badly on the new model, and nothing errors. The score is the only thing that tells you.
