Got a call from a client last week. They’d just moved their WooCommerce store off some ancient hosting setup, and things were acting strange. Their European customers were complaining that reviews and contact forms submitted with special characters, think umlauts and accent marks, came out as garbled nonsense. Just a sea of those black diamond question mark symbols (�). A real mess.
It was a classic WordPress UTF-8 encoding problem. My first instinct was to find where the data got saved and slap a utf8_encode() on it, the quick and dirty fix you see all over Stack Overflow. It might have worked for some characters, but it’s the wrong tool. It assumes the original text is ISO-8859-1, which is a gamble and often makes things worse. You end up with double-encoded garbage, or you break strings that were perfectly fine to begin with.
Why most UTF-8 fixes are just guesswork
For years, WordPress shipped functions like seems_utf8() that encouraged exactly this kind of guesswork. The name says it all. It seems right? That is not a solid basis for handling data reliably, especially in an e-commerce store. A string in PHP is just a sequence of bytes. Without knowing the encoding, you have no way to tell whether the byte 0xA9 is meant to be © or something else entirely. Guessing invites data corruption.
WordPress 6.9 finally modernizes how it handles UTF-8, and it’s a change I’m glad to see. The core team deprecated those old, misleading functions and gave us more reliable tools. It’s all laid out in a recent dev note on the Make WordPress Core blog, which you can find at https://make.wordpress.org/core/2025/11/18/modernizing-utf-8-support-in-wordpress-6-9/.
The right way to handle WordPress UTF-8 encoding
Instead of guessing, the new approach is built around validation. There are two functions for it: wp_is_valid_utf8() and wp_scrub_utf8(). One checks, the other cleans.
wp_is_valid_utf8( $string ): exactly what the name suggests. It returnstruewhen the string is a valid sequence of UTF-8 bytes andfalsewhen it isn’t, with nothing left to guess.wp_scrub_utf8( $string ): for the cases where you have to accept a string that might contain invalid bytes. It replaces the bad parts with the standard Unicode Replacement Character (�), so you can save it to the database without breaking something downstream, like an XML feed.
So for my client’s site, the fix wasn’t to encode anything. It was to validate the input when the form was submitted. Here’s what the logic looks like now:
<?php
$submitted_review = $_POST['customer_review'];
// Stop guessing and just validate the input.
if ( ! wp_is_valid_utf8( $submitted_review ) ) {
// The input is not valid UTF-8. Reject it.
// You could also try to scrub it, but rejecting is often safer.
wp_die( 'Invalid character encoding detected. Please use UTF-8.' );
}
// If we're here, the string is valid. Proceed with saving it.
$safe_review = sanitize_textarea_field( $submitted_review );
// ... rest of the save logic ...So, what’s the point?
The takeaway is a shift in mindset. Stop trying to magically “fix” broken strings and start enforcing a standard at the point of entry. It isn’t about converting character sets on the fly, it’s about deciding what your application will accept.
- Don’t guess: If you don’t know a string’s encoding for certain, don’t try to convert it.
- Validate on input: Use
wp_is_valid_utf8()to check data coming from forms or APIs, and reject anything that isn’t valid. - Scrub, don’t strip: If you must clean a string, use
wp_scrub_utf8()to replace bad bytes. Don’t just strip them out, since that can create new and dangerous byte combinations.
This stuff gets complicated fast. If you’re tired of debugging someone else’s mess and just want your site to work, drop my team a line. We’ve probably seen it before.
Character encoding isn’t the most exciting topic, but getting it right is the difference between a site that works for a global audience and one that quietly corrupts its own data.
