Web Development

UTF-8 from Browser to MariaDB - Keep Text Intact Across a PHP Application

Written by Adam Muiz 01 Sep 2026 Updated: 01 Sep 2026 10 min read

A form accepts a name correctly, PHP prints it correctly in a debug message, and MariaDB stores something that still looks plausible. Later, the page shows replacement symbols, a search misses an apparently identical word, or an emoji disappears. Which layer changed the text?

There is no single "UTF-8 switch" for a web application. Text crosses several boundaries: browser to HTTP request, request bytes to PHP strings, PHP to the database connection, connection to a column, and stored bytes back to an HTML response. Each boundary has its own contract. One wrong assumption can remain hidden while an ASCII-only test suite continues to pass.

This article builds a layer-by-layer audit for a small PHP application using PDO and MariaDB. It is aimed at new or already UTF-8-based systems. Repairing legacy data is a separate forensic problem because changing a label cannot reveal how damaged bytes were originally interpreted.

Separate characters, bytes, and comparison rules

Unicode assigns code points to characters. UTF-8 is an encoding that represents Unicode scalar values as byte sequences. They are related, but they are not synonyms. The WHATWG Encoding Standard defines encoding as the mapping between scalar values and bytes, requires UTF-8 for new formats and contexts, and documents why disagreement between a producer and consumer can affect correctness and security.

A database collation answers another question: how should strings be compared and sorted? Two columns can both use utf8mb4 yet apply different rules for case, accents, or linguistic ordering. A font adds another independent layer by deciding which glyph appears on screen. Seeing an empty box can be a font problem; seeing U+FFFD, the replacement character, usually points toward an invalid or misdecoded byte sequence. Neither observation alone identifies where the original mistake occurred.

This separation suggests a useful debugging habit: at every boundary, ask what characters are intended, what bytes are present, which encoding is declared, and which comparison policy is active. "It looks fine here" is weaker evidence than those four answers.

Declare UTF-8 at the web boundary

An HTML response should identify its encoding consistently. PHP can send the media type and charset in the HTTP header, while the document carries an early metadata declaration:

<?php
header('Content-Type: text/html; charset=UTF-8');
?>
<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Text boundary test</title>
</head>

The HTML Standard defines <meta charset="utf-8"> as the document encoding declaration and recommends placing encoding information early. The HTTP response and HTML should agree rather than relying on browser detection. PHP source files, templates, translation files, and imported fixtures also need to be saved as UTF-8; a correct response label does not transcode a source file that was written in another encoding.

Requests need an equally clear contract. Modern HTML forms served as UTF-8 normally submit text using UTF-8, while JSON APIs commonly use UTF-8 as part of JSON's web ecosystem. The application should still define what it accepts and reject malformed input at a controlled boundary instead of guessing among several encodings.

Validate expected input instead of converting blindly

PHP strings are byte sequences; PHP does not attach a permanent UTF-8 type to each string. The mbstring function mb_check_encoding can test whether a byte stream is valid for a named encoding:

function requireUtf8(string $value): string
{
    if (!mb_check_encoding($value, 'UTF-8')) {
        throw new InvalidArgumentException('Expected valid UTF-8 text.');
    }

    return $value;
}

Validity is deliberately a narrow result. It says the bytes form legal UTF-8. It does not say the text is normalized, safe to put into HTML, acceptable as a username, or authorized for the current user. It also cannot detect every case of mojibake: text that was decoded incorrectly and then encoded as valid UTF-8 can pass the check while displaying the wrong characters.

Blind conversion is therefore risky. A call that "converts from whatever this might be" can turn uncertainty into permanent data loss. If an import genuinely uses a legacy encoding, identify that encoding from its documented source, convert once at the import boundary, and retain enough evidence to reproduce or reverse the process. Do not repeatedly run conversion over data whose history is unknown.

Set the database connection charset during connection setup

A utf8mb4 column does not by itself tell MariaDB how to interpret bytes arriving from a client. The client-server connection has character-set variables of its own. The PDO_MYSQL DSN documentation provides a charset element, so the intent can be established when PDO connects:

$pdo = new PDO(
    'mysql:host=127.0.0.1;dbname=appdb;charset=utf8mb4',
    $username,
    $password,
    [
        PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
        PDO::ATTR_EMULATE_PREPARES => false,
    ]
);

Specifying charset=utf8mb4 is preferable to hoping that a server, distribution, or upgraded installation has the expected default. It also places connection negotiation before application queries. Prepared statements remain necessary for separating SQL structure from values; the character-set option does not replace parameterization.

When diagnosing an environment, inspect the active session rather than only a configuration file:

SHOW SESSION VARIABLES LIKE 'character_set_%';
SHOW SESSION VARIABLES LIKE 'collation_connection';

The relevant values should tell one coherent story for the client, connection, and results. This observation is evidence for that connection, not proof that every worker or command-line import uses the same settings.

Audit the schema, not just the database default

MariaDB's character-set and collation documentation describes settings at server, database, table, column, and connection levels. Defaults cascade, but an older table or an explicitly configured column can differ from its database. Defaults also vary across MariaDB versions and distributions. A server-wide value is therefore not a schema audit.

SHOW CREATE TABLE exposes a table's effective definition. Information Schema can inventory all textual columns:

SELECT
  TABLE_NAME,
  COLUMN_NAME,
  CHARACTER_SET_NAME,
  COLLATION_NAME
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'appdb'
  AND CHARACTER_SET_NAME IS NOT NULL
ORDER BY TABLE_NAME, ORDINAL_POSITION;

For new schema, spell out utf8mb4 instead of the historical utf8 name. MariaDB's Unicode documentation explains that utf8 is an alias whose meaning can depend on configuration, while utf8mb4 stores supplementary characters. Then choose a collation according to actual comparison requirements and the MariaDB versions that must interoperate. There is no universal collation for every language and identifier policy. A case-insensitive title search, a case-sensitive token, and a unique username may need different reasoning.

Changing a collation is not merely cosmetic. It can change which values compare equal, how results sort, and whether existing values satisfy a unique index. Changing a character set can be more dangerous: MariaDB warns that conversion may lose data if the old declaration does not match the actual content. Back up first, inspect representative bytes, rehearse on a copy, and verify application behavior before altering production columns.

Keep output escaping as a separate control

Correct UTF-8 preserves text; it does not make untrusted text safe in HTML. Characters such as <, >, &, and quotes still have syntax-level meaning. For a plain HTML text or quoted-attribute context, PHP's htmlspecialchars documentation supports an explicit encoding and substitution policy:

echo htmlspecialchars(
    $value,
    ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML5,
    'UTF-8'
);

Other contexts, including JavaScript, CSS, URLs, and HTTP headers, have different encoding rules. Likewise, HTML escaping should normally happen when data enters an HTML output context, not before storage. Storing pre-escaped text mixes presentation with data and invites double encoding when the same value is later used elsewhere.

Test a real round trip with difficult text

An ASCII sentence cannot reveal whether a UTF-8 path works because ASCII bytes have the same basic representation inside UTF-8. A small test corpus should cross scripts and byte lengths, include a supplementary character such as an emoji, and include canonically equivalent sequences:

$samples = [
    'Semarang',
    'Grüße',
    '日本語',
    'مرحبا',
    'emoji: 🧭',
    "composed: é",
    "decomposed: e\u{0301}",
];

The test should submit these values through the same request parser used by the application, bind them through the same PDO path, read them from the target column, serialize the normal response, and compare the result with the original expectation. Inspect the HTTP Content-Type, database session variables, column definition, stored value, and returned bytes when a comparison fails. A direct database insert tests less of the system and can miss a broken web boundary.

Exact byte equality is appropriate when the application promises to preserve input exactly. Search keys and identifiers may require a documented normalization policy instead. The distinction should be explicit rather than discovered after visually identical strings become separate accounts or tags.

UTF-8 validity is not Unicode normalization

The visible character é can be represented as one precomposed code point or as e followed by a combining acute accent. Both sequences can be valid UTF-8 and look the same while failing byte-for-byte comparison. The W3C's Character Model for String Matching, currently a First Public Working Draft, explains canonical equivalence and also warns that normalization cannot make every identical-looking character sequence equivalent.

Normalization is consequently a product and data-model decision. Normalizing an identifier to NFC at a clearly documented boundary may improve predictable matching. Applying compatibility normalization indiscriminately can erase distinctions that matter. Database collation may handle some equivalences for comparison, but its behavior must be tested for the chosen version and collation; it should not be assumed to rewrite stored text into one normalization form.

Approach legacy corruption as evidence, not guesswork

When existing text is already wrong, changing the response header or column declaration may only change how the same bytes are interpreted next time. A string can also have been misdecoded and re-encoded more than once. There is no safe universal SQL statement that reconstructs an unknown history.

A cautious repair starts by stopping unnecessary writes, taking a verified backup, sampling raw bytes alongside current declarations, identifying the ingestion path and likely source encoding, and reproducing the proposed transformation on a copy. The acceptance test should include known original text from a trustworthy source. If that evidence is unavailable, uncertainty should be recorded rather than hidden behind a conversion that merely makes a few examples look better.

Conclusion

Reliable Unicode text in a PHP/MariaDB application is a chain of explicit agreements. The response declares UTF-8, inputs are validated against a known contract, PDO negotiates utf8mb4, schema columns use intentional character sets and collations, output is escaped for its context, and tests exercise the complete round trip with more than ASCII.

Even that chain has limits. It does not choose the right normalization or collation policy for a product, guarantee that every font has every glyph, or repair bytes whose history is unknown. What it provides is more practical: observable boundaries. When text changes, the investigation can identify which contract disagreed instead of adding another conversion and hoping the symptom disappears.