Why Are Accented Characters Wrong After Decoding Base64 XML?

Base64 restores bytes; it does not choose or change the XML character encoding. For intact decoded bytes, check the XML encoding declaration and the character decoding performed before parsing: a wrong default can create an exception or mojibake. Pass the restored bytes directly to an XML parser that supports the declared encoding, rather than blindly converting every payload to UTF-8 text first.

Declared bytes versus forced UTF-8

On 2026-10-28, Python 3.12.14 with ElementTree 1.3.0 restored windows-1252.xml from 88 Base64 characters to exactly 64 bytes. ET.fromstring(restored).text returned café from those bytes, whose declaration was encoding="windows-1252". Calling restored.decode("utf8") first instead raised UnicodeDecodeError: byte 0xe9 at position 55 was an invalid continuation byte. The input SHA-256 was 3558f2afe3f66b30bbfc62538c15e13393aa8d4deb56197ab983ef0d39a3d378, and the restored bytes had the same hash.

A declaration cannot undo a wrong predecode

The utf-8.xml control restored 80 Base64 characters to 58 bytes, including c3 a9 for its accented letter. Direct byte parsing returned café, and explicitly decoding those bytes as UTF-8 before parsing also returned café. In contrast, ET.fromstring(restored.decode("latin1")).text returned café for that same 58-byte input. Its encoding="utf-8" declaration did not repair the already misdecoded characters in the Python string. The UTF-8 control SHA-256 was 2435b1a9769e7c960422042c9d060e4e7554a2cf5ca0e64a2fa32eb9314f9b53 before and after Base64 restoration.

Why the string path differs

Once bytes have been converted into a Unicode string, the text decoder has already assigned characters. An XML declaration inside that string cannot recover discarded byte interpretation. Do not use Latin-1 as a universal repair: it happens to map the accented byte in this particular Windows-1252 fixture to the intended character. That coincidence does not establish that the two encodings are interchangeable.

Missing and false declarations

The no-declaration.xml fixture restored 28 Base64 characters to 19 bytes, with e9 at offset 10 and no byte-order mark. Parsing its restored bytes raised ParseError: not well-formed (invalid token): line 1, column 10. For this same input, explicit UTF-8 text decoding failed at position 10, whereas Latin-1 decoding followed by parsing returned café. The mismatched declaration control, mislabeled.xml, restored 76 Base64 characters to 57 bytes; it declared UTF-8 but contained e9 at offset 48. Its byte parse failed at line 1, column 48, while the forced UTF-8 text path raised UnicodeDecodeError at position 48. All four restored files were byte-for-byte equal to their respective inputs, including the two that the byte parser rejected.

Keep byte decoding and XML parsing separate

Keep the restored bytes and inspect the declaration before choosing a text decoder. For a trusted XML payload, try ET.fromstring(base64.b64decode(value, validate=True)) without the intermediate text decode. If it fails, compare the declared encoding, the actual offending bytes and the producing application. Correct a false declaration at the producer or transcode the bytes with a known source encoding and update the declaration together; merely relabeling arbitrary bytes is not a conversion.

Questions about the failure boundary

Can I fix this by changing Base64 padding? No: padding cannot reconcile an XML declaration with the wrong character bytes. Can a successful XML parse still contain mojibake? Yes: parsing validates markup, not the intended spelling of a label. Compare the expected characters with the parsed result instead of treating absence of an exception as proof of correct text.

Reproduce this check and its limits

These generated, trusted fixtures isolate encoding declarations and predecoding with the named Python parser; they are not tests of every XML implementation. They contain no DTD, external entities, HTTP headers or BOM. Transport metadata and explicit parser overrides can affect encoding selection in other paths. These checks do not establish a safe policy for parsing untrusted XML. Run reproduce.py in an empty directory to regenerate the four files, their Base64 strings and raw-output.json, including every exact exception message.

If your issue instead appears only when a multibyte character crosses a read boundary, use the streaming UTF-8 boundary experiment; that is a decoder-state problem, not this declaration test.