Guides

Why Excel Garbles UTF-8 CSVs — and How UTF-8 BOM Fixes It

Stop Microsoft Excel from turning Chinese, Japanese, Korean, Spanish, and German CSV text into mojibake by adding a 3-byte UTF-8 BOM locally.

3 min read

Quick Answer: Microsoft Excel on Windows opens UTF-8 `.csv` files as the local ANSI code page (`Windows-1252`, `GBK`, or `Shift-JIS`) whenever the 3-byte UTF-8 Byte Order Mark (`0xEF 0xBB 0xBF`) is missing. Prepending the BOM plus Windows `CRLF` line endings locally in browser memory lets double-clicked CSVs display CJK and accented text cleanly.

Why double-clicking a valid UTF-8 CSV fails in Windows Excel

Modern web dashboards, Python pandas scripts, and API exports save CSV files as UTF-8 without a Byte Order Mark (BOM). However, when Windows Excel opens a `.csv` file via double-click, it skips UTF-8 detection if no BOM is present and falls back to the OS ANSI code page (`Windows-1252`, `GBK`, `Shift-JIS`, or `CP949`).

As a result, perfectly valid characters like `张伟`, `東京`, `José`, or `Müller` immediately render as garbled mojibake such as `å¼ ä¼Ÿ` or `José`.

  • UTF-8 without BOM works in Unix/VS Code but fails Excel double-click.
  • Excel checks the first 3 bytes (`0xEF 0xBB 0xBF`) before decoding CSV rows.
  • Windows CRLF (`\r\n`) line endings also prevent single-column row collapse.

The 3-byte EF BB BF signature explained

Prepending the 3-byte UTF-8 Byte Order Mark (`0xEF 0xBB 0xBF`) directly at byte offset 0 signals Excel, LibreOffice Calc, and Power BI to decode the entire stream as strict UTF-8.

Unlike opening an external text editor and re-saving manually (which often truncates leading zeros on ZIP codes or converts long IDs to scientific notation), byte-level BOM prepending leaves every cell value untouched.

text
Raw UTF-8 bytes:     E5 A7 93 E5 90 8D 2C E5 9F 8E E5 B8 82  -> Excel reads ANSI/GBK
With UTF-8 BOM:  EF BB BF E5 A7 93 E5 90 8D 2C E5 9F 8E E5 B8 82  -> Excel renders 姓名,城市

Add UTF-8 BOM & CRLF to your CSV locally

When your CSV is actually legacy GBK, Big5, or Shift-JIS

Sometimes the file isn't UTF-8 at all—it came from an ERP system or bank portal in GB18030, Big5, Shift-JIS, or EUC-KR, and your Mac or Linux script tries to read it as UTF-8.

TxtTo runs strict UTF-8 validation first, then `chardet` byte analysis with locale-aware CJK fallbacks to decode the real bytes cleanly before emitting Excel-ready UTF-8 BOM.

Repair GBK, Big5, Shift-JIS & EUC-KR files

Should you keep BOM for SQL or Unix CLI tools?

While Excel loves UTF-8 BOM, tools like `psql`, `jq`, or naive Bash scripts may treat `0xEF 0xBB 0xBF` as part of the first column header name.

When feeding CSVs back into Linux pipelines or version control, use CRLF to LF with Strip BOM enabled so headers stay clean.

Strip BOM & normalize line endings for Unix

References

Frequently asked questions

Does adding UTF-8 BOM corrupt cell values or formulas?

No. The 3-byte EF BB BF signature sits before the first character of the file so Excel strips it automatically during import while leaving numbers, dates, and text intact.

Why not just use Excel's Data -> From Text/CSV wizard every time?

Fixing the file once with a UTF-8 BOM means anyone on your team can double-click the CSV directly in Explorer or Finder without seeing mojibake.

Are customer names and financial CSVs uploaded during repair?

Never. Encoding detection and byte prepending run 100% in your browser RAM with zero server requests.

Related formats

Related Comparisons

Related Guides