When the output stream is not a terminal, sccache passes the compiler's stdout and stderr through strip-ansi-escapes before writing them. That crate parses with vte, whose Perform::execute forwards only '\n'. Bytes 0x80-0x9f are treated as C1 control codes and dropped; 0xa0-0xc1 and 0xf5-0xff have no ground-state entry and are dropped; '\r' is dropped too. Only bytes that form valid UTF-8 survive.
Reproduces on Linux with gcc, no special locale needed:
printf '#warning caf\xe9 na\xefve\nint f(void){ return 1; }\n' > lat1.c
gcc -c -Wall lat1.c -o /dev/null 2> bare.txt
sccache gcc -c -Wall lat1.c -o /dev/null 2> sccache.txt
cmp bare.txt sccache.txtgcc echoes the raw 0xe9 and 0xef from the source into its diagnostic. bare.txt is 94 bytes containing those two bytes; sccache.txt is 98 bytes with each replaced by U+FFFD. Redirecting is what matters - on a terminal sccache writes the bytes through unchanged, so this only shows up in piped or logged builds.
Where it actually hurts is Windows on a codepage that is not UTF-8. MSVC emits /showIncludes as multi-byte text, and ninja's deps = msvc finds header dependencies by matching msvc_deps_prefix against that output. With the prefix mangled ninja records no dependencies at all, so touching a header rebuilds nothing. Measured on a 3-file CMake project, Japanese Windows, CUDA/MSVC 2022:
| launcher |
ninja -t deps |
touch the header, then ninja |
| none |
#deps 1 → ../a.h |
rebuilds |
| sccache |
#deps 0 |
"no work to do" |
Silent and inverted: a cache hit leaves the dependency graph empty, so a rebuild that was owed does not happen.
A fix - a byte-level scanner that only looks for a literal ESC - is on https://github.com/beru/sccache/tree/msvc-nonascii-output-passthrough
🤖 Generated with Claude Code
When the output stream is not a terminal, sccache passes the compiler's stdout and stderr through strip-ansi-escapes before writing them. That crate parses with vte, whose Perform::execute forwards only '\n'. Bytes 0x80-0x9f are treated as C1 control codes and dropped; 0xa0-0xc1 and 0xf5-0xff have no ground-state entry and are dropped; '\r' is dropped too. Only bytes that form valid UTF-8 survive.
Reproduces on Linux with gcc, no special locale needed:
gcc echoes the raw 0xe9 and 0xef from the source into its diagnostic. bare.txt is 94 bytes containing those two bytes; sccache.txt is 98 bytes with each replaced by U+FFFD. Redirecting is what matters - on a terminal sccache writes the bytes through unchanged, so this only shows up in piped or logged builds.
Where it actually hurts is Windows on a codepage that is not UTF-8. MSVC emits /showIncludes as multi-byte text, and ninja's
deps = msvcfinds header dependencies by matching msvc_deps_prefix against that output. With the prefix mangled ninja records no dependencies at all, so touching a header rebuilds nothing. Measured on a 3-file CMake project, Japanese Windows, CUDA/MSVC 2022:ninja -t deps#deps 1→../a.h#deps 0Silent and inverted: a cache hit leaves the dependency graph empty, so a rebuild that was owed does not happen.
A fix - a byte-level scanner that only looks for a literal ESC - is on https://github.com/beru/sccache/tree/msvc-nonascii-output-passthrough
🤖 Generated with Claude Code