JNI Has Two UTF-8s

Three functions with a lie in the name

Every JNI tutorial opens the same way. Java strings go into native code through GetStringUTFChars, and come back out through NewStringUTF. Both say UTF-8. Both are the obvious choice, they are in the first page of the documentation, and they do not do what their names say.

The JVM's internal string representation is UTF-16. The UTF functions were specified to hand you a char *, and rather than convert UTF-16 into real UTF-8, the specification defines a modified UTF-8 with two differences:

An embedded NUL is encoded as two bytes, c0 80, instead of one zero byte, so the result can still be passed to C string functions without being truncated.
Anything outside the Basic Multilingual Plane is encoded as its UTF-16 surrogate pair, with each half written out in full as a three-byte sequence. Six bytes where real UTF-8 has four.

The first difference is invisible in practice. The second one is not. Take an emoji:

U+1F600
  UTF-16:        D83D DE00
  real UTF-8:    F0 9F 98 80                    4 bytes
  modified UTF-8: ED A0 BD ED B8 80              6 bytes

Six bytes, and every one of them is a byte sequence that real UTF-8 forbids. The surrogate code point D83D cannot appear in UTF-8 at all; that is precisely why the standard's designers introduced the four-byte encoding. C0 80 is the overlong encoding of NUL, banned for the same reason. The result is technically malformed UTF-8, and no correct UTF-8 library will accept it.

Where it bites

Our engine is Rust, and Rust has no such thing as a string that is not UTF-8. It is a type-level guarantee, not a convention, and it is the reason a memory-safe language can pass &str across a module boundary without anyone checking. That guarantee is also what turns this into a failure rather than a cosmetic issue.

The boundary does not get &str. It gets a C string pointer, and it converts it with a strict check that returns an error if a single byte is invalid. So the sequence is:

Java hands the runtime a string containing an emoji.
The runtime asks for its UTF-8 form and gets six bytes.
Our C shim copies those bytes, unchanged, into a buffer.
The engine parses the buffer. ED A0 BD is an encoded surrogate, which real UTF-8 forbids, and the parse fails.
The call returns "no value" instead of the value it was given.

Nothing crashed. Nothing logged an encoding error. A user typed an emoji into a field that eventually becomes a persisted key, or an identifier the engine looks up, and the value simply was not there.

There is a second failure in the other direction, and it is worse. NewStringUTF is specified to accept modified UTF-8. Our engine produces real UTF-8. When a string contained a four-byte character and went out through that function, we handed the runtime a byte sequence it had no obligation to understand, and different Android versions reacted differently. Some truncated, some substituted a replacement character, and one threw. That is the difference between a defined conversion and an undefined one.

The shape of the fix

Ten minutes apart on the same afternoon, two commits. The first converted one path, the scene loader, with the transcoder written as a separate file and wired into the build. Ten minutes later the second commit did the rest of the surface: every remaining call site in the JNI layer, both directions, twenty-seven files.

The second commit is the interesting one, because it is where the fix stops being a patch and starts being a rule. Instead of fixing the calls that happened to be wrong, it removed the possibility:

The functions were deleted from the stub header our native tests compile against. Our native test binary links a hand-written jni.h with a function table, and that table no longer has those three slots. Any source file that calls GetStringUTFChars now fails to compile. The error arrives on the author's own machine, in seconds, and it says exactly which identifier does not exist.
The stub also has to be honest about the encoding, not just the arity. So it grew a UTF-16 constructor alongside the UTF-8 one, and the fake runtime implements the real semantics: it stores the UTF-16 units, and the transcoder is the only thing that can read them out.

That second point is the part worth generalising. A test double for an encoding function that silently agrees with the bug is worse than no double at all, because it converts an unknown into a green check. Ours stored the string it was given and handed it straight back, which meant a test could pass while production produced six bytes. Once the double held UTF-16 units, the transcoder had to actually transcode something, and that is when the emoji case became expressible.

The transcoder itself

About a hundred lines, and most of the decision is about what to do with inputs that should not occur:

A supplementary character arrives as two adjacent units. Recombining them is the one thing this code exists to do, and it has to be done from the array, not from a decoded string, because a decoder operating on modified UTF-8 would have to un-mangle first.
An unpaired surrogate is not a character. Rather than guess, it becomes U+FFFD, the replacement character, which is what every other layer in the stack would have done anyway. Being explicit about it means the value is comparable and searchable instead of being a byte pattern that differs per implementation.
An embedded NUL is rejected outright, returning no value. This is the case modified UTF-8 was designed to hide, and our boundary is a C-string boundary, so a string containing one cannot survive the crossing. Failing explicitly is correct: silently truncating at the NUL turns a value into a different value, and the bug would surface much later, somewhere that has no idea strings are involved.
Invalid input on the way out is rejected rather than repaired, including the two classic encodings that look like valid sequences: the overlong NUL and the surrogate range itself. A permissive decoder turns corruption into plausible-looking text, and plausible-looking text is what you cannot debug later.

The shape

This is not really about JNI. It is about a convenience API that is named after a format rather than after what it produces, and about the cost of trusting a name.

When an API is named after an encoding, find out whether it produces that encoding.

GetStringUTFChars produces a thing that people call UTF-8. The c0 80 and the six-byte surrogate are documented, in the specification, and nobody reads the specification for a function they already know. The JVM has been doing this since the beginning of Android, so the functions are correct and the ecosystem is built on them. The only thing wrong is the name, and a name is the only part everybody trusts without checking.

The same shape appears elsewhere. Serialisers that promise "JSON" and accept trailing commas. Compression helpers that return a buffer and do not say who frees it. Database drivers that hand back a string in whatever encoding the connection negotiated. The pattern is always the same: the contract lives in a header comment, the header comment loses to the name, and the bug report arrives as a mystery three layers away from the call.

The second half of the fix is the part that generalises. A stub header with three slots removed costs nothing and prevents the class of error permanently. The transcoder as a separate file with a name means every call site reads the same two function names, and a reviewer can check the call without reading the logic. And a test double that stores the real representation, rather than the convenient one, is what makes any of it testable at all.

Takeaways

GetStringUTFChars does not return UTF-8. It returns modified UTF-8, in which a supplementary character is six bytes and an embedded NUL is two, and both are sequences real UTF-8 forbids. On the way out, NewStringUTF accepts modified UTF-8, so passing it real UTF-8 is undefined behaviour that different Android versions handle differently.

Where a language guarantees its strings are UTF-8, that guarantee is load-bearing. The engine boundary parses strictly, so those six bytes are not a mangled string but a rejected one. A user typed an emoji, the value silently was not there, and nothing crashed.

Make the transcoder a named file with two entry points, one per direction, and point every call site at it. Fixing the calls that happened to be wrong leaves the rest of the surface one review away from the same bug, which is why the two commits for this were ten minutes apart: the first wrote the transcoder, the second swept the layer.

Delete the functions from the header your tests compile against. A compile error is a guard that runs on every machine, every build, forever. A rule in a document is read by whoever opens the document. And make the test double store the real representation, so a fake cannot agree with the bug by being convenient.