Possible problems with String reversing using charAt method

Question

I saw a comment here that all solutions with charAt are wrong. I could not exactly understand and find something about charAt on internet. As I look the source code it just returns an element from the char array. So my question is that if there any problem or issue about using charAt?

Comment is like that

Strictly speaking, all the solutions based on charAt are wrong, as charAt doesn't give you the "character at", but the "code unit at", and there are code units that are not characters and characters that need multiple code units.

It is referring to the fact that some symbols can't be represented by a single `char` - only ones with a codepoint less than 65536. Characters like Emoji are outside that range. — Andy Turner, Mar 14 '16 at 13:29
Relevant: http://stackoverflow.com/questions/14150530/java-charat-used-with-characters-that-have-two-code-units — Matthijs, Mar 14 '16 at 13:30

Maroun · Answer 1 · 2016-03-14T21:42:54.427

Different characters are encoded with a different numbers of bytes (using UTF-16 scheme). For example, the "A" character is represented as follows:

01000001

So far so good.

But if you have a character like , you'll have a problem. Its UTF-16 representation (BE) is:

11011000 00110101 11011101 00110100

And then charAt can indeed return the second code unit for that character.

See the JDK 7 implementation of String#charAt:

public char charAt(int index) {
    if ((index < 0) || (index >= count)) {
        throw new StringIndexOutOfBoundsException(index);
    }
    return value[index + offset];
}

score 11 · Answer 2 · answered Mar 14 '16 at 13:35

In Java, String is essentially an array of char. Likewise, a char is a UCS-2 (UTF-16) code point.

There are two problems with this:

Not all characters can be expressed with a single code point in UTF-16.
Unicode supports combining characters.

Reordering characters that are part of either of these situations will result in a String that is incorrect.

StringBuilder's reverse takes the first situation into account, but I'm not aware of anything that takes the second into account.

Edwin Buck · Answer 3 · 2016-03-14T13:43:54.937

What is said above is true, some code units require two characters to be represented. As Java uses 16 bit characters, it is encountered infrequently; but strictly speaking, any code that uses charAt(...) without considering if the accessed char is part of a two char code unit is exposing itself to character processing issues.

To test if you are working with a two char code unit, you should check to see if the initial value from the charAt(...) is in the range 0xD800 to 0xDFFF; as that range indicates the start of a two char code unit.

CupawnTae · Answer 4 · 2016-03-14T13:55:17.853

As other answers point out, some characters can take multiple code units, and you will get invalid characters if you try to interpret either of these code units by itself, or in combination with other code units.

One other thing to keep in mind is that having a 2-code-unit character in your string will shift all the subsequent indices by one, so e.g. the tenth character will be charAt(10) instead of charAt(9) - so even if you're not hit by encoding issues with the character itself, you could find yourself extracting the wrong character by index later in the string.

score 5 · Answer 5 · answered Mar 14 '16 at 13:31

5

Strictly speaking, yes there is a problem, as is outlined in the reason you highlighted. The problem is that some characters can need more than 1 char to represent. So by using, String.charAt, when you reverse the string, you'll have a new semi-random character because of the switch in order of the two chars that make up that character.

But again, this is strictly speaking

answered Mar 14 '16 at 13:31

ControlAltDel

28,815
6
42
68

Can you add an example?..I am hearing this for the first time.Please. – Mathews Mathai Mar 14 '16 at 13:33
@MathewsMathai any character in the planes https://en.wikipedia.org/wiki/Plane_(Unicode) i.e. anything > 65535 or 0xFFFF – Peter Lawrey Mar 14 '16 at 13:34
@PeterLawrey Gor it. Thanks. – Mathews Mathai Mar 14 '16 at 13:35
Is there a better method for referring to a character at a particular index then? – Mathews Mathai Mar 14 '16 at 13:36

score 5 · Answer 6 · edited May 23 '17 at 11:53

There are numerous common fatally-broken assumptions about text, especially if you leave the niche of "just one western country", which you do when using unicode.
Just to start some relevant points specifically when dealing with UTF-16:

A codepoint might be multiple codeunits.
A character might be multiple codepoints.
A codepoint might be multiple characters.

Of additional relevance when reversing text are LTR and RTL overrides, which need special handling.

I suggest you read the accepted answer to Why does modern Perl avoid UTF-8 by default?, specifically the section assume brokenness, that part is programming-language-agnostic.

score 3 · Answer 7 · answered Mar 14 '16 at 13:39

The String.charAt method is safe (for some definition of "safe"), but it can be used unsafely, if your string contains characters outside the Basic Multilingual Plane, which has codepoints in the range 0 to 65535.

You can implement string reversal using String.charAt - AbstractStringBuilder uses the char[] directly, but this is logically the same as using String.charAt(). It basically implements two passes:

The first reverses the characters, but also checks for any surrogate pairs
The second re-reverses the surrogate pairs.

score 2 · Answer 8 · answered Mar 14 '16 at 13:39

The simplest example to your question is the case of UTF-8 characters like ñ..

charAt() will easily return the ASCII characters as ASCII characters occupy 1 byte. On the other hand UTF-8 / UTF-16 characters can occupy multiple bytes and therefore you may get an unexpected output.

Many languages have alphabets /symbols in UTF-8 format, so let's say if your application is giving some locale specific information you might be using utf-8 chars and charAt() will fail in that case..

Possible problems with String reversing using charAt method

8 Answers8