Remove zero-width space characters from a JavaScript string

Question

I take user-input (JS code) and execute (process) them in realtime to show some output.

Sometimes the code has those zero-width spaces; it's really weird. I don't know how the users are inputting that. Example: "($".length === 3

I need to be able to remove that character from my code in JS. How do I do so? or maybe there's some other way to execute that JS code so that the browser doesn't take the zero-width space characters into account?

How did you infer that there is a zero-width character? From the length alone? The length of a non-BMP character is 2. — Jukka K. Korpela, Jul 03 '12 at 08:23
when i go to the end of the string ,and hit the left arrow, at one point, it doesnt moves to the left until hitting the left arrow key twice. that's how i infered. — user1437328, Jul 03 '12 at 08:29
Then you need to analyze the characters e.g. by writing out the numeric codes. The data may contain combining marks so that two or more characters are treated as a unit when moving to the left. — Jukka K. Korpela, Jul 03 '12 at 15:26

score 127 · Accepted Answer · answered Jul 03 '12 at 06:58

127

Unicode has the following zero-width characters:

U+200B zero width space
U+200C zero width non-joiner Unicode code point
U+200D zero width joiner Unicode code point
U+FEFF zero width no-break space Unicode code point

To remove them from a string in JavaScript, you can use a simple regular expression:

var userInput = 'a\u200Bb\u200Cc\u200Dd\uFEFFe';
console.log(userInput.length); // 9
var result = userInput.replace(/[\u200B-\u200D\uFEFF]/g, '');
console.log(result.length); // 5

Note that there are many more symbols that may not be visible. Some of ASCII’s control characters, for example.

answered Jul 03 '12 at 06:58

Mathias Bynens

130,201
49
208
240

4

[@Iván Castellanos mentioned](https://twitter.com/ivanca/status/354252316435689472) some other characters that may be considered for this: U+200E LEFT-TO-RIGHT MARK and U+200F RIGHT-TO-LEFT MARK. As I said, there may be other symbols that are not strictly visible by themselves. – Mathias Bynens Jul 08 '13 at 16:38
How do we detect if these values actually exist on the page after the DOM loads? Thanks! – klewis Jul 18 '14 at 12:41
1

`var HTMLe=document.getElementsByTagName('html')[0]; HTMLe.outerHTML = HTMLe.outerHTML.replace(/[\u200B-\u200D\uFEFF\u200E\u200F]/g, '');` ... that's how – May 16 '17 at 16:20
This does not work if the string is a string of unicode chars - it will give error just to create a var with them `var s = "\ud83d\ude0d\ud83d\ude0d\ud83d\ude0d\ud83d\ude0d\ud83d\ude0d‌\ud83d\ude0d\ud83d\u‌de0d\ud83d\ude0d\ud8‌3d\ude0d\ud83d\ude18‌\ud83d\ude18\ud83d\u‌de18"` – mplungjan Dec 05 '17 at 14:03

Technotronic · Answer 2 · 2015-09-21T11:38:41.293

9

I had a problem some invisible characters were corrupting my JSON and causing Unexpected Token ILLEGAL exception which was crashing my site.

Here is my solution using RegExp variable:

    var re = new RegExp("\u2028|\u2029");
    var result = text.replace(re, '');

More about Javascript and zero width spaces you can find here: Zero Width Spaces

edited Sep 21 '15 at 11:38

answered Oct 19 '14 at 13:50

Technotronic

7,011
3
35
51

The or symbol would probably be slower (in IE) because it is optimised for multi-character matches. But, with google's V8, who knows, it probably runs just as fast. – Jack Giffin May 17 '17 at 23:11
These invisible zero-width unicode characters can be used to hide metadata credentials for those users who dare Copy and Paste through a browser to another editor that knows to receive the message and convert the zero width metadata back to the absence of characters. So what happens is you copy and paste the word "hi" and what gets transmitted is the `h`, then the string of metadata credentials, then the `i`. But the source and destination just show the word `hi`. It's going to be a struggle to keep these zero width barbarians and their persian messengers out at the spartan moat. Sad! – Eric Leschinski Jul 30 '18 at 19:15

score 3 · Answer 3 · answered Jul 03 '12 at 06:54

3

[].filter.call( str, function( c ) {
    return c.charCodeAt( 0 ) !== 8203;
} );

Filter each character to remove the 8203 char code (zero-width space unicode number).

answered Jul 03 '12 at 06:54

Florian Margaine

50,873
14
87
110

This is a clever solution, using modern JavaScript it could be reduced to this one-liner: `[].filter.call(strVal, c => c.charCodeAt() !== 8203).join('')` – Grant Humphries Jan 11 '19 at 21:52

score 3 · Answer 4 · answered Jul 26 '16 at 12:50

3

str.replace(/\u200B/g,'');

200B is the hexadecimal of the zero width space 8203. replace this with empty string to remove this

answered Jul 26 '16 at 12:50

Tarek Salah uddin Mahmud

876
7
18

Remove zero-width space characters from a JavaScript string

4 Answers4

Linked