Strip special characters from string, retain, alphabets, numbers and punctuation marks

Question

I am trying to clean a string from all special characters and retain everything else, including punctuation marks.

mystring = "Q18. On a scale from 0 to 10 where 0 means â€˜not at all interested' and 10 means â€˜very interested', how interested are you in helping to address problems that affect poor people in poor countries?"

My effort so far:

newlabel = re.sub('[^A-Za-z0-9]+', ' ', newstring)

Output:

Q18 On a scale from 0 to 10 where 0 means not at all interested and 10 means very interested how interested are you in helping to address problems that affect poor people in poor countries

How can I retain the punctuation marks in the regex I currently have or is there a better solution?

Do you really have that unicode character before `very interested'`? — thefourtheye, Feb 21 '14 at 12:15
Yep. No idea why or how, but its there and I want to get rid of it :) — Boosted_d16, Feb 21 '14 at 12:16

score 4 · Accepted Answer · answered Feb 21 '14 at 12:19

4

Solved,

print (newstring.decode('unicode_escape').encode('ascii','ignore'))

Output:

Q18. On a scale from 0 to 10 where 0 means not at all interested' and 10 means very interested', how interested are you in helping to address problems that affect poor people in poor countries?

answered Feb 21 '14 at 12:19

Boosted_d16

10,018
29
80
133

2

Did you refer something else to come up with that solution? – thefourtheye Feb 21 '14 at 12:21
Once you said they were "unicodes" I did a quick search and found the answer here: http://bit.ly/1nSSmjY – Boosted_d16 Feb 21 '14 at 12:32
Great. I am marking this question as a dup of that question. It will be helpful for the future readers. – thefourtheye Feb 21 '14 at 12:34

score 1 · Answer 2 · answered Feb 21 '14 at 12:20

1

If all you need to change is to retain the dot than adding it to the regex will solve that.

re.sub('[^A-Za-z0-9\.]+', ' ', mystring)

answered Feb 21 '14 at 12:20

Károly Nagy

1,645
10
13

score 0 · Answer 3 · answered Feb 21 '14 at 12:23

0

Just add backslash before each punctuation mark in the regular expression .....

answered Feb 21 '14 at 12:23

Ian Ellis

379
3
8

Strip special characters from string, retain, alphabets, numbers and punctuation marks

3 Answers3