How to remove quote characters around array when loading a csv in Python?

Question

I am having trouble removing quote characters that appear around my arrays. When I read in my file like this:

data = pd.read_csv('filepath.csv', sep='|', index_col=0, nrows=5)

the dtype of my problematic column is object but the individual entries are strings:

print(type(data.body_tokens[0]))
data.body_tokens[0]

<class 'str'>
"['he', 'knows', 'what', 'he', 's', 'doing']"

How can I remove the quotation marks around the array?

You just see quotation marks because you're looking at the repr of a string. The real issue is, do you want this to be a string, or do you expect this to be a list? Because it's a string in your CSV as well, just that you don't see the quotes there because, well, it's not a Python repr. If you do `print(data.body_tokens[0])` instead you'll get it without quotes; but is that really useful to you? — deceze, Mar 23 '20 at 08:54
The fundamental problem here is that you are trying to use the csv format for something it wasn't designed to do. — juanpa.arrivillaga, Mar 23 '20 at 08:57
Does this answer your question? [Convert string representation of list to list](https://stackoverflow.com/questions/1894269/convert-string-representation-of-list-to-list) — Dishin H Goyani, Mar 23 '20 at 09:11

score 0 · Accepted Answer · edited Mar 23 '20 at 08:52

0

import ast

string = "['he', 'knows', 'what', 'he', 's', 'doing']"

list = ast.literal_eval(string)

type(list)    #list

print(list)   #['he', 'knows', 'what', 'he', 's', 'doing']

want this one?

edited Mar 23 '20 at 08:52

deceze

471,072
76
664
811

answered Mar 23 '20 at 08:47

JY Won

78
5

How to remove quote characters around array when loading a csv in Python?

1 Answers1