Re-writing a CSV file using csv

Question

I opened as CSV file, converted the contents to a numpy array, and then manipulated the arrays. How can I now write this into a new CSV file, 'newtable.csv'?

file_name = "table.csv" 

import numpy as np

reader=csv.reader(open("table.csv","rb"),delimiter=',')
x=list(reader)
result=np.array(x).astype('str') # convert to numpy array

I somehow should use

writer = csv.writer(open('newtable.csv', 'wb'), delimiter=',')

However, I did not change the overall structure of the CSV file; I still use the same sizes of columns and rows. (I only changed a few numbers) Do I have to re-write the entire thing?

EDIT: In the case of a numpy array, how would I re-write the entire thing? The problem with np.savetxt("foo.csv", result, delimiter=",") is that it is not the correct format. One must work with a format specifier somehow.

EDIT2: I should somehow use writerow() but I'm not sure the best way to do this. I have a numpy array shaped (700,12), so that's 12 "columns" of 700 row entries, and the "title" already exists.

with open("new_file.csv", "w") as toWrite:
    writer = csv.writer(toWrite, delimiter=",")
    writer.writerow([.....])

Are you asking "How can I now write this into a new CSV file, 'newtable.csv'?" or are you asking "Do I have to re-write the entire thing?" If the former, this is a duplicate of http://stackoverflow.com/questions/6081008/dump-a-numpy-array-into-a-csv-file. If the latter, the answer is "yes". — Robᵩ, Nov 04 '15 at 19:48
I don't use numpy for csv read/write operations often but it's very simple to use pandas for this kind of thing. Consider using pandas for CSV read/write and tabular data manipulation. — user2962397, Nov 04 '15 at 19:49
Are you manipulating row-by-row of the CSV file, or the columns of the CSV? If row-by-row, you could just process/manipulate each row while reading the CSV, and write to a new file as you are done with each row. `Pandas` is definitely a great way to go - you could convert the 2D `numpy array` to a `DataFrame` and just go `.to_csv()`(see http://stackoverflow.com/questions/21938932/how-do-i-convert-a-numpy-array-into-a-pandas-dataframe for converting to `DataFrame`) — vk1011, Nov 04 '15 at 21:07

vk1011 · Accepted Answer · 2015-11-05T21:51:23.870

Yes, unless you are just appending to the end of the existing file, you will have to re-write the whole file for any changes you made to the contents.

Response after EDIT2:

For writing a 2D numpy array to CSV file, you can do the following.

Let's say the 2D array you manipulated is now called result_edited

with open('new_file.csv', 'wb') as toWrite:
    writer = csv.writer(toWrite) # if the delimiter is ',', you don't need to specify

    for row in result_edited:
        writer.writerow(row)

You were mostly there!

Alternatively, if each row is independent of each other when you manipulate, you can just read and write row-by-row. See below.

file_name_in = 'numeric_table.csv'
file_name_out = 'new_file.csv'

with open(file_name_in, 'rb') as f_in:
    reader = csv.reader(f_in)

    with open(file_name_out, 'wb') as f_out:
        writer = csv.writer(f_out)

        for row in reader:

            # manipulate row, change numbers, etc.

            writer.writerow(row) # Write row to new file

score 0 · Answer 2 · answered Nov 05 '15 at 22:38

Convert the numpy array with result.tolist(). This outputs a Python list. After that, you can write the rows.

with open("new_file.csv", "w") as toWrite:
    writer = csv.writer(toWrite, delimiter=",")
    resultList = result.tolist()
    writer.writerows(resultList)

Re-writing a CSV file using csv

2 Answers2