Stack Overflow
Stack Overflow

Questions tagged [genfromtxt]

Numpy function to create arrays from tabular data.

References

268 questions
470
votes
12 answers

How do I read CSV data into a record array in NumPy?

I wonder if there is a direct way to import the contents of a CSV file into a record array, much in the way that R's read.table(), read.delim(), and read.csv() family imports data to R's data frame? Or is the best way to use csv.reader() and then…
hatmatrix
  • 36,897
  • 38
  • 126
  • 217
37
votes
1 answer

numpy.genfromtxt produces array of what looks like tuples, not a 2D array—why?

I'm running genfromtxt like below: date_conv = lambda x: str(x).replace(":", "/") time_conv = lambda x: str(x) a = np.genfromtxt(input.txt, delimiter=',', skip_header=4, usecols=[0, 1] + radii_indices, converters={0: date_conv, 1:…
robintw
  • 24,689
  • 48
  • 125
  • 196
35
votes
4 answers

Using numpy.genfromtxt to read a csv file with strings containing commas

I am trying to read in a csv file with numpy.genfromtxt but some of the fields are strings which contain commas. The strings are in quotes, but numpy is not recognizing the quotes as defining a single string. For example, with the data in…
CraigO
  • 515
  • 1
  • 5
  • 6
23
votes
4 answers

Python: Convert string (in scientific notation) to float

I'm trying to import a large .csv file containing text and numbers using genfromtxt in numpy. I'm only interested in two columns. I have most of the import sorted out with: def importfile(root): data = root.entry.get() atw =…
Dr. Toboggan
  • 345
  • 1
  • 4
  • 9
20
votes
4 answers

Read in all csv files from a directory using Python

I hope this is not trivial but I am wondering the following: If I have a specific folder with n csv files, how could I iteratively read all of them, one at a time, and perform some calculations on their values? For a single file, for example, I do…
FaCoffee
  • 6,303
  • 19
  • 76
  • 148
19
votes
1 answer

Reading data into numpy array from text file

I have a file with some metadata, and then some actual data consisting of 2 columns with headings. Do I need to separate the two types of data before using genfromtxt in numpy? Or can I somehow split the data maybe? What about placing the file…
Nirvan
  • 307
  • 1
  • 2
  • 7
14
votes
8 answers

"Got 1 columns instead of ..." error in numpy

I'm working on the following code for performing Random Forest Classification on train and test sets; from sklearn.ensemble import RandomForestClassifier from numpy import genfromtxt, savetxt def main(): dataset =…
user3466132
  • 239
  • 1
  • 4
  • 10
11
votes
3 answers

NumPy dtype issues in genfromtxt(), reads string in as bytestring

I want to read in a standard-ascii csv file into numpy, which consists of floats and…
user2489252
8
votes
1 answer

NumPy genfromtxt: using filling_missing correctly

I am attempting to process data saved to CSV that may have missing values in an unknown number of columns (up to around 30). I am attempting to set those missing values to '0' using genfromtxt's filling_missing argument. Here is a minimal working…
Thav
  • 355
  • 4
  • 6
7
votes
3 answers

NumPy: mismatch in size of old and new data-descriptor

I ran into the following problem with NumPy 1.10.2 when reading a CSV file. I cannot figure out how to give explicit datatypes to genfromtxt. Here is the CSV, minimal.csv: x,y 1,hello 2,hello 3,jello 4,jelly 5,belly Here I try to read it with…
Akseli Palén
  • 24,406
  • 8
  • 56
  • 68
7
votes
2 answers

numpy genfromtxt/pandas read_csv; ignore commas within quote marks

Consider a file, a.dat, with contents: address 1, address 2, address 3, num1, num2, num3 address 1, address 2, address 3, 1.0, 2.0, 3 address 1, address 2, "address 3, address4", 1.0, 2.0, 3 I am trying to import with numpy.genfromtxt. However the…
atomh33ls
  • 23,172
  • 18
  • 91
  • 149
7
votes
1 answer

numpy.genfromtxt: Ambiguous delimiters?

I'm trying to write a generic script, part of which imports files that are either comma-separated or white-space-separated. I'd like the script to recognize either type. Is there a way to specify something like arrayobj = np.genfromtxt(file.txt,…
PikalaxALT
  • 315
  • 2
  • 11
6
votes
2 answers

How to read columns of varying length from a text file in NumPy using genfromtxt()?

I have hundreds of text files like these, with each column separated by three spaces. The data is for a year: 12 months and 31 days for each month. Below, I'm only showing below what's relevant to question: 001 DIST - ADILABAD ANDHRA …
user3707588
  • 63
  • 1
  • 5
6
votes
3 answers

unable to read a tab delimited file into a numpy 2-D array

I am quite new to nympy and I am trying to read a tab(\t) delimited text file into an numpy array matrix using the following code: train_data = np.genfromtxt('training.txt', dtype=None, delimiter='\t') File contents: 38 Private 215646 …
Abhi
  • 153
  • 1
  • 1
  • 8
5
votes
2 answers

Using genfromtxt to import csv data with missing values in numpy

I have a csv file that looks something like this (actual file has many more columns and rows): 1,2,3,4,5 6,7,8,9,10 11,12,13,14,15 16 Say the name of the file is info.csv If I try to import this using data = numpy.genfromtxt('info.csv', delimiter…
Curious2learn
  • 26,027
  • 37
  • 97
  • 121
1
2 3
…
17 18