14

I downloaded a dataset which is stored in .h5 files. I need to keep only certain columns and to be able to manipulate the data in it.

To do this, I tried to load it in a pandas dataframe. I've tried to use:

pd.read_hdf(path)

But I get: No dataset in HDF5 file.

I've found answers on SO (read HDF5 file to pandas DataFrame with conditions) but I don't need conditions, and the answer adds conditions about how the file was written but I'm not the creator of the file so I can't do anything about that.

I've also tried using h5py: 

df = h5py.File(path)

But this is not easily manipulable and I can't seem to get the columns out of it (only the names of the columns using df.keys()) Any idea on how to do this ?

Community
  • 1
  • 1
Graham Slick
  • 5,766
  • 9
  • 43
  • 71

2 Answers2

6

Easiest way to read them into Pandas is to convert into h5py, then np.array, and then into DataFrame. It would look something like: 

df = pd.DataFrame(np.array(h5py.File(path)['variable_1']))
Ivan Mitevski
  • 121
  • 2
  • 2
5

Pandas HDF support needs the HDF file to be formated very specifically. You can see https://stackoverflow.com/a/33644128/4128030 for more info.

Community
  • 1
  • 1
drj
  • 126
  • 1
  • 8