3

I have a big matrix in MATLAB, which I need to save it in order to use it in python. However, when I save it, MATLAB can't save it except with -v7.3 but that file version can't be read in Python using the following: 

import scipy.io as spio
Data = spio.loadmat('example_file.mat', squeeze_me=True)
A = Data[‘B’]

So it’s required to read that data using h5py, as below:

import h5py 
Data = h5py.File(‘example_file.mat’. ‘r’)
A = Data[‘B’]

In that way of using h5py, the matrix A can be read and shown with an extension of H5 and I can’t see the matrix numbers as the file A resulted from the first way (which is using import scipy.io as spio) .

Is it possible to save a big matrix from MATLAB in similar way to saving a small matrix and then read it in Python using import scipy.io as spio?
Or the other option, is it possible to read a big matrix saved in MATLAB with –v7.3 in Python and read the matrix with same extension using import scipy.io as spio

EDIT

Here is an example of what happens when saving the matrix in format 7.3

In MATLAB, we can generate the matrix and save it in version -v7.3: 

example = randn(16,200) + i*randn(16,200); 
save('example.mat', '-v7.3')

I saved it here in v7.3 because if the matrix is very big, I can't save it except in that version !!

Then, to read that file in python, we must use the h5py as below: 

import h5py as h5 
data_try = hs.File('example.mat', 'r')
A = np.array(data_try )

First, the matrix A is not of size of (16,200) in python!! it's of size (200,16) Second, when I used for example the matrix A as an input of neural network, it shows an error of "can't cast from structure to non-structure, except if the structure only has a single field python". I think that is because the dtype of matrix A in python is something like that [('real','<f8'), ('imag','<f8')]. !!

all that is different when saving the matrix from matlab in version newer than v7.3 and then be able to read it using the import scipy.io as spio which gives us all simply a matrix of dtype : complex

New_student
  • 253
  • 1
  • 6
  • What is the problem using `h5py`? Since MAT is now HDF5, there is no purpose in using MATLAB specific libraries. – Daniel Nov 14 '19 at 11:45
  • A related question: https://stackoverflow.com/questions/874461/read-mat-files-in-python – Daniel Nov 14 '19 at 11:48
  • 1
    @Daniel The problem of using h5py I can't read the files as numpy. – New_student Nov 14 '19 at 11:56
  • That is really odd, with the larger size the data must appear from somewhere. Could you provide an example? Does it happen if you just save x=rand(16,200) to a mat file? – Daniel Nov 14 '19 at 19:14
  • @Daniel I gave an example in the question, and tried explained the issue more. – New_student Nov 15 '19 at 02:01

2 Answers2

1

For one matrix save it into binary file:

clear
SomeData = 1:100*300;
MyMatrix=reshape(SomeData,100,[]);
disp(class(MyMatrix)) % you must know the data type in python. maybe save this info in file name.

MyMatrixBytes=typecast(MyMatrix(:),'uint8');

fid=fopen('z:/MyMatrixBytes.bin','wb');
fwrite(fid,MyMatrixBytes);
fclose(fid);

And in python:

import numpy as np

# dtype: double->float64 , single->float32
MyMatrixBytes = np.fromfile("z:/MyMatrixBytes.bin",dtype='float64')
Mendi Barel
  • 2,375
  • 1
  • 17
  • 15
  • it shows this error: Error using typecast The first input argument must be a full, non-complex numeric value. . . second I need to save it as mat file, ... the bin file can be used in python as an array ? – New_student Nov 16 '19 at 02:19
  • Your array is complex? If so then separate real and imag values and save them. this is a simple matlab procedure. Any in python i gave you one line code to convert the bin file to array – Mendi Barel Nov 16 '19 at 05:18
1

You can save the big matrix in MATLAB using -v7.3 and then to read it in python, process these steps: 

import numpy as np
import h5py as h5 

data_try = h5.File('example.mat', 'r')
A = np.array(data_try) 
A = A.view(np.complex)  or 
A = A.view(np.complex128)

That will enable you to use the resulted output A without that error. Good luck

Zeyad_Zeyad
  • 221
  • 1
  • 6