How to import and read a shelve or Numpy file in Google Colaboratory?

Question

I have file.npy and I want to load it in Google Colaboratory Notebook. I already know that I must load the file from Google Drive, however I have no idea how to do so.

Any help is welcome

Answer 1

Upload your file into Colaboratory Notebook with the following:

from google.colab import files
uploaded = files.upload()

Then you can reach the contents of your file from uploaded object and then write it into a file:

with open("my_data.h5", 'w') as f:
    f.write(uploaded[uploaded.keys()[0]])

If you run:

!ls

you will see my_data.h5 file in the current directory.

This is the method that worked for me. Hope it helps.

Answer 2

Actually, you can directly upload and download local files.

There are examples of local file upload/download as well as Drive file upload / download in the I/O example notebook

The first cell shows local file upload:

from google.colab import files
uploaded = files.upload()
for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn])))

Answer 3

Uploading files and folders contain subfolders and files(images), Colab google:
Please, try this function to upload files and folders to Colab google:

from google.colab import files
import zipfile, io, os

    def read_dir_file(case_f):  # case_f = 0 for uploading one File and case_f = 1 for uploading one Zipped Directory
        uploaded = files.upload()    # to upload a Full Directory, please Zip it first (use WinZip)
        for fn in uploaded.keys():
            name = fn  #.encode('utf-8')
            #print('\nfile after encode', name)
            #name = io.BytesIO(uploaded[name])
        if case_f == 0:    # case of uploading 'One File only'
            print('\n file name: ', name)
            return name
        else:   # case of uploading a directory and its subdirectories and files
            zfile = zipfile.ZipFile(name, 'r')   # unzip the directory 
            zfile.extractall()
            for d in zfile.namelist():   # d = directory
                print('\n main directory name: ', d)
                return d
    print('Done!')

1- To upload One File:

fileName = read_dir_file(0)

If the file you are going to upload is a .csv file then:

import pandas as pd
df = pd.read_csv(fileName)
df.head()

You can read any file that has different formats by using the same manner.

2- To upload a Full Directory that has subdirectories and files: first zip the directory by using one zip and use:

dirName = read_dir_file(1)

Then, you can work with (dirName) as the root directory, as an example, if it has 3 subdirectories, say, (training, validation and test):

train_data_dir = dirName + 'training'  
validation_data_dir = dirName + 'validation'  
test_data_dir = dirName + 'test' 

That is it! Enjoy!