1

I am using google drive JAVA SDK API to download the files below is the code, getGoogleDriveServiceInstance() method will return Drive instance in the below code. This code works perfect for small files but but for files > 500 MB i am getting Out of Memory Error, I need some help how to fix this issue.

public boolean downloadFile(String fileId, java.io.File path) {
         FileOutputStream fos=null;
         try{
             fos = new FileOutputStream(path);
             ByteArrayOutputStream baos = new ByteArrayOutputStream();
             getGoogleDriveServiceInstance().files().get(fileId).executeMediaAndDownloadTo(baos);
             baos.writeTo(fos);
         } catch(Exception ex) {
             ex.printStackTrace();
             return false;
         } finally {
             try{fos.close();fos=null;} catch(Exception ex){}
         }
        return true;
    }

If the file size is big, below error is coming->

Exception in thread "Thread-22" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2271)
        at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
        at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
        at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
        at com.google.api.client.util.ByteStreams.copy(ByteStreams.java:55)
        at com.google.api.client.util.IOUtils.copy(IOUtils.java:94)
        at com.google.api.client.util.IOUtils.copy(IOUtils.java:63)
        at com.google.api.client.googleapis.media.MediaHttpDownloader.executeCurrentRequest(MediaHttpDownloader.java:247)
        at com.google.api.client.googleapis.media.MediaHttpDownloader.download(MediaHttpDownloader.java:199)
        at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeMediaAndDownloadTo(AbstractGoogleClientRequest.java:562)
        at com.google.api.services.drive.Drive$Files$Get.executeMediaAndDownloadTo(Drive.java:3560)
ammu
  • 824
  • 2
  • 13
  • 27
  • 1
    Hint: Have you ever came to think about it... Why do we need streams to begin with? Why not the whole file as a simple object? – Mordechai Jan 19 '17 at 06:57
  • With streams, you can read a little, save on local disk, and repeat. – Mordechai Jan 19 '17 at 06:59
  • 1
    I am not familiar with API but I guess, its not @ammu 's fault as error occurs while calling `executeMediaAndDownloadTo(..)`. There might be size limitations or you can try setting your client memory as needed for larger files. Your `baos` keeps growing till a point that there is no more memory left. Unless API gives some options to read a little and write a little, I think, you are helpless. – Sabir Khan Jan 19 '17 at 07:16
  • 2
    Possible solution as suggested [here](http://stackoverflow.com/questions/27617258/memoryerror-how-to-download-large-file-via-google-drive-sdk-using-python). That code is in python and you should have similar approach in java. – Sabir Khan Jan 19 '17 at 07:18
  • Possible duplicate of [How to deal with "java.lang.OutOfMemoryError: Java heap space" error (64MB heap size)](http://stackoverflow.com/questions/37335/how-to-deal-with-java-lang-outofmemoryerror-java-heap-space-error-64mb-heap) – DaImTo Jan 19 '17 at 07:33

1 Answers1

2

See https://developers.google.com/api-client-library/java/google-api-java-client/media-download which says "When you download a large media file from a server, use resumable media download to download the file chunk by chunk".

pinoyyid
  • 19,003
  • 12
  • 50
  • 100