send large files over socket java

Question

I am trying to transfer larger files over socket.I will be transferring the file in chunks.As shown in the code.link

int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0)
{
  out.write(buffer, 0, count);
}

before sending this file,i want to send an object which holds the details of the file.Using which stream should i send Object + File.

I am new to streams,can i get any sample code.

Can i send the byte length of the object first to read the object ,save it and send file data.is it possible,any sample code ?

Thanks 421

Answer 1

I'm not sure whether using Java's serialization mechanism is the best way to do a simple file transfer. As your question suggest, you try to avoid keeping the whole file in memory at any time. This can be done with objects using the Proxy pattern but if all you want to do is transfer the file, this might not be the most straight-forward solution. (Also, it will effectively tie your peer to be implemented in Java too.)

Instead, why not take a look at an extremely successful protocol that does exactly what you need: HTTP.

Content-Type: application/octet-stream
Content-Length: 542183

542183 bytes of data follow...

It should not be too hard for you to write a parser for the meta-data header.

Answer 2

You need to ensure the order of writing/reading. If write an object -> write raw bytes on client, then read an object -> read raw bytes on server. When reading, ObjectInputStream should be able to find the boundary of the serialized object data.

If you want to keep a socket connection long-live and use its streams multiple times, wrapping socket's Output/InputStream in a ObjectOutput/InputStream is not a good idea IMO. When you close a object stream, it closes the underlying stream as well.

So you may want to write the length of serialized object data first (file length is contained in the object so you don't need to write it explictly), e.g. 4 bytes of BigEndian encoded int. Then serialize the object into a ByteArrayOutputStream, and write the bytes in its buffer. On the server, read 4 bytes first, decode the bytes back to an int, and read that many bytes into a byte[], wrap the byte array with a ByteArrayInputStream and deserialize the object from it.

Write like this:

......
OutputStream out = socket.getOutputStream();

ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);  

oos.writeObject(fileInfoObject);
oos.close();

byte[] header = encodeInteger(baos.size());

out.write(header, 0, 4);
baos.writeTo(out);

// write the file to out just as your question shows

On the receiving side: ...... InputStream in = socket.getInputStream();

// read the int
byte[] header = new byte[4];

in.read(header, 0, 4);

int size = decodeInteger(header);

// read the object
byte[] objectbuf = new byte[size];

int count;

while((count += in.read(objectbuf)) < size); // not sure if this works...

ObjectInputStram ois = new ObjectInputStream(new ByteArrayInputStream(objectbuf));

Object fileInfoObject = ois.readObject();
ois.close();

// read the file
FileOutputStream fos = new FileOutputStream(new File("somefile"));

byte[] buffer = new byte[8192];
count = 0;
long left = castedFileInfoObject.fileSize;

// also not sure if this works, not tested.
int maxRead = buffer.length;

while (true) {
    count = in.read(buffer, 0, maxRead);

    left -= count;

    if (left < 8192) {
        maxRead = (int)left;
    }

    fos.write(buffer, 0, count);

    if (left == 0) {
        break;
    }
}

I haven't tested the sample code in my answer.. just to show the idea.

Answer 3

Class MyClass should implement the Serializable interface. Then, an object of this class can be written to an ObjectOutputStream and read back from ObjectInputStream using writeObject and readObject methods (See below).

On Client:

Socket socket = new Socket(url, port);  
OutputStream os = socket.getOutputStream();  
ObjectOutputStream oos = new ObjectOutputStream(os);  
MyClass obj = new Myclass();  
oos.writeObject(obj);  
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0) {
  out.write(buffer, 0, count);
}

On server:

ServerSocket sSocket = new ServerSocket(port);
Socket socket = sSocket.accept();  
InputStream is = socket.getInputStream();  
ObjectInputStream ois = new ObjectInputStream(is);  
MyClass obj = (MyClass)ois.readObject();
byte arr[];
try {
  while(arr = (byte[])ois.readObject()) {
    //do something with arr
  }
} catch(java.io.EOFException) {
  // End of data
}

If you need to send more data after the file is finished, you need a way to figure out the number of bytes the file consists of. Then, you can send the number of bytes beforehand over the socket to the server. On the server, read only that many bytes of information for the file and then do the same for the rest of the data you are going to send. This strategy of pre-sending the file size is recommended and is mostly used while doing any data transfer. If you can do that, you don't have to rely on catching java.io.EOFException to detect end of data.