Background upload of large amount of data

Question

I need to upload large number of photos to a server from my iOS device. For example, 500 photos. How should I do this right?

I created upload task with background session configuration with NSURLSession for each photo. I tried making my own custom queue where each next task would launch after completion of previous one. But at one moment new task wasn't starting. I guess, because it all was happening in the background. More on this issue, you can read here

(If approach with a queue was right, would you please advise a good realisation of a queue for async tasks, because I may messed up something in my implementation)

So after article linked above, my guess was I should start all upload tasks at once (not one after another). But I have efficiency concern. If I would create a task for each photo, it would 500 background async task and about 1 gigabyte of a data uploading parallel. I guess, it would cause some problem with network.

To sum up all said above, which is the right way to upload large piece of data in background in iOS (500 photos in my case)?

Answer 1

Unfortunately, Apple's APIs are terrible at this task because of bad design decisions. There are a couple of major obstacles that you face:

• There are limits to how many simultaneous tasks you can create. I think performance starts to break down at somewhere on the order of 100 tasks.
• There are limits to how often the OS will wake your app up. The more often it wakes your app, the longer it will wait before waking it again. At some point, this will result in not being able to schedule new tasks.
• Uploads don't continue from where they left off in the event of a failure; they restart. This can result in huge bandwidth costs on a bad network.

I suspect that the best approach is to:

• Chunk the requests into several large groups, each of which can be written to a single ZIP archive that is not so big that the user runs out of disk space, but not so small that it uploads too quickly.
• Write the first set of those files into a single file (e.g. in ZIP format).
• Use a custom script on the server side that lets you resume uploads from where it left off by adding extra CGI parameters.
• On failure, ask the server how much data it got, then truncate the front of the file and reupload from the current spot.
• On success, compute how quickly the first large file finished uploading, and if it is not O(minutes), combine the next few sets. Write the set/sets to a file, and start the next request.

With the caveat that all of this must be done fairly quickly. You may find it necessary to pre-combine the files into ZIP archives ahead of time to avoid getting killed. But do not be tempted to combine them into a single file, because then you'll take too long when truncating the head on retry. (Ostensibly, you could also provide any parameters as part of the URL, and make the POST body be raw data, and provide a file stream to read from the ZIP archive starting at an offset.)

If you're not banging your head against a wall already, you soon will be. :-)

Answer 2

For downloading this much amount of images you may need to ask the user not to stop the application or putting in background mode.

Because in that two cases we won't be able to perform this much large task to get done.

If the user phone is in active state,

Create a NSoperation corresponding to each upload process. In your case it may be around 500.

And add thus NSOperations into a queue called NSOperationQueue and just start the NSOperationQueue tasks.

It will perform one by one.

For more details like caching and all.. please follow the SO POST

Here is the Swift Version