AWS SDK - Implementing a network queue for CloudFront downloads

Question

I'm currently working on an iOS project that utilises the AWS SDK to download large media files to the device. I am using CloudFront to distribute the content and the downloads are working well, however I am having problems implementing a network queue for these operations. No matter what I try, all the files want to download at once.

I am using the AWSContent downloadWithDownloadType: method to initiate and monitor progress on the actual downloads.

I have tried using an NSOperationQueue and setting setMaxConcurrentOperationCount, and all the code blocks execute at once. :(

I have a feeling it might be configurable with AWSServiceConfiguration in the AppDelegate, but the documentation is extremely vague on what variables you can pass into that object... http://docs.aws.amazon.com/AWSiOSSDK/latest/Classes/AWSServiceConfiguration.html

Has anyone had any experience with this?

TIA

Answer 1

Your problem is most likely that you misunderstand an approach of asynchronous operations.

I have tried using an NSOperationQueue and setting setMaxConcurrentOperationCount, and all the code blocks execute at once. :(

It's difficult to say what's definitely wrong without seeing an actual code, however most likely it's tied to the following steps:

1. You create NSOperationQueue
2. You set maxConcurrentOperationsCount to 2 for example
3. You add 4 blocks to it with AWSContent downloadWithDownloadType:
4. You expect no more 2 downloads to be run simultaneously

What do you probably do wrong

The key is inside point 3. What exactly the block does? My guess is that it completes before actual download completes. So if you have something like:

NSOperationQueue *queue = [NSOperationQueue new];
queue.maxConcurrentOperationsCount = 2;
for (AWSContent *content in contentArray) { // Assume you already do have this array
    [queue addOperationWithBlock:^() {
        [content downloadWithDownloadType:AWSContentDownloadTypeIfNotCached
                         pinOnCompletion:YES
                           progressBlock:nil
                       completionHandler:^(AWSContent *content, NSData *data, NSError *error) {
            // do some stuff here on completion
        }];
    }];
}

Your block exits before your download is finished, allowing next blocks to run on queue and starting further downloads.

What to try

You should simply add some synchronization mechanism to your block to let operation complete only on completion block. Say:

NSOperationQueue *queue = [NSOperationQueue new];
queue.maxConcurrentOperationsCount = 2;
for (AWSContent *content in contentArray) { // Assume you already do have this array
    [queue addOperationWithBlock:^() {
        dispatch_semaphore_t dsema = dispatch_semaphore_create(0);
        [content downloadWithDownloadType:AWSContentDownloadTypeIfNotCached
                         pinOnCompletion:YES
                           progressBlock:nil
                       completionHandler:^(AWSContent *content, NSData *data, NSError *error) {
            // do some stuff here on completion
            // ...
            dispatch_semaphore_signal(dsema); // it's important to call this function in both error and success cases of download to free the block from queue
        }];
        dispatch_semaphore_wait(dsema, DISPATCH_TIME_FOREVER); // or another dispatch_time if you want your custom timeout instead of AWS
    }];
}

Effectively your answer is https://stackoverflow.com/a/4326754/2392973

You just schedule plenty of such blocks to your operation queue.

More reading

https://developer.apple.com/reference/dispatch