iOS: Why my big files are not converted with NSData(contentsOfFile: options:)? Error Domain=NSCocaErrorDomain Code=256

Question

I try to convert a video with NSData, it works well with little videos or 100mb, but my big files (4.44Gb) are not sent...

   var video_data: NSData?
    do {
        video_data = try NSData(contentsOfFile: (videoPath), options: NSData.ReadingOptions.alwaysMapped)
    } catch let error as NSError {
        video_data = nil
        return
    }

How can I put big files in NSData?

Error Domain=NSCocoaErrorDomain Code=256 "Impossible d’ouvrir le fichier « D9C7DABF-4BE3-4105-8D76-AA92B1D1502E_video.notsend »." UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/EAE9B4C4-BE6B-490C-BEE7-381B2DF27CC9/Library/LEADS/D9C7DABF-4BE3-4105-8D76-AA92B1D1502E_video.notsend, NSUnderlyingError=0x283be1380 {Error Domain=NSPOSIXErrorDomain Code=12 "Cannot allocate memory"}}

Any ideas ?

Thanks in advance.

EDIT 1: PARAMETERS TO SEND:
Here is the entire function. I need all that parameters to send to my server. I need to send the eventId, the contactId, the type, and the file in a Data value. The problem is that I have an error, I don't know how to put a 4.44Go file in a Data with InputStream.

 func uploadVideo(_ videoPath: String, fileName: String, eventId: Int, contactId: Int, type: Int, callback: @escaping (_ data:Data?, _ resp:HTTPURLResponse?, _ error:NSError?) -> Void)
    {
        var video_data: Data
        video_data = self.getNextChunk(urlOfFile: NSURL(string: videoPath)! as URL)!

        let WSURL:String =  "https://" + "renauldsqffssfd3.sqdfs.fr/qsdf"

        let requestURLString = "\(WSURL)/qsdfqsf/qsdf/sdfqs/dqsfsdf/"
        let url = URL(string: requestURLString)
        let request = NSMutableURLRequest(url: url!)
        request.httpMethod = "POST"

        let boundary = generateBoundaryString()
        request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
        request.setValue("Keep-Alive", forHTTPHeaderField: "Connection")

        let body = NSMutableData()
        let mimetype = "video/mp4"

        //define the data post parameter
        body.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
        body.append("Content-Disposition:form-data; name=\"eventId\"\r\n\r\n".data(using: String.Encoding.utf8)!)
        body.append("\(eventId)\r\n".data(using: String.Encoding.utf8)!)

        body.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
        body.append("Content-Disposition:form-data; name=\"contactId\"\r\n\r\n".data(using: String.Encoding.utf8)!)
        body.append("\(contactId)\r\n".data(using: String.Encoding.utf8)!)

        body.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
        body.append("Content-Disposition:form-data; name=\"type\"\r\n\r\n".data(using: String.Encoding.utf8)!)
        body.append("\(type)\r\n".data(using: String.Encoding.utf8)!)

        body.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
        body.append("Content-Disposition:form-data; name=\"file\"; filename=\"\(fileName)\"\r\n".data(using: String.Encoding.utf8)!)
        body.append("Content-Type: \(mimetype)\r\n\r\n".data(using: String.Encoding.utf8)!)
        body.append(video_data)
        body.append("\r\n".data(using: String.Encoding.utf8)!)

        body.append("--\(boundary)--\r\n".data(using: String.Encoding.utf8)!)

        request.httpBody = body as Data

        let configuration = URLSessionConfiguration.default
        let session = URLSession(configuration: configuration, delegate: self, delegateQueue: OperationQueue.main)

        let task = session.uploadTask(with: request as URLRequest, from: body as Data) { loc, resp, err in
            if (resp != nil)
            {
                let status = (resp as! HTTPURLResponse).statusCode
            }
            callback(loc, resp as? HTTPURLResponse, err as NSError?)
        }

        task.resume()
}

  public func getNextChunk(urlOfFile: URL) -> Data?{
        if inputStream == nil {
            inputStream = InputStream(url: urlOfFile)!
            inputStream!.open()
        }
        var buffer = [UInt8](repeating: 0, count: 1024*1024)
        let len = inputStream!.read(&buffer, maxLength: 1024*1024)
        if len == 0 {
            return nil
        }
        return Data(buffer)
    }

EDIT 2: COMPLEMENT TO THE SOLUTION:

The Rob solution above is perfect. I just added a control of the space on the disk to alert if the temporary file cannot copy, to delete it if incomplete, and finally advice the problem to the user.
Indeed, without that control the app will try to send to the server the file even if the file is incomplete...

  func sizeOfFileAtPath(path: String) -> UInt64
        {
            var fileSize : UInt64

            do {
                //return [FileAttributeKey : Any]
                let attr = try FileManager.default.attributesOfItem(atPath: path)
                fileSize = attr[FileAttributeKey.size] as! UInt64

                //if you convert to NSDictionary, you can get file size old way as well.
                let dict = attr as NSDictionary
                fileSize = dict.fileSize()
                            return fileSize

            } catch {
                print("Error: \(error)")
            }

            return 0
        }

    private func buildPayloadFile(videoFileURL: URL, boundary: String, fileName: String, eventId: Int, contactId: Int, type: Int) throws -> URL {
        let mimetype = "video/mp4"

        let payloadFileURL = URL(fileURLWithPath: NSTemporaryDirectory())
            .appendingPathComponent(UUID().uuidString)

        guard let stream = OutputStream(url: payloadFileURL, append: false) else {
            throw UploadError.unableToOpenPayload(payloadFileURL)
        }

        stream.open()

        //define the data post parameter
        stream.write("--\(boundary)\r\n")
        stream.write("Content-Disposition:form-data; name=\"eventId\"\r\n\r\n")
        stream.write("\(eventId)\r\n")

        stream.write("--\(boundary)\r\n")
        stream.write("Content-Disposition:form-data; name=\"contactId\"\r\n\r\n")
        stream.write("\(contactId)\r\n")

        stream.write("--\(boundary)\r\n")
        stream.write("Content-Disposition:form-data; name=\"type\"\r\n\r\n")
        stream.write("\(type)\r\n")

        stream.write("--\(boundary)\r\n")
        stream.write("Content-Disposition:form-data; name=\"file\"; filename=\"\(fileName)\"\r\n")
        stream.write("Content-Type: \(mimetype)\r\n\r\n")
        if stream.append(contentsOf: videoFileURL) < 0 {
            throw UploadError.unableToOpenVideo(videoFileURL)
        }
        stream.write("\r\n")

        stream.write("--\(boundary)--\r\n")
        stream.close()

/*-------BEGIN ADDITION TO THE CODE---------*/
        //check the size
        let temporaryFileSize = self.sizeOfFileAtPath(path: payloadFileURL.relativePath)
        let originalFileSize = self.sizeOfFileAtPath(path: videoFileURL.relativePath)

        if (temporaryFileSize < originalFileSize || temporaryFileSize == 0)
        {
            let alert = UIAlertView()
            alert.title = "Alert"
            alert.message = "There is not enough space on the disk."
            alert.addButton(withTitle: "Ok")
            alert.show()

            do {
                try FileManager.default.removeItem(at: payloadFileURL)
            } catch let error as NSError {
                print("Error: \(error.domain)")
            }
        }  
/*-------END ADDITION TO THE CODE---------*/

        return payloadFileURL
    }

What are you doing with this `video_data`? If you tell us what you are trying to do, we can probably advise ways to do this without loading it into `NSData`. E.g. if uploading to web service, use file-based upload task, etc. — Rob, Dec 13 '18 at 17:39
@Rob I edited my question to put what I do. I don't really know how to use the InputStream, I don't find so much things on the web... — ΩlostA, Dec 14 '18 at 10:13
See my answer below, which shows you how to do what you want, without loading the asset into a `Data`/`NSData`. Forgive the unrelated changes (excising `NSError` in favor of `Error`, excising the use of `NSHTTPURLResponse` in favor of `URLResponse`, the functional decomposition, etc.). — Rob, Dec 14 '18 at 17:49

Rob · Accepted Answer · 2021-03-15T15:07:45.740

When dealing with assets as large as that, you want to avoid using Data (and NSData) entirely. So:

read the video using an InputStream;
write the body of the request to another file using OutputStream; and
upload that payload as a file rather than setting the httpBody of the request; and
make sure to clean up afterwards, removing that temporary payload file.

All of this avoids ever loading the whole asset into memory at one time and your peak memory usage will be far lower than it would have been if you use Data. This also ensures that this is unlikely to ever fail due to a lack of RAM.

func uploadVideo(_ videoPath: String, fileName: String, eventId: Int, contactId: Int, type: Int, callback: @escaping (_ data: Data?, _ resp: HTTPURLResponse?, _ error: Error?) -> Void) {
    let videoFileURL = URL(fileURLWithPath: videoPath)
    let boundary = generateBoundaryString()
    
    // build the request
    
    let request = buildRequest(boundary: boundary)
    
    // build the payload
    
    let payloadFileURL: URL
    
    do {
        payloadFileURL = try buildPayloadFile(videoFileURL: videoFileURL, boundary: boundary, fileName: fileName, eventId: eventId, contactId: contactId, type: type)
    } catch {
        callback(nil, nil, error)
        return
    }
    
    // perform the upload
    
    performUpload(request, payload: payloadFileURL, callback: callback)
}

enum UploadError: Error {
    case unableToOpenPayload(URL)
    case unableToOpenVideo(URL)
}

private func buildPayloadFile(videoFileURL: URL, boundary: String, fileName: String, eventId: Int, contactId: Int, type: Int) throws -> URL {
    let mimetype = "video/mp4"

    let payloadFileURL = URL(fileURLWithPath: NSTemporaryDirectory())
        .appendingPathComponent(UUID().uuidString)
    
    guard let stream = OutputStream(url: payloadFileURL, append: false) else {
        throw UploadError.unableToOpenPayload(payloadFileURL)
    }
    
    stream.open()
    
    //define the data post parameter
    stream.write("--\(boundary)\r\n")
    stream.write("Content-Disposition:form-data; name=\"eventId\"\r\n\r\n")
    stream.write("\(eventId)\r\n")
    
    stream.write("--\(boundary)\r\n")
    stream.write("Content-Disposition:form-data; name=\"contactId\"\r\n\r\n")
    stream.write("\(contactId)\r\n")
    
    stream.write("--\(boundary)\r\n")
    stream.write("Content-Disposition:form-data; name=\"type\"\r\n\r\n")
    stream.write("\(type)\r\n")
    
    stream.write("--\(boundary)\r\n")
    stream.write("Content-Disposition:form-data; name=\"file\"; filename=\"\(fileName)\"\r\n")
    stream.write("Content-Type: \(mimetype)\r\n\r\n")
    if stream.append(contentsOf: videoFileURL) < 0 {
        throw UploadError.unableToOpenVideo(videoFileURL)
    }
    stream.write("\r\n")
    
    stream.write("--\(boundary)--\r\n")
    stream.close()
    
    return payloadFileURL
}

private func buildRequest(boundary: String) -> URLRequest {
    let WSURL = "https://" + "renauldsqffssfd3.sqdfs.fr/qsdf"
    
    let requestURLString = "\(WSURL)/qsdfqsf/qsdf/sdfqs/dqsfsdf/"
    let url = URL(string: requestURLString)!
    var request = URLRequest(url: url)
    request.httpMethod = "POST"
    
    request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
    request.setValue("Keep-Alive", forHTTPHeaderField: "Connection")
    
    return request
}

private func performUpload(_ request: URLRequest, payload: URL, callback: @escaping (_ data: Data?, _ resp: HTTPURLResponse?, _ error: Error?) -> Void) {         
    let task = session.uploadTask(with: request, fromFile: payload) { data, response, error in
        try? FileManager.default.removeItem(at: payload) // clean up after yourself
        
        if let response = response as? HTTPURLResponse {
            let status = response.statusCode
        }
        
        callback(data, response as? HTTPURLResponse, error)
    }
    
    task.resume()
}

By the way, uploading this as a file also has the virtue that you can consider using a background URLSessionConfiguration at some future date (i.e. the upload of a 4 gb video is likely to take so long that the user might not be inclined to leave the app running and let the upload finish; background sessions let the upload finish even if your app is no longer running; but background uploads require file-based tasks, not relying on the httpBody of the request).

That's a whole different issue, beyond the scope here, but hopefully the above illustrates the key issue here, namely don't use NSData/Data when dealing with assets that are this large.

Please note, the above uses the following extension to OutputStream, including method to write strings to output streams and to append the contents of another file to the stream:

extension OutputStream {
    @discardableResult
    func write(_ string: String) -> Int {
        guard let data = string.data(using: .utf8) else { return -1 }
        return data.withUnsafeBytes { (buffer: UnsafePointer<UInt8>) -> Int in
            write(buffer, maxLength: data.count)
        }
    }
    
    @discardableResult
    func append(contentsOf url: URL) -> Int {
        guard let inputStream = InputStream(url: url) else { return -1 }
        inputStream.open()
        let bufferSize = 1_024 * 1_024
        var buffer = [UInt8](repeating: 0, count: bufferSize)
        var bytes = 0
        var totalBytes = 0
        repeat {
            bytes = inputStream.read(&buffer, maxLength: bufferSize)
            if bytes > 0 {
                write(buffer, maxLength: bytes)
                totalBytes += bytes
            }
        } while bytes > 0
        
        inputStream.close()

        return bytes < 0 ? bytes : totalBytes
    }
}

Hi, I have problems to download file more than 3Go. Very strange. I have a code status 500 Internal server error... Should I have to download it in background ? Is there any question of activity ? — ΩlostA, Dec 18 '18 at 12:43
500 suggests that your server is failing, not that the request is failing. Maybe a hard limit there. Maybe a configuration setting. Maybe a poor implementation server side... — Rob, Dec 18 '18 at 14:15
@ Rob It is okay, it works. It is not the above code, it is because of a problem of empty space of the device. I had to add a space checking in the size of the temporary file correspond to the original file to be sure that it will be well checked by the server. I will add it in your code if it doesn't disturb you. In my app I will add this to show the empty space in real time: https://stackoverflow.com/a/29417935/3581620 — ΩlostA, Dec 18 '18 at 16:42
I’d suggest putting your space checking code as a separate answer. You are allowed to [answer your own question](https://stackoverflow.com/help/self-answer). — Rob, Dec 18 '18 at 16:54

score 3 · Answer 2 · answered Dec 13 '18 at 17:04

3

According to Apple documentation, you can use NSData(contentsOf:options:) to "read short files synchronously", so it's not supposed to be able to handle a 4 GB file. Instead you could use InputStream and initialize it with the URL with your file path.

answered Dec 13 '18 at 17:04

johnyu

2,152
1
13
33

I tried to do it, but with no success... I edited my post. – ΩlostA Dec 14 '18 at 10:12

Grigory S Shushakov · Answer 3 · 2018-12-13T21:42:09.277

1

In the catch area you have a error object, this is your answer.

UPD: I supposed this error, and right cause is Code=12 "Cannot allocate memory"

You can try to split like - Is calling read:maxLength: once for every NSStreamEventHasBytesAvailable correct?

edited Dec 13 '18 at 21:42

answered Dec 13 '18 at 16:52

Grigory S Shushakov

61
5

Thanks, I added the message, but I don't find the solution – ΩlostA Dec 13 '18 at 16:56

iOS: Why my big files are not converted with NSData(contentsOfFile: options:)? Error Domain=NSCocaErrorDomain Code=256

3 Answers3

Linked