11

We can either ask AVFoundation to capture a photo in a processed format like BGRA, or ask it to capture a RAW and then process it to BGRA using Core Image. The latter lets us control the parameters used in the process rather than leaving it to the defaults iOS uses.

In my case, I want to turn off noise-reduction, but otherwise have the image similar to BGRA. So I tried:

let rawData = AVCapturePhotoOutput.dngPhotoDataRepresentation(forRawSampleBuffer: self,
                                                                     previewPhotoSampleBuffer: nil)!
let options: [AnyHashable : Any] = [
  kCIInputNoiseReductionAmountKey: 0.0,
  kCIInputLuminanceNoiseReductionAmountKey: 0.0,
  kCIInputColorNoiseReductionAmountKey: 0.0]

let rawFilter = CIFilter(imageData: rawData, options: options)!
let outputImage = rawFilter.outputImage!
let context = CIContext()
let cgImage = context.createCGImage(outputImage, from: outputImage.extent)!
let uiImage = UIImage(cgImage: cgImage)
let data = UIImageJPEGRepresentation(uiImage, jpegQuality)!
// Save the data using PHPhotoLibrary

But this results in a RAW that looks much brighter...

enter image description here

... than the BGRA:

enter image description here

Right-click the photos and open in new tabs to see the difference.

This is just one photo, as an example. I tested for many different scenes, and found that no matter what settings I tried, I couldn't match the quality of the JPEG — it was too bright, or too dark, or had too much contrast, etc.

I then noticed that there's an input boost parameter with a default value of 1 for a full boost, so I set it to 0:

let options: [AnyHashable: Any] = [kCIInputBoostKey: 0.0]

But that produced a dull image without adequate contrast:

enter image description here

So, the question is what settings should I use to have Core Image develop the RAW to BGRA with the same visual look as a BGRA captured by AVFoundation, except without noise reduction.

Kartick Vaddadi
  • 4,265
  • 5
  • 33
  • 45
  • How are you rendering the CIImage? [This sample code](https://developer.apple.com/library/content/samplecode/RawExpose/Listings/RawExposeEmbedded_ImageViewController_swift.html#//apple_ref/doc/uid/TP40017310-RawExposeEmbedded_ImageViewController_swift-DontLinkElementID_8) from Apple uses a CIContext with kCIContextWorkingFormat : Int(kCIFormatRGBAh). Not sure if that might be part of the problem. – Rob MacEachern Apr 12 '18 at 23:01
  • There are 7 keys related to Noise, https://github.com/apple/swift-3-api-guidelines-review/blob/64e3132a6a383b4a4603605180ded31efd37dcdc/Platforms/OSX/CoreImage/CIRAWFilter.swift. I would try all of them – Tarun Lalwani Apr 13 '18 at 05:30
  • @TarunLalwani Noise-reduction isn't the focus here. It's brightness, contrast, sharpness, etc. – Kartick Vaddadi Apr 14 '18 at 02:36
  • @RobMacEachern I updated the question to show the code I'm using to render the CIImage. I don't think the working format should be the problem, because we're talking about things like brightness and contrast. If we get the working format wrong, it usually fails to render, rather than producing photos that look mediocre. The default working format is anyway a 32-bit float per channel, which is plenty of dynamic range and precision for this purpose. – Kartick Vaddadi Apr 14 '18 at 02:46

0 Answers0