Create CVPixelBuffer with pixels data, but the final image is distorted

Question

I get pixels by OpenGLES method(glReadPixels) or other way, then create CVPixelBuffer (with or without CGImage) for video recording, but the final picture is distorted. This happens on iPhone6 when I test on iPhone 5c, 5s and 6.

It looks like:

Here is the code:

CGSize viewSize=self.glView.bounds.size;
NSInteger myDataLength = viewSize.width * viewSize.height * 4;

// allocate array and read pixels into it.
GLubyte *buffer = (GLubyte *) malloc(myDataLength);
glReadPixels(0, 0, viewSize.width, viewSize.height, GL_RGBA, GL_UNSIGNED_BYTE, buffer);

// gl renders "upside down" so swap top to bottom into new array.
// there's gotta be a better way, but this works.
GLubyte *buffer2 = (GLubyte *) malloc(myDataLength);
for(int y = 0; y < viewSize.height; y++)
{
    for(int x = 0; x < viewSize.width* 4; x++)
    {
        buffer2[(int)((viewSize.height-1 - y) * viewSize.width * 4 + x)] = buffer[(int)(y * 4 * viewSize.width + x)];
    }
}

free(buffer);

// make data provider with data.
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, buffer2, myDataLength, NULL);

// prep the ingredients
int bitsPerComponent = 8;
int bitsPerPixel = 32;
int bytesPerRow = 4 * viewSize.width;
CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;

// make the cgimage
CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
CGImageRef imageRef = CGImageCreate(viewSize.width , viewSize.height, bitsPerComponent, bitsPerPixel, bytesPerRow, colorSpaceRef, bitmapInfo, provider, NULL, NO, renderingIntent);

//UIImage *photo = [UIImage imageWithCGImage:imageRef];

int width = CGImageGetWidth(imageRef);
int height = CGImageGetHeight(imageRef);

CVPixelBufferRef pixelBuffer = NULL;
CVReturn status = CVPixelBufferPoolCreatePixelBuffer(NULL, _recorder.pixelBufferAdaptor.pixelBufferPool, &pixelBuffer);

NSAssert((status == kCVReturnSuccess && pixelBuffer != NULL), @"create pixel buffer failed.");

CVPixelBufferLockBaseAddress(pixelBuffer, 0);
void *pxdata = CVPixelBufferGetBaseAddress(pixelBuffer);
NSParameterAssert(pxdata != NULL);//CGContextRef
CGContextRef context = CGBitmapContextCreate(pxdata,
                                             width,
                                             height,
                                             CGImageGetBitsPerComponent(imageRef),
                                             CGImageGetBytesPerRow(imageRef),
                                             colorSpaceRef,
                                             kCGImageAlphaPremultipliedLast);
NSParameterAssert(context);

CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

CGColorSpaceRelease(colorSpaceRef);
CGContextRelease(context);
CGImageRelease(imageRef);

free(buffer2);

//CIImage *image = [CIImage imageWithCVPixelBuffer:pixelBuffer];

// ...

CVPixelBufferRelease(pixelBuffer);

I was able to resolve this issue by ensuring that every frame is the exact same dimensions. Turned out some frames were slightly different size which resulted in the distortion. — jamesfzhang, Jun 01 '17 at 22:44

score 2 · Answer 1 · answered May 04 '17 at 13:42

NOTE - this answer relates the overall problem with the image and not to the specific code.

This sort of problem is usually the 'stride' and relates to the memory layout used to hold the image where each row of pixels are not packed tightly together

As an example the source image may be 240 pixels wide. The CMPixelBuffer may allocate 320 pixels for each rows where the first 240 pixels hold the image and the extra 80 pixels are padding. In this case the width is 240 pixels, the stride is 320 pixels.

Strides usually mean you have to copy over each row of pixels one in a loop

Any other tips? Can you explain "stride" and how it relates to the CGContext code? — jamesfzhang, Jun 01 '17 at 21:31

score 1 · Answer 2 · answered Jul 05 '18 at 04:39

1

Use this size everywhere in the code

int width_16 = (int)yourImage.size.width - (int)yourImage.size.width%16; 
int height_ = (int)(yourImage.size.height/yourImage.size.width * width_16) ;
CGSize video_size_ = CGSizeMake(width_16, height_);

answered Jul 05 '18 at 04:39

jjpp

1,272
1
15
30

score 1 · Answer 3 · answered Feb 28 '19 at 17:02

I had the same problem and I think the solution is following: Try to change CGImageGetBytesPerRow(imageRef) to CVPixelBufferGetBytesPerRow(pxbuffer) in CGBitmapContextCreate call. The reason is that your context is backed with the raw data of pixel buffer you have created, not CGImage you are drawing. CVPixelBuffer's bytes per row count may be greater than bytes per pixel * pixel buffer width.

Create CVPixelBuffer with pixels data, but the final image is distorted

3 Answers3