23

I have a C#/.NET utility I wrote that loads PNG images from disk

Bitmap b = Bitmap.FromStream(new MemoryStream(File.ReadAllBytes(filename))) as Bitmap;

performs several transformations on them (rotation, scaling, alpha) and then saves the resulting PNG images back to disk with different file names based on the transformations applied

b.Save(outputName, ImageFormat.Png);

I've successfully written thousands of PNGs using the utility. However, occasionally one of the PNGs fails to load in a separate program which uses libpng. In that program, libpng gives the error "Too many IDATs found"

Looking into the PNG file reveals a 'rogue' IDAT chunk at the end of the file just before the IEND chunk. One such IDAT chunk (and the following IEND chunk) looks like this in a hex editor. These are the final 24 bytes in the file.

IDAT: 0x00 0x00 0xFF 0xF4 0x49 0x44 0x41 0x54 0x35 0xAF 0x06 0x1E    
IEND: 0x00 0x00 0x00 0x00 0x49 0x45 0x4e 0x44 0xAE 0x42 0x60 0x82

The IDAT chunk length is shown as 0xFFF4. However, as is obvious, there aren't that many bytes in the IDAT chunk (or even the file for that matter.)

Has anyone else come across this problem? I can fix the problem in one of several ways. I can hand edit the PNG file to remove that last IDAT chunk (or set its size to 0.) I can run a secondary program which fixes broken PNGs. However, I'd like a C#/.NET solution which I can easily add to my original program. Ideally, I'd like a solution which doesn't require me to re-open the PNG as a binary file; check for the bad IDAT chunk; and re-write the PNG. However, I'm beginning to think that's what I'll need to do.

David Welch
  • 341
  • 1
  • 7
  • Do your images vary in size/row stride? Have you been able to narrow down if there is something unique (width, height, stride, format) to the images that are broken? – Chris Oct 10 '13 at 17:13
  • 1
    I think you're going to have to isolate the problem more. If you use C# to immediately read back each file you save, can you repro the error? If so, you need to track down the precise set of circumstances that led to that broken image and go from there. – Kirk Woll Oct 10 '13 at 17:14
  • @Chris The image sizes and row strides are all over the spectrum. They are all Format32bppArgb. I've found nothing unique about the failing PNG. – David Welch Oct 10 '13 at 17:31
  • If you rerun your code with the same inputs, do you get the same results? Same set of broken files? – Chris Oct 10 '13 at 17:35
  • @KirkWoll I added code in my program to re-load (Image.FromStream) the bad PNG just after it is save to disk. I then immediately re-save (Image.Save) it to disk with a different filename. The re-saved version still has the rogue IDAT chunk. – David Welch Oct 10 '13 at 17:35
  • @Chris Yes, same inputs, same results. The one output PNG that is bad comes from an input PNG - let's call it X.png. X.png is a valid PNG. I take X.png and apply 73 different rotations to it; saving each rotated version to a separate file (X-rrr.png where rrr is the rotation in degrees.) 72 of those rotated versions are fine. One is bad. That happens to be 195 degrees of rotation, but I don't that that matters. – David Welch Oct 10 '13 at 17:43
  • 1
    It seems other users have stumbled onto the [same problem](http://osdir.com/ml/graphics.png.devel/2007-11/msg00003.html). Some code seem to recover from the problem and some don't (like libpng). Still doesn't explain where that extra chunk comes from though. – Chris Oct 10 '13 at 17:43
  • And you can find nothing that sets that specific image apart from the others? Must be some bug in the encoder, very strange. – Chris Oct 10 '13 at 17:46
  • @Chris Thanks for the input and for finding that libpng reference. Though libpng gives an error (or is it a warning) GIMP will successfully load the PNG with the rogue IDAT. It still spits the message out to the console. However, some apps which use libpng treat such an error/warning as a error and refuse to load the PNG instead of just skipping the bad IDAT. I do believe there is some error in the encoder used by .NET System.Drawing namespace. I have tried mucking around with the version of Save that takes encoder parameters, but with no luck. – David Welch Oct 10 '13 at 18:41
  • System.Drawing.Bitmap uses GDI+, so this much be a major bug in windows (unless it's actually allowed by the png spec). One option would be to use a 3rd party encoder, google says there are a few around (freeimage.net, etc). – Chris Oct 10 '13 at 18:54
  • If you can reproduce it, you should fill a bug report. – leonbloy Oct 11 '13 at 16:41
  • I have had a similar problem with this files below – 06needhamt Oct 16 '13 at 15:53
  • Original File https://docs.google.com/file/d/0B4RkktioBqoYNXUwZ0hvRWFqUXM/edit?usp=sharing – 06needhamt Oct 16 '13 at 15:53
  • Bad File https://docs.google.com/file/d/0B4RkktioBqoYYjV1MVhKUTZIckk/edit?usp=sharing – 06needhamt Oct 16 '13 at 15:54
  • 1
    I understand this is 4 years old, but am working on root causing this now. Am curious what OS you were on at the time, if you remember. Given the timeframe, presume this was Windows 7/Server 2012? – Rafael Rivera Mar 01 '18 at 03:43
  • Yes, the OS at the time was Windows 7 64-bit. – David Welch Mar 05 '18 at 14:36
  • 2
    I do not have an answer to your question, but in addressing the exact same issue we found a way to **reproduce** it. We figured out that the size of the resulting PNG-file is a key indicator of the problem being present. If the size of the file (in bytes) is 0x1001C + n * 0x10000 with n 0, 1, 2, 3, 4 (and probably larger values, but I cannot confirm that), the problem is consistently present. I posted a question with some code to reproduce the issue: https://stackoverflow.com/questions/52100703/bug-in-windows-nets-system-drawing-savestream-imageformat-corrupt-png-pro I hope this helps. – Duurt Aug 30 '18 at 20:55
  • What .NET framework is it on (aka name and version - .NET Framework, net core, mono, with version) as well as Windows version? Have you built a collection of these bad behaving images? This seems like Bitmap.Save() has a bug, or, more likely GDI+ PNG encoder that is responsible for transforming your bitmap to PNG, which is Windows dependent, including it's version and installed service packs. Though it is likely problem doesn't often surface. Also can you share some image for repro? – aiodintsov Oct 03 '18 at 20:01

2 Answers2

2

Old question.

.NET is notoriously poor at handling images. The codecs are old win32 ones with many bugs.

.NET does not always free up the OS resources used when reading/writing image files even if you follow the recommended dispose and/or using methods.

JohnT
  • 71
  • 4
0

This isn't really a complete answer since I'm unable to reproduce the image. But you could try loading the image using another one of the Bitmap constructors:

http://msdn.microsoft.com/en-us/library/0cbhe98f.aspx

Which would change your code into:

Bitmap b = new Bitmap(filename);

Because that constructor uses some native GDI+ functions to load the image into memory, instead of you reading the raw bytes:

http://dotnetframework.org/default.aspx/4@0/4@0/DEVDIV_TFS/Dev10/Releases/RTMRel/ndp/fx/src/CommonUI/System/Drawing/Bitmap@cs/1305376/Bitmap@cs

There might be a difference, even though the problem seems to be when writing the image to disk.

Swen Kooij
  • 855
  • 1
  • 7
  • 18
  • Thanks for the input. However, I've tried the other Bitmap constructors with no luck. I specifically chose the FromStream version because others can leave a file handle open longer than I want. – David Welch Oct 28 '13 at 20:25