3

Question:

I've made a homepage for my brother, accessible here:
http://www.daniel-steiger.ch

It uses Microsoft ASP.NET MVC3 on Linux with mono 3, over fastcgi with nginx (plus my own DNS server).

Now I know this is an unusual constellation, but so far, it all works fine.
However, I ran into a minor very subtle bug.

When in the gallery one clicks on a thumbnail image, I wanted to display the full-sized image via the FullImage method of the gallery controller in a new tab.
For example, this direct url:
http://www.daniel-steiger.ch/gallery/FullImage/001.jpg


In internet Explorer, I got the image as text instead.
In all other browsers displayed an "invalid image" message.
I solved the problem by calling the image via it's direct file URL, which works fine:
http://www.daniel-steiger.ch/Content/images/gallery/001.jpg?LastWriteTimeUTC=1358694795000

Subsequently, I reported the bug to the mono mailing list
http://mono.1490590.n4.nabble.com/Bug-in-mono-3-0-1-MVC3-File-FileResult-td4658382.html

Now I got as a response, that this is all my fault, because I set the wrong image mime type, which is/was true.
However, I found it strange that if that was the case, the same code works fine on Windows, plus smart browsers like Chrome usually detect a wrong mime setting and use the correct one.

So I changed the mime-type from "image/jpg" to "image/jpeg", and redeployed the project to the server.
I also checked with the file utility if the image is actually a jpeg image, and it is.

What is strange is that it still doesn't show the image.
On internet explorer, I now get "not available".
On all other browsers, I get: the image cannot be displayed because it contains errors.

I did now wget the image from the URL where the image contains errors.
wget http://www.daniel-steiger.ch/gallery/fullimage/001.jpg

Then I ran a binary compare between the invalid and the original file:

cmp -l 001.jpg 001a.jpg   | awk '{printf "%08X %02X %02X\n", $1, strtonum(0$2), strtonum(0$3)}' >> comparison.txt

And this is the comparison result:

BinDiff What springs into my eye, is that the image that internet explorer says he cannot find is actually 1.7 MB in size, and contains the extra bytes:

31 39 36 62 36 38 0D 0A 

at the beginning...
Anybody has an idea what's going wrong here/ where these bytes could come from (apart from the fact that it's most likely from a bug in mono/fastcgi) ?

This is the new controller code btw:

namespace Homepage.Controllers
{


    public class GalleryController : Controller
    {


        protected static string GetImageDirectory()
        {
            string bd = AppDomain.CurrentDomain.BaseDirectory;
            string strImageDirectory = System.IO.Path.Combine(bd,
            "Content");
            strImageDirectory =
            System.IO.Path.Combine(strImageDirectory, "images");
            strImageDirectory =
            System.IO.Path.Combine(strImageDirectory, "gallery");

            return strImageDirectory;
        } // End Function GetImageDirectory


        protected static string strImageDirectory = GetImageDirectory();


        public FileResult FullImage(string id)
        {
            string strFileName =
            System.IO.Path.Combine(strImageDirectory, id);

            //return new FilePathResult("CorrectFullPathAndFileName", "CorrectMime");
            //return File(strFileName, "image/jpg"); // Old
            return File(strFileName, "image/jpeg"); // New
        } // End Action FullImage



        public FileResult Thumb(string id)
        {
            //return Redirect(id);

            string strFileName =
            System.IO.Path.Combine(strImageDirectory, id);

            System.IO.Stream ms =
            Tools.Imaging.GetThumbnailStream(strFileName,
            System.Drawing.Imaging.ImageFormat.Png);
            return File(ms, "image/png");
            /*
            using (System.IO.Stream ms =
            Tools.Imaging.GetThumbnailStream(strFileName,
            System.Drawing.Imaging.ImageFormat.Png))
            {
                return File(ms, "image/png");
            }*/
        } // End Action Thumb


    } // End Class GalleryController : Controller


} // End Namespace Homepage.Controllers

Edit:
it gets stranger:
There is also an extra sequence with

0d 0a 0d 0a 30 0d 0a 0d  0a 

at the end...

I just did the hexdump, and found the original file has filesize (hold your breath):

00196b68

Here the hexdumps (8 MB each):

Canonical format (hexdump -C 001.jpg > 001.txt):
Original file: http://www.daniel-steiger.ch/001.txt
Botched file: http://www.daniel-steiger.ch/001a.txt

Pure dump (hexdump 001.jpg > 001_1.txt):
Original file: http://www.daniel-steiger.ch/001_1.txt
Botched file: http://www.daniel-steiger.ch/001a_1.txt

Hmmm, the hex dump for the botched files is 5 MB, the one for the original is 8.2 MB...

I use the latest stable nginx:

sudo -s
nginx=stable # use nginx=development for latest development version
add-apt-repository ppa:nginx/$nginx
apt-get update 
apt-get install nginx

The full code for Tools.Imaging (renamed Tools to MvcTools in the meantime, to match the assembly namespace)

using System;
using System.Text;


namespace MvcTools
{


    public class Imaging
    {

        //public static System.Drawing.Size m_sMaxThumbNailDimensions = new System.Drawing.Size(200, 200);
        public static System.Drawing.Size m_sMaxThumbNailDimensions = new System.Drawing.Size(300, 300);

        public static System.IO.Stream GetImageAsStream(
                                                 string strOrgFileName
                                                , System.Drawing.Imaging.ImageFormat ifOutputFormat
                                           )
        {
            return GetImageAsStream(strOrgFileName, ifOutputFormat, 1024);
        } // End Function GetImage


        public static System.IO.Stream GetImageAsStream(
                                                 string strOrgFileName
                                                ,System.Drawing.Imaging.ImageFormat ifOutputFormat
                                                ,int rez
                                           )
        {
            System.IO.MemoryStream ms = null;

            if (!System.IO.File.Exists(strOrgFileName))
                throw new System.IO.FileNotFoundException(strOrgFileName);

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {

                    ms = new System.IO.MemoryStream();
                    imgSourceImage.Save(ms, ifOutputFormat);
                    ms.Position = 0;
                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                System.Windows.Forms.MessageBox.Show(ex.GetType().ToString());
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

            return ms;
        } // End Function GetImageAsStream


        public static System.Drawing.Size GetThumbnailSize(string strOrgFileName)
        {
            System.Drawing.Size sThumbNailSizeToUse = new System.Drawing.Size();

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {
                    decimal decPixToSubstract = 0;
                    decimal decPercentage;

                    if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    {
                        if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Width - (decimal)m_sMaxThumbNailDimensions.Width) / (decimal)imgSourceImage.Size.Width);
                            decPixToSubstract = decPercentage * imgSourceImage.Size.Height;
                            sThumbNailSizeToUse.Width = m_sMaxThumbNailDimensions.Width;
                            sThumbNailSizeToUse.Height = imgSourceImage.Size.Height - (int)decPixToSubstract;
                        } // End if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        else
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Height - (decimal)m_sMaxThumbNailDimensions.Height) / (decimal)imgSourceImage.Size.Height);
                            decPixToSubstract = decPercentage * (decimal)imgSourceImage.Size.Width;
                            sThumbNailSizeToUse.Height = m_sMaxThumbNailDimensions.Height;
                            sThumbNailSizeToUse.Width = imgSourceImage.Size.Width - (int)decPixToSubstract;
                        } // End else of if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)

                    } // End if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    else
                    {
                        sThumbNailSizeToUse.Width = imgSourceImage.Size.Width;
                        sThumbNailSizeToUse.Height = imgSourceImage.Size.Height;
                    } // End else of if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)

                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

            return sThumbNailSizeToUse;
        } // End Sub GetThumbnailSize(string strOrgFileName)


        // http://stackoverflow.com/questions/7319842/mvc3-razor-thumbnail-resize-image-ideas
        // http://stackoverflow.com/questions/1528525/alternatives-to-system-drawing-for-use-with-asp-net/1528908#1528908
        public static void GenerateThumbnailFile(
                                        string strPhysicalPath,
                                        string strOrgFileName, string strThumbnailFileName,
                                        System.Drawing.Imaging.ImageFormat ifOutputFormat, int rez
                                     )
        {

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {
                    //System.Drawing.Image oImg = System.Drawing.Image.FromStream(fil.InputStream);

                    decimal decPixToSubstract = 0;
                    decimal decPercentage;

                    //default
                    System.Drawing.Size sThumbNailSizeToUse = new System.Drawing.Size();
                    if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    {
                        if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Width - (decimal)m_sMaxThumbNailDimensions.Width) / (decimal)imgSourceImage.Size.Width);
                            decPixToSubstract = decPercentage * imgSourceImage.Size.Height;
                            sThumbNailSizeToUse.Width = m_sMaxThumbNailDimensions.Width;
                            sThumbNailSizeToUse.Height = imgSourceImage.Size.Height - (int)decPixToSubstract;
                        } // End if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        else
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Height - (decimal)m_sMaxThumbNailDimensions.Height) / (decimal)imgSourceImage.Size.Height);
                            decPixToSubstract = decPercentage * (decimal)imgSourceImage.Size.Width;
                            sThumbNailSizeToUse.Height = m_sMaxThumbNailDimensions.Height;
                            sThumbNailSizeToUse.Width = imgSourceImage.Size.Width - (int)decPixToSubstract;
                        } // End else of if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)

                    } // End if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    else
                    {
                        sThumbNailSizeToUse.Width = imgSourceImage.Size.Width;
                        sThumbNailSizeToUse.Height = imgSourceImage.Size.Height;
                    } // End else of if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)

                    using (System.Drawing.Bitmap bmpThumbnail = new System.Drawing.Bitmap(sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height))
                    {
                        bmpThumbnail.SetResolution(rez, rez);
                        using (System.Drawing.Image imgThumbNail = bmpThumbnail)
                        {

                            using (System.Drawing.Graphics gGraphicsContext = System.Drawing.Graphics.FromImage(imgThumbNail))
                            {
                                gGraphicsContext.CompositingQuality = System.Drawing.Drawing2D.CompositingQuality.HighQuality;
                                gGraphicsContext.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
                                gGraphicsContext.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;

                                System.Drawing.Rectangle rThumbnailDimension = new System.Drawing.Rectangle(0, 0, sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height);
                                gGraphicsContext.DrawImage(imgSourceImage, rThumbnailDimension);
                            } // End Using gGraphicsContext

                            imgThumbNail.Save(System.IO.Path.Combine(strPhysicalPath, strThumbnailFileName), ifOutputFormat);
                        } // End Using imgThumbNail

                    } // End Using bmpThumbnail

                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

        } // End Function GenerateThumbNail


        public static System.IO.Stream GetThumbnailStream(
                                                               string strOrgFileName
                                                             , System.Drawing.Imaging.ImageFormat ifOutputFormat
                                                         )
        {
            return GetThumbnailStream(strOrgFileName, ifOutputFormat, 1024);
        } // End Function GetThumbnailStream


        public static System.IO.Stream GetThumbnailStream(
                                      string strOrgFileName
                                     ,System.Drawing.Imaging.ImageFormat ifOutputFormat
                                     ,int rez
                                  )
        {

            System.IO.MemoryStream ms = null;

            try
            {

                using (System.Drawing.Image imgSourceImage = System.Drawing.Image.FromFile(strOrgFileName))
                {
                    decimal decPixToSubstract = 0;
                    decimal decPercentage;

                    System.Drawing.Size sThumbNailSizeToUse = new System.Drawing.Size();
                    if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    {
                        if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Width - (decimal)m_sMaxThumbNailDimensions.Width) / (decimal)imgSourceImage.Size.Width);
                            decPixToSubstract = decPercentage * imgSourceImage.Size.Height;
                            sThumbNailSizeToUse.Width = m_sMaxThumbNailDimensions.Width;
                            sThumbNailSizeToUse.Height = imgSourceImage.Size.Height - (int)decPixToSubstract;
                        } // End if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)
                        else
                        {
                            decPercentage = (((decimal)imgSourceImage.Size.Height - (decimal)m_sMaxThumbNailDimensions.Height) / (decimal)imgSourceImage.Size.Height);
                            decPixToSubstract = decPercentage * (decimal)imgSourceImage.Size.Width;
                            sThumbNailSizeToUse.Height = m_sMaxThumbNailDimensions.Height;
                            sThumbNailSizeToUse.Width = imgSourceImage.Size.Width - (int)decPixToSubstract;
                        } // End else of if (imgSourceImage.Size.Width > imgSourceImage.Size.Height)

                    } // End if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)
                    else
                    {
                        sThumbNailSizeToUse.Width = imgSourceImage.Size.Width;
                        sThumbNailSizeToUse.Height = imgSourceImage.Size.Height;
                    } // End else of if (m_sMaxThumbNailDimensions.Width < imgSourceImage.Size.Width || m_sMaxThumbNailDimensions.Height < imgSourceImage.Size.Height)


                    using (System.Drawing.Bitmap bmpThumbnail = new System.Drawing.Bitmap(sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height))
                    {
                        bmpThumbnail.SetResolution(rez, rez);
                        using (System.Drawing.Image imgThumbNail = bmpThumbnail)
                        {

                            using (System.Drawing.Graphics gGraphicsContext = System.Drawing.Graphics.FromImage(imgThumbNail))
                            {
                                gGraphicsContext.CompositingQuality = System.Drawing.Drawing2D.CompositingQuality.HighQuality;
                                gGraphicsContext.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
                                gGraphicsContext.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;

                                System.Drawing.Rectangle rThumbnailDimension = new System.Drawing.Rectangle(0, 0, sThumbNailSizeToUse.Width, sThumbNailSizeToUse.Height);
                                gGraphicsContext.DrawImage(imgSourceImage, rThumbnailDimension);

                                ms = new System.IO.MemoryStream();
                                imgThumbNail.Save(ms, ifOutputFormat);
                                ms.Position = 0;
                            } // End Using gGraphicsContext

                        } // End Using imgThumbNail

                    } // End Using bmpThumbnail


                    /*
                    byte[] buffer = null;
                    using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
                    {
                        imgThumbNail.Save(ms, ifOutputFormat);
                        buffer = ms.ToArray();
                    }
                    */

                    // Exerts from Page_Load method 
                    //Response.ContentType = "image/" + extension;
                    //Response.OutputStream.Write(pBuffer, 0, pBuffer.Length);
                    //Response.End();

                    //imgThumbNail.Save(System.IO.Path.Combine(strPhysicalPath, strThumbnailFileName), ifOutputFormat);
                } // End Using imgSourceImage

            } // End Try
            catch (Exception ex)
            {
                //Console.WriteLine(ex.Message);
                System.Windows.Forms.MessageBox.Show(ex.Message);
                //Response.Write(ex.Message);
            } // End Catch

            //System.Windows.Forms.MessageBox.Show("image/" + ifOutputFormat.ToString().ToLower());
            return ms;
        } // End Function GenerateThumbNail


    } // End Class Imaging


} // End Namespace Tools
Stefan Steiger
  • 68,404
  • 63
  • 337
  • 408

3 Answers3

4

This seems to be a bug involving Chuncked transfer encoding in

Class: System.Web.HttpResponse (or one of its dependencies)
Method: TransmitFile(string filename)

Edit:
There is this code in the constructor:

if (worker_request != null)
      use_chunked = (worker_request.GetHttpVersion () == "HTTP/1.1");

Patched it to check for CGI (if CGI, the server handles the file transfer, so there may be no chuncked encoding given back from the FastCGI server as per RFC 3875.

        internal HttpResponse (HttpWorkerRequest worker_request, HttpContext context) : this ()
        {
            WorkerRequest = worker_request;
            this.context = context;

#if !TARGET_J2EE
            if (worker_request != null)
            {

                if(worker_request.GetHttpVersion () == "HTTP/1.1")
                {
                    string GatewayIface = context.Request.ServerVariables["GATEWAY_INTERFACE"];
                    use_chunked = (GatewayIface == null || !GatewayIface.StartsWith("CGI"));
                }
                else
                    use_chunked = false;

            }
#endif
            writer = new HttpWriter (this);
        }

Added patch to https://bugzilla.xamarin.com/show_bug.cgi?id=10001
Fixed in mono 3.2.3

Stefan Steiger
  • 68,404
  • 63
  • 337
  • 408
  • 1
    Chunked encoding will send "[byte-count in hex]\n" before the chunk, which is exactly what's happening here. Also note that chunked encoding will send an empty chunk, the "0\n\n" added at the end, to signal the end of the transfer. – sisve Feb 03 '13 at 11:25
  • I want to edit this answer to note that this is _not a bug_, and just a part of the HTTP 1.1 protocol, but I think that would change the answer radically. – sisve Feb 03 '13 at 11:28
  • Simon Svensson: Why not a bug ? It doesn't work with the default settings in all browsers on all operating systems, because the transmitted file is invalid, while the file that should be transmitted is perfectly valid -> bug. In the worst case, it's a bug in nginx and not mono, but I hope not. – Stefan Steiger Feb 03 '13 at 12:17
  • From the RFC2616 (HTTP/1.1), 3.6.1 Chunked Transfer Coding: "All HTTP/1.1 applications MUST be able to receive and decode the "chunked" transfer-coding [...]". He's comparing the data sent over the wire, including the protocol used to send the data, with the expected resulting data after any transformations are done. – sisve Feb 03 '13 at 13:26
  • The bug, if any, may be in his webserver sending out a HTTP/1.1 response to wget, when wget only issues HTTP/1.0 requests. Curl, however, supports HTTP/1.1. – sisve Feb 03 '13 at 13:31
  • @Simon Svensson: If that was true, then it should work on web browsers, which it doesn't. And Wget 1.13, released August 2011, supports HTTP/1.1. I have Wget 1.13.4. IE since version 5, Firefox since version 2) don't even have an option to use anything but HTTP/1.1. – Stefan Steiger Feb 03 '13 at 13:59
  • @Simon Svensson: But you may be right, according to http://www.quora.com/Why-doesnt-Nginx-talk-HTTP-1-1-to-upstream-servers , nginx does not talk http 1.1 to upstream servers, which might just be the reason for the bug. But I'm not so sure about that. But according to this http://mailman.nginx.org/pipermail/nginx/2011-August/028324.html , nginx supports chuncked transfer encoding in upstream servers. I use nginx 1.2.4. – Stefan Steiger Feb 03 '13 at 14:01
  • @Quandary, have you tried the patches posted in that nginx thread? – knocte Feb 15 '13 at 00:05
  • @knocte: No need to, I have a new enough nginx version. The patches should be in there. I tried the config options, but that didn't help. In addition, I got an answer from the nginx mailing list that a CGI/FastCGI interface may not return "chunked encoding headers" or anything else that could hinder the webserver's ability to determine how to send the file as per RFC 3875. (nginx will then determine whether it sends the file chunked or not) – Stefan Steiger Feb 15 '13 at 07:45
  • ok, please update the patch in xamarin's bugzilla with this reference to the RFC added – knocte Feb 15 '13 at 12:59
1
31 39 36 62 36 38 0D 0A

is 196b68/r/n in ascii.

  • does this color some how related to your code?
  • does the string 196b68/r/n say something related to your platforms?
0x90
  • 34,073
  • 33
  • 137
  • 218
  • 0x90: NOP, not to my knowledge ;) – Stefan Steiger Feb 02 '13 at 14:32
  • It might have something to do with the image itselfs. File says: 001.jpg: JPEG image data, EXIF standard – Stefan Steiger Feb 02 '13 at 14:35
  • why don't you use a tkdiff. and what is the output for 'file' on each of the files? – 0x90 Feb 02 '13 at 14:46
  • I use meld instead of tkdiff. Added additinal info. Just found out looking at the hex dump: 196b68 is the file size of the original image in hex, so it has nothing to do with any color that shows up when googling that number. – Stefan Steiger Feb 03 '13 at 04:22
  • ok, so you may trunc this number off? is it helps you? I think it something relating to file system of windows and linux.... @Quandary – 0x90 Feb 03 '13 at 07:37
  • "96b68\r\n" is Chunked Transfer Coding's way to say "here comes a chunk of 0x96b68 bytes. The ending, "0\r\n\r\n" also matches this, it means "here comes 0 bytes", which is the way of ending a chunked transfer coding, and the last newline is for terminating the "trailer", a chunk which may contain extra http headers if supported. – sisve Feb 03 '13 at 13:28
0

I don't recognize that Tools.Imaging.GetThumbnailStream() function, and neither does Google. Assuming that's your own code or a 3rd party library, I'd look there for the source of the strange bytestream.

David Pope
  • 6,099
  • 2
  • 33
  • 45
  • You're correct, it's my own code. Added the code in question. But I don't think the source for the strange bytestream is there. But thanks anyway, because when copy-pasting the code here, I saw I forgot to remove the messagebox.show, which came from the Winforms application I used to write the thumbnail code. – Stefan Steiger Feb 03 '13 at 09:17