2

Capturing a screen shot is quite easy but cropping is a different story, or at least appears so. We are attempting to emulate the solution here in Xamarin using SkiaSharp (System.Drawing not supported in Xamarin).

Currently we are able to capture the screenshot and return the image BUT if we crop our image the image returned is all black.

How do we crop a captured screenshot correctly?

NOTE*: image = image.Subset(rec); under the "crop image" section is how we are trying to crop.

iOS screenshot

public byte[] Capture()
{
    var capture = UIScreen.MainScreen.Capture();
    using (NSData data = capture.AsPNG())
    {
        var bytes = new byte[data.Length];
        Marshal.Copy(data.Bytes, bytes, 0, Convert.ToInt32(data.Length));
        return bytes;
    }
}

Droid screenshot

public byte[] Capture()
{
    var rootView = context.Window.DecorView.RootView;

    using (var screenshot = Bitmap.CreateBitmap(
                            rootView.Width,
                            rootView.Height,
                            Bitmap.Config.Argb8888))
    {
        var canvas = new Canvas(screenshot);
        rootView.Draw(canvas);

        using (var stream = new MemoryStream())
        {
            screenshot.Compress(Bitmap.CompressFormat.Png, 90, stream);
            return stream.ToArray();
        }
    }
}

Attempting to crop captured screenshot

// Use this function to crop a screen shot to a specific element.  
public byte[] test(byte[] screenshotData, View element)
{
    //  locate IntPtr to byte[] of uncropped screenshot
    GCHandle gch = GCHandle.Alloc(screenshotData, GCHandleType.Pinned);
    IntPtr addr = gch.AddrOfPinnedObject();

    //  assign initial bounds
    SKImageInfo info = new SKImageInfo((int)App.Current.MainPage.Width,
        (int)App.Current.MainPage.Height);

    //  create initial pixel map
    using SKPixmap pixmap = new SKPixmap(info, addr);
    //  Release
    gch.Free();

    //  create bitmap
    using SKBitmap bitmap = new SKBitmap();
    //  assign pixel data
    bitmap.InstallPixels(pixmap);

    //  create surface
    using SKSurface surface = SKSurface.Create(info);
    //  create a canvas for drawing
    using SKCanvas canvas = surface.Canvas;
    //  draw
    canvas.DrawBitmap(bitmap, info.Rect);

    //  get an image subset to save
    SKImage image = surface.Snapshot();
    SKRectI rec = new SKRectI((int)element.Bounds.Left, (int)element.Bounds.Top,
        (int)element.Bounds.Right, (int)element.Bounds.Bottom);

    //  crop image
    image = image.Subset(rec);

    byte[] bytes = SKBitmap.FromImage(image).Bytes;
    image.Dispose();
    return bytes;
}

EDIT: Alternative solution attempt (not working)

// Use this function to crop a screen shot to a specific element.  
public byte[] test(byte[] screenshotData, View element)
{
    //  locate IntPtr to byte[] of uncropped screenshot
    GCHandle gch = GCHandle.Alloc(screenshotData, GCHandleType.Pinned);
    IntPtr addr = gch.AddrOfPinnedObject();
    //  assign initial bounds
    SKImageInfo info = new SKImageInfo((int)App.Current.MainPage.Width,
        (int)App.Current.MainPage.Height);

    //  create bitmap
    SKBitmap bitmap = new SKBitmap();
    bitmap.InstallPixels(info, addr);

    //  boundaries
    SKRect cropRect = new SKRect((int)element.Bounds.Left, (int)element.Bounds.Top,
        (int)element.Bounds.Right, (int)element.Bounds.Bottom);
    SKBitmap croppedBitmap = new SKBitmap((int)cropRect.Width,
        (int)cropRect.Height);
    SKRect dest = new SKRect(0, 0, cropRect.Width, cropRect.Height);
    SKRect source = new SKRect(cropRect.Left, cropRect.Top,
                               cropRect.Right, cropRect.Bottom);

    //  draw with destination and source rectangles 
    //      to extract a subset of the original bitmap
    using SKCanvas canvas = new SKCanvas(croppedBitmap);
    canvas.DrawBitmap(bitmap, source, dest);
    return croppedBitmap.Bytes;
    //return bitmap.Bytes;
}
jtth
  • 756
  • 10
  • 29
  • I am not clear what does it means? **NOTE*: Commenting out image = image.Subset(rec); under the "crop image" section is returns the image perfectly. However we would like to crop the image if at all possible.** What problem do you encounter when cropping the captured image? I find one sample about cropping and capture image, you can take a look:https://github.com/MobMaxime/xamarin-android-samples/tree/master/CropImageSample – Cherry Bu - MSFT Dec 13 '19 at 06:25
  • can you provide one simple sample at github that can reproduce your issue here? Because I have difficult to reproducing your issue at my side. – Cherry Bu - MSFT Dec 17 '19 at 08:32
  • @jtth sorry about my post not working. I deleted it so as to not waste people's time. Usually the .NET libraries work in Xamarin so I figured this very basic one would work aswell and I had also read online (from undeleted posts) that it *would* work. I'll see if I can find another solution – hexagod Dec 18 '19 at 01:24
  • @jtth,Glad to hear that you have solved your issue, can you mark your reply as answer,thanks. – Cherry Bu - MSFT Dec 18 '19 at 05:52
  • @jtth have you checked out this page? https://docs.microsoft.com/en-us/xamarin/xamarin-forms/user-interface/graphics/skiasharp/bitmaps/cropping .. I was trying to create a solution for you but VS is locking up when I try to add SkiaSharp nuget. What you might want to try is not converting to an image just create a `CroppingRectangle` then `SKCanvas` and use those to crop the `SKBitmap` directly. The documentation for that library is really confusing and since I can't test it I'm not going to try another answer but this is one document you could try. – hexagod Dec 18 '19 at 22:54

1 Answers1

0

iOS solution - As seen here.

// crop the image, without resizing
private UIImage CropImage(UIImage sourceImage, int crop_x, int crop_y, int width, int height)
{
    var imgSize = sourceImage.Size;
    UIGraphics.BeginImageContextWithOptions(new System.Drawing.SizeF(width, height), false, 0.0f);
    var context = UIGraphics.GetCurrentContext();
    var clippedRect = new RectangleF(0, 0, width, height);
    context.ClipToRect(clippedRect);
    var drawRect = new RectangleF(-crop_x, -crop_y, imgSize.Width, imgSize.Height);
    sourceImage.Draw(drawRect);
    var modifiedImage = UIGraphics.GetImageFromCurrentImageContext();
    UIGraphics.EndImageContext();
    return modifiedImage;
}

Android solution - getNavigationBarSize(context) as seen here

// crop the image, without resizing
private byte[] CropImage(byte[] screenshotBytes, int top)
{
    Android.Graphics.Bitmap bitmap = Android.Graphics.BitmapFactory.DecodeByteArray(
      screenshotBytes, 0, screenshotBytes.Length);

    int viewStartY = (int)(top * 2.8f);
    int viewHeight = (int)(bitmap.Height - (top * 2.8f));
    var navBarXY = getNavigationBarSize(context);
    int viewHeightMinusNavBar = viewHeight - navBarXY.Y;

    Android.Graphics.Bitmap crop = Android.Graphics.Bitmap.CreateBitmap(bitmap,
        0, viewStartY,
        bitmap.Width, viewHeightMinusNavBar
        );

    bitmap.Dispose();

    using MemoryStream stream = new MemoryStream();
    crop.Compress(Android.Graphics.Bitmap.CompressFormat.Jpeg, 100, stream);
    return stream.ToArray();
}

*NOTE: Unsure why a multiplication of 2.8 is required, though this works correctly. It should be stated testing was only done in the Android emulator. Perhaps it's emulator specific.

*NOTE2: x = 0 and width is equal to the entire width of the screen as that's per our requirement. Likewise top is Element.Bounds.Top.

jtth
  • 756
  • 10
  • 29