Reconize image from running process

Question

So let's say that i want my application to reconize a picture from google, example this:

How can i make it pop up a messagebox, if the picture is visible in the process iexplorer.exe?
I dont know any possible way to do this, can someone please help me?

Do you want to know if it loads a particular image file? Or can be from any file that matches the image (regardless of fortmat)? Do you care about the resolution of the image? Does it need to visible in the page (but you would need to scroll) or does it need to be in the current viewport of the page (but it can be hidden in a second tab), or does it need to be visible on the window (but it can be minimized) or does it need to be on screen? And... why on earth do you want that? — Theraot, Dec 05 '13 at 14:12
@Theraot - 1. I want to load the image from a folder from desktop. 1.2 Let's say that a folder contains 2 images, i want the application to look for those 2 images. 2. The messagebox should appear when the picture is visible on the screen. — user2944342, Dec 05 '13 at 14:17
So, your first problem is to grab/analyze the content of what is currently displayed within the IE window or any window, is that it ? — Francis Ducharme, Dec 05 '13 at 15:36
@Francis - Correct. It's say the picture is a Blue button like in the question i asked. And if the "IE" process shows that image, it will pop up a messagebox. But the picture has to be visible on the screen, not scroll down. — user2944342, Dec 06 '13 at 07:26

score 0 · Answer 1 · edited May 23 '17 at 12:24

Its not going to be easy but the best thing I would know to do is try to search the current screen buffer to see if that image is anywhere on it. I would first program based on a fixed image size, then try to see how much performance I could get off a resized image. You could also pick "unique" color values from the image to see if those are matched anywhere on the screen before you start trying to do a more complete search.

In case you don't know how to access the screen buffer from C# the following resources should help:

The only other thought I would have is to check file handles to see what files are open and what process is using them. The big problem with this approach is that IE isn't necessarily going to cache every file and you are going to have a really tough time identifying which files match the file you are looking for. For that reason I wouldn't think this is a good approach. I hope this helps some!

Theraot · Answer 2 · 2013-12-05T15:34:57.643

I would take this approach, pseudo code:

Find the list of running processes
Select only the processes on which you are interested
Find the windows of those processes
For each window
    Get the current graphic context of that handle ^1
    Create a compatible graphic context and bitmap
    Copy the image to your bitmap ^2
    Search the bitmap on that graphic context (I suggest Boyer–Moore algorithm ^3)
    If there is a match
        Show message
Repeat and rinse

^1: This may be problematic if you want to support DirectX or OpenGl, in particular in old system.

^2: Believe or not, doing the whole copy and then scan is faster.

^3: Search it, but where it says "string" you read "image" and where it says "character" you read "pixel". You are going to use to scan individual rows of the image.

You will need:

score 0 · Answer 3 · edited May 23 '17 at 11:53

I think you will need GetForegroundWindow, from there you can determine if the foreground window is indeed IE's. GetClassName will return information about a window providing it the handle (HWND) of said window as parameter.

If so, then you need to grab a screenshot of it, like suggested in here.

You could also grab a screenshot of the whole display every X second, that would make it simpler.

AForge can be used for image analysis/manipulation.

I don't know if there would be a way to do this by event driven, but I think you'll need to use a Timer and get the whole process going at a preset interval.

Reconize image from running process

3 Answers3