0

I'm having a little trouble trying to get WarpPointer to work out appropriately for me. The scenario I'm trying to do is akin to recording a user's clicks and then replaying them on the screen. I've been able to record the screen motions and clicks without any issues; however, I'm stumbling when it comes to replaying what the user recorded.

def MoveCursorAndClick(self, e):
    # get the current screen position
    sp = self.GetPosition()
    p = self.getNextPosition() # returned in absolute x,y from screen rec
    self.WarpPointer(p.x - sp[0], p.y - sp[1])
    # wx.PostEvent(EVT_LEFT_DOWN, self.GetId())

Obviously, the above code is incorrect because WarpPointer appears to work within the window only and not the full screen. Attempting to base the coordinated off of the functioning window as well proves to be troublesome because they always seem to be off in the Y direction by about 50 pix. My only current assumption is I'm making this harder than it needs to be and there's a better why I'm not seeing due to my inexperience with wxpython.

Additionally, the busted PostEvent line is the only thing I've managed to google across that looks like the right thing for sending a mouse click to something that may or may not be another window. Is that even the right direction?

Updated As mentioned below, there's not a good cross platform way to accomplish this. Here's what I did (found various places on the internet):

def ClickScreenPositon(self, x, y):
    win32api.SetCursorPos((x,y))
    win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
    win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)
Wyatt
  • 90
  • 11

3 Answers3

3

I don't think you can do this in wxPython alone. wx.Window.WarpPointer() can only move the mouse to an area controlled by that window, as you've discovered. Similarly, you're wx.EVT_LEFT_DOWN call won't work because you're setting the ID to the window, meaning it's the window that was clicked, not a click at (x,y).

After some light Googling and I found this article on Unit Testing in wxPython that breifly talks about testing at the GUI level. This isn't exactly what you're trying to do, but it is conceptually similiar. They suggest using an external library to manipulate the interface. The main problem they note about this approach is that most of the tools for simulation GUI interaction are under corporate licenses and/or platform dependent.

That article suggests using pywinauto for Windows systems. For your purposes, pywinauto.controls.HwndWrapper seems to have the methods you require; MoveMouse() and Click(). If you aren't using Windows or if you need cross-platform support you will need to research different libraries.

Basically, you will need to install and import pywinauto into your wxPython application and call its methods instead of wxPython's. As long as cross-platform compatibility is not an issue I think this is your simplest, best solution.

acattle
  • 2,893
  • 1
  • 14
  • 19
  • I tried both of those and didn't manage to have much luck. Sadly, you're correct in that there sin't really any good x-platform way to do this. Here's what I ended up doing (found several other places on the internet) : def click(self, x, y): win32api.SetCursorPos((x,y)) win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0) win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0) – Wyatt Jun 19 '12 at 04:49
  • Well I'm sorry that I couldn't help. I am happy that you were able to find something that works, even if it isn't cross platform – acattle Jun 19 '12 at 05:53
  • Sometimes ... that's just sadly the way it rolls; but maybe some adventurous programmer will make it their mission to solve in the future :) – Wyatt Jul 02 '12 at 16:33
1

wxPython only has control of the cursor when it's over a wxPython generated window, frame or widget. When you leave the confines of your wxPython application, it loses control of the mouse. You'll have to hook into the OS at that point. On Windows, the PyWin32 package is probably your best bet.

There's also a cool little project called sikuli that might work too if you can figure out how to take a screenshot of the screen: http://sikuli.org/

Mike Driscoll
  • 31,394
  • 6
  • 39
  • 83
0

I have created the open source application JW_Record_Playback to be used as a record playback mechanism in python wxwindows. It actually uses TK / TCL as the external resource. It works fine on Linux (Tested both in the Suse and the Ubuntu distribution).

Please check: http://members.home.nl/wijnenjl/Record_Playback_Python_TclTk_INI_Wx_EN_01.html

Jon Lin
  • 135,941
  • 26
  • 200
  • 209
Sjaak Wijnen
  • 1