Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

Examples of Python driven Microsoft UI Automation wanted

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


duane.kaufman at gmail

Jul 9, 2009, 10:14 AM

Post #1 of 14 (2894 views)
Permalink
Examples of Python driven Microsoft UI Automation wanted

Hi,

I am embarking on teaching myself Microsoft UI Automation using Python
as the scripting language.

I have asked some questions in the IronPython users group, but have
yet to get a response, so I thought I would broaden the audience by
asking here.

Specifically, I have a WinForms application I will be wanting to
automate. Does anyone have some Python examples of driving Microsoft
UI Automation they could share with me to get me started? The
structure of the UI automation classes etc. seem quite convoluted, and
I am having difficulty getting my brain wrapped around it.

Alternatives to Microsoft's UI Automation are welcome too, but I have
tried using winguiauto and watsup (along with AutoIt), and there seems
to be severe limitations when using these tools with WinForm
applications.

Thanks in advance,
Duane
--
http://mail.python.org/mailman/listinfo/python-list


usernet at ilthio

Jul 9, 2009, 10:18 AM

Post #2 of 14 (2856 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On 2009-07-09, TheSeeker <duane.kaufman [at] gmail> wrote:
> Specifically, I have a WinForms application I will be wanting to
> automate. Does anyone have some Python examples of driving Microsoft
> UI Automation they could share with me to get me started? The
> structure of the UI automation classes etc. seem quite convoluted, and
> I am having difficulty getting my brain wrapped around it.

If you find a way to work through the UI using the keyboard (tabs, etc);
then, you can send it keyboard commands using WScript.WshShell.SendKeys():

http://msdn.microsoft.com/en-us/library/8c6yea83(VS.85).aspx
--
http://mail.python.org/mailman/listinfo/python-list


duane.kaufman at gmail

Jul 9, 2009, 10:36 AM

Post #3 of 14 (2845 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On Jul 9, 12:18 pm, Tim Harig <user...@ilthio.net> wrote:
> On 2009-07-09, TheSeeker <duane.kauf...@gmail.com> wrote:
>
> > Specifically, I have a WinForms application I will be wanting to
> > automate. Does anyone have some Python examples of driving Microsoft
> > UI Automation they could share with me to get me started? The
> > structure of the UI automation classes etc. seem quite convoluted, and
> > I am having difficulty getting my brain wrapped around it.
>
> If you find a way to work through the UI using the keyboard (tabs, etc);
> then, you can send it keyboard commands using WScript.WshShell.SendKeys():
>
> http://msdn.microsoft.com/en-us/library/8c6yea83(VS.85).aspx

Thanks for the link.

Unfortunately, I need to be able to find out the contents of a few
text-boxes as well,
so SendKeys isn't all I need.

Thanks again,
Duane
--
http://mail.python.org/mailman/listinfo/python-list


usernet at ilthio

Jul 9, 2009, 10:45 AM

Post #4 of 14 (2835 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On 2009-07-09, DuaneKaufman <duane.kaufman [at] gmail> wrote:
> On Jul 9, 12:18 pm, Tim Harig <user...@ilthio.net> wrote:
>> On 2009-07-09, TheSeeker <duane.kauf...@gmail.com> wrote:
>> > Specifically, I have a WinForms application I will be wanting to
>> > automate. Does anyone have some Python examples of driving Microsoft
>> > UI Automation they could share with me to get me started? The
>> > structure of the UI automation classes etc. seem quite convoluted, and
>> > I am having difficulty getting my brain wrapped around it.
>> If you find a way to work through the UI using the keyboard (tabs, etc);
>> then, you can send it keyboard commands using WScript.WshShell.SendKeys():
>> http://msdn.microsoft.com/en-us/library/8c6yea83(VS.85).aspx
> Unfortunately, I need to be able to find out the contents of a few
> text-boxes as well,
> so SendKeys isn't all I need.

Then you will need some way to import your application as an object so that
you can gain access to the methods and properties directly. Using
IronPython you should be able to access anyting in the GAC. If you
register your UI componets then you should be able to access them. If not,
or using cpython, you might be able to do something like:

http://msdn.microsoft.com/en-us/library/fh1h056h(VS.71).aspx

to get a mashalled COM object.
--
http://mail.python.org/mailman/listinfo/python-list


duane.kaufman at gmail

Jul 9, 2009, 11:09 AM

Post #5 of 14 (2840 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On Jul 9, 12:45 pm, Tim Harig <user...@ilthio.net> wrote:
> On 2009-07-09, DuaneKaufman <duane.kauf...@gmail.com> wrote:
>
> > On Jul 9, 12:18 pm, Tim Harig <user...@ilthio.net> wrote:
> >> On 2009-07-09, TheSeeker <duane.kauf...@gmail.com> wrote:
> >> > Specifically, I have a WinForms application I will be wanting to
> >> > automate. Does anyone have some Python examples of driving Microsoft
> >> > UI Automation they could share with me to get me started? The
> >> > structure of the UI automation classes etc. seem quite convoluted, and
> >> > I am having difficulty getting my brain wrapped around it.
> >> If you find a way to work through the UI using the keyboard (tabs, etc);
> >> then, you can send it keyboard commands using WScript.WshShell.SendKeys():
> >>http://msdn.microsoft.com/en-us/library/8c6yea83(VS.85).aspx
> > Unfortunately, I need to be able to find out the contents of a few
> > text-boxes as well,
> > so SendKeys isn't all I need.
>
> Then you will need some way to import your application as an object so that
> you can gain access to the methods and properties directly.  Using
> IronPython you should be able to access anyting in the GAC.  If you
> register your UI componets then you should be able to access them.  If not,
> or using cpython, you might be able to do something like:
>
> http://msdn.microsoft.com/en-us/library/fh1h056h(VS.71).aspx
>
> to get a mashalled COM object.

Thanks again Tim.

I believe I probably mis-communicated my requirements (or at least,
was not explicit enough)

The application I wish to interact with is not my own, but an ERP
system GUI front-end.

With MS utilities like UISpy and the like, I can 'see' the controls in
the application, but I
do not seem to be able to manipulate them programatically, and I
believe it us simply due
to my not understanding of the UI Automation API.

Hence, my request for example code to boot-strap myself.

Thanks,
Duane
--
http://mail.python.org/mailman/listinfo/python-list


ptmcg at austin

Jul 9, 2009, 11:29 AM

Post #6 of 14 (2845 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On Jul 9, 1:09 pm, DuaneKaufman <duane.kauf...@gmail.com> wrote:
> The application I wish to interact with is not my own, but an ERP
> system GUI front-end.
>

I have used pywinauto to drive a Flash game running inside of an
Internet Explorer browser - that's pretty GUI!

-- Paul
--
http://mail.python.org/mailman/listinfo/python-list


duane.kaufman at gmail

Jul 9, 2009, 11:50 AM

Post #7 of 14 (2837 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On Jul 9, 1:29 pm, Paul McGuire <pt...@austin.rr.com> wrote:
> On Jul 9, 1:09 pm, DuaneKaufman <duane.kauf...@gmail.com> wrote:
>
> > The application I wish to interact with is not my own, but an ERP
> > system GUI front-end.
>
> I have used pywinauto to drive a Flash game running inside of an
> Internet Explorer browser - that's pretty GUI!
>
> -- Paul

Hi,

Could you share some code examples?

Thanks,
Duane (duanek (at) chorus (dot) net)
--
http://mail.python.org/mailman/listinfo/python-list


ptmcg at austin

Jul 9, 2009, 1:41 PM

Post #8 of 14 (2840 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On Jul 9, 1:50 pm, DuaneKaufman <duane.kauf...@gmail.com> wrote:
> > I have used pywinauto to drive a Flash game running inside of an
> > Internet Explorer browser - that's pretty GUI!
>
> > -- Paul
>
> Hi,
>
> Could you share some code examples?
>
> Thanks,
> Duane (duanek (at) chorus (dot) net)

I just went on a brief fishing expedition through two disk backups,
and no luck. I guess it's been a while since I worked on this.

The work I did was entirely graphical, which is to say, my script
interacted with the Flash program by using PIL to take image snapshots
of the window, and then sifting through the bitmap looking for the
status of a "time remaining" thermometer-style gauge in the game.
Then the script could click on X-Y coordinates within the window,
which would get picked up by the Flash game, and the script would
monitor the "time remaining" gauge some more, and so on.

I'm not sure how well pywinauto would work in allowing you to access
controls such as textboxes within a form. I remember that I had to
access the IE window using a caption name, and then found the embedded
Flash program as an embedded control of some sort, again, I probably
needed to indicate that it was some sort of "Adobe.FlashWidget1"
object or something.

I may have another, older disk backup at home, I can look for it later
this evening.

-- Paul
--
http://mail.python.org/mailman/listinfo/python-list


torriem at gmail

Jul 9, 2009, 2:09 PM

Post #9 of 14 (2851 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

DuaneKaufman wrote:
> With MS utilities like UISpy and the like, I can 'see' the controls in
> the application, but I
> do not seem to be able to manipulate them programatically, and I
> believe it us simply due
> to my not understanding of the UI Automation API.

You're probably better off using a free app like AutoIt3 rather than
roll your own. AutoIt's scripting language isn't as nice as Python, but
it does the job and it turing-complete.

http://www.autoitscript.com/autoit3/
--
http://mail.python.org/mailman/listinfo/python-list


torriem at gmail

Jul 9, 2009, 2:12 PM

Post #10 of 14 (2842 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

TheSeeker wrote:
> Alternatives to Microsoft's UI Automation are welcome too, but I have
> tried using winguiauto and watsup (along with AutoIt), and there seems
> to be severe limitations when using these tools with WinForm
> applications.

http://www.autoitscript.com/forum/index.php?showtopic=96752 -- looks
like recent betas have much better support for WinForms, at least from
the change list.


--
http://mail.python.org/mailman/listinfo/python-list


duane.kaufman at gmail

Jul 9, 2009, 3:10 PM

Post #11 of 14 (2851 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On Jul 9, 4:12 pm, Michael Torrie <torr...@gmail.com> wrote:
> TheSeeker wrote:
> > Alternatives to Microsoft's UI Automation are welcome too, but I have
> > tried using winguiauto and watsup (along with AutoIt), and there seems
> > to be severe limitations when using these tools with WinForm
> > applications.
>
> http://www.autoitscript.com/forum/index.php?showtopic=96752-- looks
> like recent betas have much better support for WinForms, at least from
> the change list.

Hi,

Thanks for the information. Boy, you are away from a utility for a
couple of
weeks, and it starts growing things one needs!

Unfortunately, while it looks like AutoIt is gaining the ability to
'see' WinForms controls,
this beta doesn't seem to be able to retrieve the text say, in a
TextBox. Darn.

Thanks,
Duane
--
http://mail.python.org/mailman/listinfo/python-list


timr at probo

Jul 9, 2009, 8:50 PM

Post #12 of 14 (2823 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

TheSeeker <duane.kaufman [at] gmail> wrote:
>
>I am embarking on teaching myself Microsoft UI Automation using Python
>as the scripting language.

The University of North Carolina at Chapel Hill has a great Python package
called pyAA that does exactly this:

http://www.cs.unc.edu/Research/assist/developer.shtml
http://mindtrove.info/articles/gui-automation-with-pyaa/
--
Tim Roberts, timr [at] probo
Providenza & Boekelheide, Inc.
--
http://mail.python.org/mailman/listinfo/python-list


kf9150 at gmail

Jul 10, 2009, 12:13 AM

Post #13 of 14 (2830 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

On Jul 9, 5:50 pm, Tim Roberts <t...@probo.com> wrote:
> The University of North Carolina at Chapel Hill has a great Python package
> called pyAA that does exactly this:
>
> http://www.cs.unc.edu/Research/assist/developer.shtmlhttp://mindtrove.info/articles/gui-automation-with-pyaa/
> --
> Tim Roberts, t...@probo.com

Thanks Tim. From this page http://sourceforge.net/projects/uncassist/files/,
there isn't a installer for Python2.5 or higher. Didn't find the
source code there either.

--
http://mail.python.org/mailman/listinfo/python-list


mtnbikingmark at gmail

Jul 10, 2009, 5:30 AM

Post #14 of 14 (2837 views)
Permalink
Re: Examples of Python driven Microsoft UI Automation wanted [In reply to]

Hi,

I am the author of pywinauto (http://pywinauto.openqa.org/).

It doesn't have specific functionality built in for winforms - but
many winforms controls are built on top of win32 functionality and
pywinauto can work with them for that.

On Jul 9, 4:41 pm, Paul McGuire <pt...@austin.rr.com> wrote:
> On Jul 9, 1:50 pm, DuaneKaufman <duane.kauf...@gmail.com> wrote:
>
> > > I have used pywinauto to drive a Flash game running inside of an
> > > Internet Explorer browser - that's pretty GUI!
>

Wow - cool :)


> > > -- Paul
>
> > Hi,
>
> > Could you share some code examples?
>

You might have a look at http://pywinauto.pbworks.com/ (the site is no
longer used - but there is a cool movie giving an example of using
pywinauto at the interactive prompt)

Other examples (Notepad, mspaint) are included with the install.

PyPi - http://pypi.python.org/pypi/pywinauto/0.3.8


> > Thanks,
> > Duane (duanek (at) chorus (dot) net)
>

Good luck! :)
Mark

> I just went on a brief fishing expedition through two disk backups,
> and no luck.  I guess it's been a while since I worked on this.
>
> The work I did was entirely graphical, which is to say, my script
> interacted with the Flash program by using PIL to take image snapshots
> of the window, and then sifting through the bitmap looking for the
> status of a "time remaining" thermometer-style gauge in the game.
> Then the script could click on X-Y coordinates within the window,
> which would get picked up by the Flash game, and the script would
> monitor the "time remaining" gauge some more, and so on.
>
> I'm not sure how well pywinauto would work in allowing you to access
> controls such as textboxes within a form.  I remember that I had to
> access the IE window using a caption name, and then found the embedded
> Flash program as an embedded control of some sort, again, I probably
> needed to indicate that it was some sort of "Adobe.FlashWidget1"
> object or something.
>
> I may have another, older disk backup at home, I can look for it later
> this evening.
>
> -- Paul

--
http://mail.python.org/mailman/listinfo/python-list

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.