yinkaisheng / python-uiautomation-for-windows Goto Github PK

while I want to choose a value from a combobox, in my .net application, I found the "IsExpandCollapsePatternAvailable" value in the Inspect was "false".
so while I run
automation.ComboBoxControl(searchFromControl=blablabla, ClassName=‘blablabla').Select(value)
it failed as in class ComboBoxControl(), method Select(), the code is self.Expand().

and I just make a workaround that replace self.Expand() with self.Click(), and it works

later, I tried to not break your original code, so add an "if" in Select() method:

    if self.IsExpandCollapsePatternAvailable():
        self.Expand()
    else:
        self.Click()

and add one property to get the IsExpandCollapsePatternAvailable value

def IsExpandCollapsePatternAvailable(self):
    '''Return bool'''
    return _automationClient.dll.GetElementIsExpandCollapsePatternAvailable(self.Element)

but it reports not found error

if self.IsExpandCollapsePatternAvailable():
File "D:\Tools\Python\Python-UIAutomation-for-Windows\automation.py", line 1585, in IsExpandCollapsePatternAvailable
return automationClient.dll.GetElementIsExpandCollapsePatternAvailable(self.Element)
File "C:\Python27\lib\ctypes__init_.py", line 378, in getattr
func = self.getitem(name)
File "C:\Python27\lib\ctypes__init__.py", line 383, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'GetElementIsExpandCollapsePatternAvailable' not found

how do I know the functions detail in the _automationClient.dll ?

我用windows画图做一个实验时发现，鼠标位置与期待的不符，我把画图软件打开并最大化，然后直接调用库中的的DragDrop函数，就像这样 DragDrop(200, 200, 1000, 1000, 2)。这样子会成功的画出一条直线，但是该直线的末端比我的光标多出了一截
就像这样

估计是在DragDrop函数内，为了实现duration功能丢失了精度问题。

1 - Global waiting gap

eg. I want to get all the operations like Click/DoubleClick/InvokePattern.Invoke/... to wait for few seconds after each operation, for just like 1 second waiting for each operation, need a parameter in global to have it configured.

2 - Control search code style

I used to be used with UFT, which has the code style like:
CheckBoxControl("AutoId1").HeaderControl("AutoId2").ButtonControl("AutoId3").Click()
AutomationId is default search parameter

Currently I need to write above code like following:
CheckBoxControl(searchFromControl=HeaderControl(searchFromControl=ButtonControl(searchFromControl=IEWindow, AutomationId = "AutoId1"),AutomationId = "AutoId2"),AutomationId="AutoId3").Click()

3 - Wait for some element available or exists function

Just like WaitForExist(SomeControl, timeout)
It can wait for the control to be existed in appointed timeout value. Another enhancement is WaitForDisappear(SomeControl, timeout), all returns a bool

Is is possible to have it?

Thanks!

Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC v.1900 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

RESTART: C:\Users\yanglei2\Desktop\Python-UIAutomation-for-Windows-master\automation_notepad_py3.py
Traceback (most recent call last):
File "C:\Users\yanglei2\Desktop\Python-UIAutomation-for-Windows-master\automation_notepad_py3.py", line 7, in
import uiautomation as automation
File "C:\Users\yanglei2\Desktop\Python-UIAutomation-for-Windows-master\uiautomation.py", line 55, in
_automationClient = AutomationClient()
File "C:\Users\yanglei2\Desktop\Python-UIAutomation-for-Windows-master\uiautomation.py", line 43, in init
self.dll = ctypes.cdll.UIAutomationClientX86
File "C:\Users\yanglei2\AppData\Local\Programs\Python\Python35-32\lib\ctypes_init.py", line 417, in getattr
dll = self.dlltype(name)
File "C:\Users\yanglei2\AppData\Local\Programs\Python\Python35-32\lib\ctypes_init.py", line 347, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] 找不到指定的模块。

The CaptureToImage() method doesn't work with UWP application. It takes black screenshots.
For "ApplicationFrameWindow" Class Name.

---- Code ---- (Exp:)
import uiautomation as uiauto calculatorWindow = uiauto.WindowControl(Name="Calculator") # The UWP calculator app calculatorWindow.CaptureToImage("d:\\captureCalculatorWindowControl.png")
---- Result ----

你好,剛接觸到自動測試領域,還在摸索中,看到你的代碼對我很有幫助
有碰到幾個問題想請問下:
當同個UI頁面有兩相通的控件類型,除了Rect位置不一樣,其餘信息完全相同,該如何選定正確的控件進行控制?
(例如:一頁面有兩ScrollBarControl
ControlType: ScrollBarControl ClassName: AutomationId: NonClientVerticalScrollBar Rect: (176, 193, 193, 715) Name: 垂直滚动条 Handle: 0x0(0) Depth: 7 ProcessId:7828 RangeValue: 0

ControlType: ScrollBarControl ClassName: LineVScrollBar AutomationId: VerticalScrollBar Rect: (1349, 193, 1366, 715) Name: 垂直滚动条 Handle: 0x410A5A(4262490) Depth: 7 ProcessId:7828 RangeValue: 0)
謝謝!

2017-06-23 04:03:28.451 Function: main, Line: 4207 -> Starts, Current Cursor Position: (867, 465)
ControlType: WindowControl ClassName: ChatWnd AutomationId: Rect: (1084, 22, 1909, 982) Name: test1 Handle: 0xF2394(992148) Depth: 0
ControlType: PaneControl ClassName: popupshadow AutomationId: Rect: (1065, 3, 1928, 1001) Name: Handle: 0x72362(467810) Depth: 1
ControlType: PaneControl ClassName: ChatContactMenu AutomationId: Rect: (-10000, -10000, -9999, -9999) Name: ChatContactMenu Handle: 0x82348(533320) Depth: 1
ControlType: PaneControl ClassName: popupshadow AutomationId: Rect: (-10019, -10019, -9980, -9980) Name: Handle: 0x62396(402326) Depth: 2
2017-06-23 04:03:28.484 Function: main, Line: 4238 -> Ends

大神，
谢谢你的这套框架，有几个问题想问下：
1. 如过我新安装了一个程序，比如可执行文件是"abc.exe"，我可以如何打开它
2.如果这是一个.net写的程序，但是没有源代码，我要如何知道它某个窗口和控件的名字？
3.如果程序不是.net写的，依然可以用这套框架吗？可以怎样知道窗口和控件的名字?

非常感谢了！！！

Does it support global time out settings?

BTW, it is a great open source automation tool.

Python3.7 Win10操作系统，参考https://www.cnblogs.com/Yinkaisheng/p/3444132.html 对记事本进行操作，、consoleWindow = automation.GetConsoleWindow() 返回结果为None

Hi,

I'm doing test automation for a UWP application, each icon of the application is covered by "Border" control. However, "BorderControl"s are not detected by UIautomation,
What i'm doing is adding the "BorderControl" widget type to uiautomation, but it doesn't work!
Someone have any suggestions?

Thanks,

Hi Kaisheng,

In win10, I keep seeing access violation reading error. Seems it's still related to dll. What' your Windows 10 version?

Thank you.

电脑上是否需提前安装.net framework？哪个版本？

遇到一个问题，就是如何在浏览文件夹的窗口中直接选定想要定位的文件夹，想直接使用文件夹的路径为参数

http://www.cnblogs.com/Yinkaisheng/p/3444132.html 比较粗糙，有详细API文档吗？

Hi Kaisheng,

In win10, it will show the "WindowsError: [Error 126] The specified module could not be found" hints after processed "import uiautomation as automation";

I took my attention with KB971513 patch but that would not fit to win10. Specially, on another win10 machine, that works well and didn't pop-up any error hint.

I guess there is something incompatible with UIAutomationClient.dll but have no idea to fix it;
If you are free please have a look at it.
Thanks.

subprocess.Popen('MessageBox.exe')
print(automation.GetRootControl())

Form= automation.WindowControl(searchDepth = 1,ClassName = "Window",Name="MainWindow")
Form.EditControl(AutomationId='messageBoxText').Click()
Form.EditControl(AutomationId='caption').Click()

ownerCheckBox=Form.CheckBoxControl(AutomateionId='ownerCheckBox')

Form.ComboBoxControl(AutomateionId="buttonComboBox").Select('YesNo')
#第一个能选择上

Form.ComboBoxControl(AutomateionId="imageComboBox").Select('Stop') Form.ComboBoxControl(AutomateionId="defaultResultComboBox").Select('Yes')
Form.ComboBoxControl(AutomateionId="optionsComboBox",).Select('RightAlign')
#这三个都不能成功操作
txtName=Form.ButtonControl(AutomationId="showMessageBoxButton").Click()

automation.py 的代码改了这么多啊， 今天一看差距太大了，看来我在Python上的道路还有很远啊，膜拜大神！但是好像还是不能实现我想要的事情，没事看看对自己也好！

Hi !

I tried to add ProcessId property

class Control():
 @property
    def ProcessId(self):
        '''Return process id'''
        return ClientObject.dll.GetProcessId(self.Element)

and when I call the property I got this error message

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\python27\lib\ctypes\__init__.py", line 378, in __getattr__
    func = self.__getitem__(name)
  File "C:\python27\lib\ctypes\__init__.py", line 383, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'GetElementProcessId' not found

怎么通过句柄 快速获取/定位到 一个Control ？这样比从root进行搜索要来的快些，不支持有没有这个支撑？

你好 我想问下支持wpf的ui自动化么

我在运行您的那个 uiautomation.py 如果我不调用最后的EnumAndLogControl 这个函数程序并不会正常退出而是出现假死 （部分窗口控件）

Hi,
Is that normal that you should do time.Sleep(x) so the BoundingRectangle of the control will be updated with the current values?
It wasn't the case before!
Thanks,

InspectX.exe
AutomationClientX.dll

Hi Yang, it is great project! where can I get the help documents for the development?

@yinkaisheng can we find control if we know part of the application name. Ex: want to find "Notepad - Akshay" using regex "Notepad - *"?

Hello yinkaisheng,
This project is fantastic and I think it could really repalce the QTP. However, I found the SetTopmost method not always work as expected when running the demo of notepadtop.py. Here are the reproducing steps:

1. Open a notepad and minimize it.
2. Run notepadtop.py.

the definition of SetWindowTopmost(hWnd, isTopmost) is:
topValue = SWP.HWND_TOPMOST if isTopmost else SWP.HWND_NOTOPMOST
return Win32API.SetWindowPos(hWnd, topValue, 0, 0, 0, 0, SWP.SWP_NOSIZE|SWP.SWP_NOMOVE)

I think it's better to change to:
topValue = ShowWindow.Restore if isTopmost else ShowWindow.Hide
Win32API.ShowWindow(hWnd, topValue)
if isTopmost:
Win32API.SetForegroundWindow(hWnd)

你的例子里调用SetActive(),不对， 你的例子错了，你的源码中这个SetActive()方法是写在WindowControl这个类里的， 你的调用没有从这个类里继承，就会报错~！

This is a question about use.

I would like to discover text under mouse on applications as Microsoft Word or Outlook. Usually I get this info with
control = ControlFromCursor()
text = control.Name+"="+control.CurrentValue()

But inside the main text content (page document) dont information about text content, and dont appear any other control inside (no more deepth)

I have discover that with inspector tool over a opened Word document, you can get info about pages, and even the paragraph parts inside pages, but dont appear any text values at alll. But I have discover that you can get the really text inside paragraphs with the option menu:
Inspector>Action>Text Pattern Explorer
With this, it appear a new window that let you navigate around the document and get info of text in every form (paragraph).

Coul you help me how to get this same with your ui-automation? I cant figure how to make this exploring
I have the same problem inside the mail content in Outlook application.

Thanks in advance for this really great tool.

I was running find control. The function hung in this spot. This is the code I used.

import uiautomation
for item, depth in uiautomation.WalkControl(uiautomation.GetRootControl(), maxDepth=1):
print item

您好老师，能出一份文档吗，概介绍一下常用api的功能和基本使用方法

Win7系统下程序执行一切正常，放在Win10系统下程序就会卡住，调试后发现在多线程中控件获取不到，一直处于Connect的状态，比如这样一句代码：automation.WindowControl(searchDepth=1, AutomationId='', Name=‘')，不放在线程中执行可以获取到控件，一旦放在线程中执行这样一句代码程序就会卡住，一直处于Connect的状态，不知道是什么原因，请老师指教，谢谢！

Hi
in your last commit, you update the ExpandCollapseState() class, and I can see this state value for a TreeItemControl in Inspect.exe, but is that any method to get this state?
I want to try to get all the TreeItemControls and loop calling Expand() until it's a LeafNode.

or shall I use walkTree() to get all of them? I tried, but walkTree() will use getChildrenFunc/getFirstChildFunc/getNextSiblingFunc, but if the tree was not expanded, all are None.

I'm on a budget I cannot buy more machine, I just have 2 monitors and one machine. I have 2 Windows apps and need to run both in parallel, could I achieve run each app on each monitor by using your library at same time ?

比pywinauto用着顺手！

日志会再补上

场景1:利用windows任务计划执行，UI自动化的脚本无法执行，会报寻找控件超时的错误
场景2:利用time.sleep() 一段时间，当电脑锁屏后也无法执行UI自动化

有没有办法解决

Traceback (most recent call last):
File "init.py", line 120, in wait_for_control
if control.Exists(search_wait_time, uiautomation.SEARCH_INTERVAL):
File "C:\Python27\Lib\site-packages\uiautomation\uiautomation.py", line 2424, in Exists
control = FindControl(self.searchFromControl, self._CompareFunction, self.searchDepth, False, self.foundIndex)
File "C:\Python27\Lib\site-packages\uiautomation\uiautomation.py", line 4299, in FindControl
for child, depth in WalkControl(control, findFromSelf, maxDepth):
File "C:\Python27\Lib\site-packages\uiautomation\uiautomation.py", line 4185, in WalkControl
child = lastControl.GetFirstChildControl()
File "C:\Python27\Lib\site-packages\uiautomation\uiautomation.py", line 2670, in GetFirstChildControl
return Control.CreateControlFromElement(comEle)
File "C:\Python27\Lib\site-packages\uiautomation\uiautomation.py", line 2790, in CreateControlFromElement
return ControlDictcontrolType

你好，有支持linux的计划吗？
Python-UIAutomation-for-linux？

Traceback (most recent call last):
File "D:/hqzf_ui_test/cases/sample.py", line 52, in
menu_item.MoveCursorToMyCenter()
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2676, in MoveCursorToMyCenter
return self.MoveCursor(simulateMove = simulateMove)
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2657, in MoveCursor
left, top, right, bottom = self.BoundingRectangle
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2643, in BoundingRectangle
_AutomationClient.instance().dll.GetElementBoundingRectangle(ctypes.c_size_t(self.Element), ctypes.byref(rect))
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2561, in Element
self.Refind(maxSearchSeconds = TIME_OUT_SECOND, searchIntervalSeconds = self.searchWaitTime)
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2546, in Refind
if not self.Exists(maxSearchSeconds, searchIntervalSeconds):
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2515, in Exists
if prev and not prev._element and not prev.Exists(maxSearchSeconds, searchIntervalSeconds):
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2518, in Exists
control = FindControl(self.searchFromControl, self._CompareFunction, self.searchDepth, False, self.foundIndex)
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 4407, in FindControl
if compareFunc(child, depth):
File "C:\Users\admin\AppData\Roaming\Python\Python36\site-packages\uiautomation\uiautomation.py", line 2488, in _CompareFunction
raise KeyError('unsupported argument: {} in {} constructor'.format(key, self.class.name))
KeyError: 'unsupported argument: LocalizedControlType in MenuControl constructor'

Hey bro! Thanks for your script. But

I got an issue, I was going to install Intel RealSense SDK core installer but your automation.py does not capture any controls of this window. I got this error message

This is the windows I need to interact

I guess it includes character ™ and ®so it does not work.

Any plans to support installing as a library, with pip install?

WPF 界面框架程序，我使用window.ButtonControl(AutomationID="XXX",ClassName="XXX")进行元素定位，然后进行Click() 实际操作时点击的位置并不是我要定位的按键位置

Hi,

I am trying to set the module on Windows 10. I am running a python script to open the Notepad and with admin rights, but i received the below log. Am I missing something that needs to be done with the DLL files of the library?

AutomationClientX86.dll
AutomationClientX64.dll

Are these files already present or should be manually included in the system path?

Traceback (most recent call last):
  File "automate.py", line 2, in <module>
    import uiautomation as automation
  File "C:\Users\anant.pande\AppData\Roaming\Python\Python35\site-packages\uiautomation.py", line 55, in <module>
    _automationClient = _AutomationClient()
  File "C:\Users\anant.pande\AppData\Roaming\Python\Python35\site-packages\uiautomation.py", line 45, in __init__
    self.dll = ctypes.cdll.UIAutomationClientX64
  File "<C:\Program Files\Anaconda3\lib\ctypes\__init__.py>", line 417, in __getattr__
    dll = self._dlltype(name)
  File "<C:\Program Files\Anaconda3\lib\ctypes\__init__.py>", line 347, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
Exception ignored in: <bound method _AutomationClient.__del__ of <uiautomation._AutomationClient object at 0x000002BF64E952E8>>
Traceback (most recent call last):
  File "C:\Users\anant.pande\AppData\Roaming\Python\Python35\site-packages\uiautomation.py", line 52, in __del__
    self.dll.ReleaseInstance()
AttributeError: '_AutomationClient' object has no attribute 'dll' 

我的使用场景是报税，里面很多表格大概几十张，每个表格里面单元格也有80多个，填写下来很慢。我想快速填写该如何修改。先谢谢尹老哥哥了。

场景：打开360杀毒，点击更新。
版本：win10 32
详细描述：可以打开360杀毒主窗口，并且识别到主窗口的Lt, Rt, Lb, Rb，但是点击“检查更新”（日志和设置也测试过），鼠标没有反应。
上述操作在 win10 64 和win8 64 上均可以正常识别并点击。
请问大大，这个问题大概是什么原因？很久不得其解。

我的使用场景是用脚本检测某个窗口是否在最前面，如果不在，就检查是否被最小化了，如果不是被最小化，就起一个新的窗口。如果已有窗口，就进行下一步活动。

现在我已经可以检查出窗口被最小化了，但没有办法调到最前面。
那个窗体是个Qt5的pannel

无论我用panel.Show() 还是 panel.SetFocus()还是一起使用都不能恢复窗体，请问是不是我的使用方法不对？