[WayBack] Top 45 Best Automation Testing Tools Ultimate List • Test Automation Made Easy: Tools, Tips & Training Awesomeness A few notes, partially related to Enable your device for development – UWP app developer | Microsoft Docs.
One research project I had a while ago, was to see how it was possible to use a generic testing tool like Katalon Studio (that can test many platforms), for testing Windows applications.
Katalon uses a Selenium interface that normally is used for web applications through the [WayBack] WebDriver protocol (formerly [WayBack] JsonWireProtocol) that continues to be updated: [WayBack] draft upcoming WebDriver protocol, which is more generic than just web applications:
- To test mobile apps, you can use [WayBack] Appium: Mobile App Automation Made Awesome.
- To test Windows applications, you can use the – also open source – WinAppDriver, which is installed as part of Appium.
Adding the WebDriver protocol was basically the biggest change between Selenium 1.0 and Selenium 2.0: [WayBack] Selenium WebDriver
WebDriver is a remote control interface that enables introspection and control of user agents. It provides a platform- and language-neutral wire protocol as a way for out-of-process programs to remotely instruct the behavior of web browsers.
Mor information on WebDriver at [WayBack] GitHub – w3c/webdriver: Remote control interface that enables introspection and control of user agents.
WinAppDriver is at [WayBack] GitHub – Microsoft/WinAppDriver: Windows Application Driver
Finding your way is not easy, as for instance “SetCapabilities” “WinAppDriver” – Google Search returns hardly anything, it af all useful.
By now I know that should have been “SetCapability” “WinAppDriver” – Google Search, which for a word-blind person like me, looks identical, but is not.
A few notes if you want to go the same way:
DesiredCapabilitiesandSetCapability:- Not sure yet, but on some levels,
DesiredCapabilitiesseems to be deprecated (more on that below the fold) - [WayBack] WinAppDriver/Session.cs at master · Microsoft/WinAppDriver · GitHub and[WayBack] WinAppDriver/TestBase.cs at master · Microsoft/WinAppDriver · GitHub have a great set of test cases showing all the Windows supported capabilities
- Many more tests at [WayBack] WinAppDriver/Tests/WebDriverAPI at master · Microsoft/WinAppDriver · GitHub.
- [WayBack] Windows – Appium has more tips on capabilities
- [WayBack] WinAppDriver/README.md at master · Microsoft/WinAppDriver · GitHub has the WinAppDriver Capabilities:
Capabilities Descriptions Example appApplication identifier or executable full path Microsoft.MicrosoftEdge_8wekyb3d8bbwe!MicrosoftEdgeappArgumentsApplication launch arguments https://github.com/Microsoft/WinAppDriverappTopLevelWindowExisting application top level window to attach to 0xB822E2appWorkingDirApplication working directory (Classic apps only) C:\TempplatformNameTarget platform name WindowsplatformVersionTarget platform version 1.0 - [WayBack] Desired Capabilities – Appium has a long list of capability names, of which these are the most important:
Capability Description Values platformNameWhich mobile OS platform to use iOS,Android, orFirefoxOSplatformVersionMobile OS version e.g., 7.1,4.4deviceNameThe kind of mobile device or emulator to use iPhone Simulator,iPad Simulator,iPhone Retina 4-inch,Android Emulator,Galaxy S4, etc…. On iOS, this should be one of the valid devices returned by instruments withinstruments -s devices. On Android this capability is currently ignored, though it remains required.appThe absolute local path or remote http URL to a .ipafile (IOS),.appfolder (IOS Simulator),.apkfile (Android) or.apksfile (Android App Bundle), or a.zipfile containing one of these (for .app, the .app folder must be the root of the zip file). Appium will attempt to install this app binary on the appropriate device first. Note that this capability is not required for Android if you specifyappPackageandappActivitycapabilities (see below). Incompatible withbrowserName. See here about.apksfile./abs/path/to/my.apkorhttp://myapp.com/app.ipanewCommandTimeoutHow long (in seconds) Appium will wait for a new command from the client before assuming the client quit and ending the session e.g. 60
- Not sure yet, but on some levels,
- .NET WinForms is supported by WinAppDriver, as this is solved: [WayBack] WinAppDriver does not work with WinForms applications · Issue #108 · Microsoft/WinAppDriver · GitHub: WPF and WinForms applications are supported by Windows Application Driver. Finding the element in those apps may differ but is consistently based on the following attributes:
Locator Strategy Client API Matched Attribute accessibility id FindElementByAccessibilityId AutomationId class name FindElementByClassName ClassName id FindElementById RuntimeId (decimal) name FindElementByName Name tag name FindElementByTagName LocalizedControlType (upper camel case) Using the inspect tool or page source, you can identify and use any suitable attribute to locate your element based on the rules above.
- .NET too has a reference implementation in [Archive.is] Accessible.cs and this demo:
- Delphi is a totally different story, as it does not expose the automation interface that WebAppDriver expects: it needs
IAccessibleimplementations. - [WayBack] Poll: are you using WinAppDriver to automate Win32, WinForms, or WPF applications? · Issue #369 · Microsoft/WinAppDriver · GitHub
- [WayBack] WinAppDriver：Windows Application Driver
IAccessible:- is all because of [WayBack] Microsoft accessibility : Microsoft believes accessibility and inclusion are essential to delivering on our mission to empower everyone, everywhere.
- Documentation:
- From Delphi:
- [WayBack] Using the MSAA IAccessible interface within a Delphi Application | Steve Trefethen
- [WayBack] accessibility – Creating Accessible UI components in Delphi – Stack Overflow
The VCL itself does not natively implement any support for MSAA. Windows provides default implementations for standard UI controls, which many standard VCL components wrap. If you need more MSAA support than Windows provides, you will have to implement the
IAccessibleinterface yourself, and then have your control respond to theWM_GETOBJECTmessage so it can return a pointer to an instance of your implementation.Update: For example, one way to add MSAA to an existing
TEdit(if you do not want to derive your own component) might look something like this:First answer uses
ObjectFromLresultSecond answer uses
AccessibleObjectFromWindowvia [WayBack] Delphi Components and Screen Readers – Stack Overflow
Be aware:
[WayBack] Memory Leak Using Windows Accessibility Tools (Microsoft Narrator, Windows Speech Recognition) With .NET Applications
- [WayBack] Delphi IAccessible Get_accState influences Get_accName? – Stack Overflow
- [WayBack] winforms – IAccessible Implementation – Accessible Name only in Control Window? – Stack Overflow
- uses
CreateStdAccessibleObject
- uses
- [WayBack] delphi – IAccessible: Get Active url with Access violation – Stack Overflow
- [WayBack] How to use
AccessibleObjectFromWindowWinAPI function in Delphi? – Stack Overflow- [WayBack] AccessibleObjectFromEvent function | Microsoft Docs: Retrieves the address of the
IAccessibleinterface for the object that generated the event that is currently being processed by the client’s event hook function.
- [WayBack] AccessibleObjectFromEvent function | Microsoft Docs: Retrieves the address of the
- [WayBack] CreateStdAccessibleObject function | Microsoft Docs: Creates an accessible object with the methods and properties of the specified type of system-provided user interface element.
- [WayBack] CreateStdAccessibleProxyA function | Microsoft Docs: Creates an accessible object that has the properties and methods of the specified class of system-provided user interface element.
- [WayBack] CreateStdAccessibleProxyW function | Microsoft Docs: Creates an accessible object that has the properties and methods of the specified class of system-provided user interface element.
- [WayBack] How WM_GETOBJECT Works – Windows applications | Microsoft Docs: Microsoft Active Accessibility sends the
WM_GETOBJECTmessage to the appropriate server application when a client calls one of theAccessibleObjectFromXfunctions. - [WayBack] Retrieving an IAccessible Object – Windows applications | Microsoft Docs:Microsoft Active Accessibility provides functions such as AccessibleObjectFromWindow and AccessibleObjectFromPoint that allow clients to retrieve accessible objects.
When a client calls
AccessibleObjectFromWindowor any of the otherAccessibleObjectFromXfunctions that retrieve an interface to an object, Microsoft Active Accessibility sends theWM_GETOBJECTwindow message to the applicable window procedure within the appropriate application. To provide information to clients, servers must respond to theWM_GETOBJECTmessage.- [WayBack] AccessibleObjectFromEvent function | Microsoft Docs:Retrieves the address of the
IAccessibleinterface for the object that generated the event that is currently being processed by the client’s event hook function. - [WayBack] AccessibleObjectFromWindow function | Microsoft Docs:Retrieves the address of the specified interface for the object associated with the specified window.
- [WayBack] AccessibleObjectFromPoint function | Microsoft Docs:Retrieves the address of the IAccessible interface pointer for the object displayed at a specified point on the screen.
- [WayBack] AccessibleObjectFromEvent function | Microsoft Docs:Retrieves the address of the
- [WayBack] LresultFromObject function | Microsoft Docs: Returns a reference, similar to a handle, to the specified object. Servers return this reference when handling
WM_GETOBJECT.
Return code Description E_INVALIDARG One or more arguments are not valid. E_NOINTERFACE The object specified in the pAccparameter does not support the interface specified in theriidparameter.E_OUTOFMEMORY Insufficient memory to store the object reference. E_UNEXPECTED An unexpected error occurred. - [WayBack] ObjectFromLresult function | Microsoft Docs: Retrieves a requested interface pointer for an accessible object based on a previously generated object reference.
- From .NET WinForms:
- Control properties
- [WayBack] Control.AccessibilityObject Property (System.Windows.Forms) | Microsoft Docs
- [WayBack] Control.AccessibleDefaultActionDescription Property (System.Windows.Forms) | Microsoft Docs
- [WayBack] Control.AccessibleDescription Property (System.Windows.Forms) | Microsoft Docs
- [WayBack] Control.AccessibleName Property (System.Windows.Forms) | Microsoft Docs
- [WayBack] Control.AccessibleRole Property (System.Windows.Forms) | Microsoft Docs
- [WayBack] Control.Name Property (System.Windows.Forms) | Microsoft Docs
- MSAA counterparts are below (see Content of Descriptive Properties)
- [WayBack] Common approaches for enhancing the programmatic accessibility of your Win32, WinForms and WPF apps: Part 3 – WinForms – Microsoft Windows UI Automation Blog
- [WayBack] Few tips on implementing a Coded UI Test Plugin Extension – Microsoft DevOps Blog
- Control properties
- WinAppDriver
- [WayBack] Best patter for waiting for a condition. · Issue #187 · Microsoft/WinAppDriver · GitHub
- [WayBack] WinAppDriver Timing Out after Delphi 11 application launched · Issue #131 · Microsoft/WinAppDriver · GitHub
- [WayBack;Translated] Windows 10 : Microsoft publie en open source WinAppDriver UI Recorder, pour simplifier l’automatisation des tests d’interface utilisateur
- [WayBack] T687329 – Support WinForms UI testing with Appium / WinAppDriver | DevExpress Support Center
- [WayBack] Windows Application Driver for PC integrates with Appium – Windows Developer BlogWindows Developer Blog
- [WayBack] WinAppDriver Archives – Windows Admins
Of course this is not the only way to test applications; for alternatives (including commercial ones):
- Comparison of GUI testing tools – Wikipedia
- [WayBack] Top 45 Best Automation Testing Tools Ultimate List • Test Automation Made Easy: Tools, Tips & Training Awesomeness
- Especially read the entries on Appium, Selenium, WinAppDriver, Gauge, Katalon, WebDriver.IO, Test.AI, AppDiff, AutoIt,
- [WayBack] Ranorex – SOFTWARE TESTING TOOLS commercial, but good; see for instance these links:
- [WayBack] Roald van Doorn på Twitter: “Integrating Ranorex with TeamCity using PsExec”
- [WayBack] T406360 – Ranorex UI test of dxRibbon | DevExpress Support Center
- Solution: create helper classes for the ribbon and all it’s child components that implements the
IAccessibleinterface. You will need to specify everything needed for this interface, but it will work with Ranorex and it’s Spy tool.
- Solution: create helper classes for the ribbon and all it’s child components that implements the
- [WayBack] Continuous delivery with Legacy VCL Applications – Entwickler KonferenzA case study of how we applied CD principles to an older VCL application. We will take a look at the challenges we faced and the solutions we chose, the frameworks we use, release procedures and feedback loops. We will demonstrate how we safely build and deploy the windows software for Albelli and Vistaprint, and the benefits this brings to our team and our organization.Outline:
- Automating your builds using TeamCity
- Unit testing using DUnitX
- Automated UI tests using Ranorex and Specflow
- Deploy to different environments with ProGet and Octopus Deploy
- Increased speed of value to customer (reduced stock)
- Increased feedback to developers
- [WayBack] Ranorex Joins the Idera Family – unlike the Delphi teams, Idera has not yet reformed Ranorex, so for now that is a good sign
- [WayBack] Bogdan Polak on Twitter: “Great session @danieleteti about Automated Tests run by pyTest at #ITDevCon #itdevcon9 testing #delphi apps. Small comment about Ranorex and VCL. It’s possible to implement this: https://t.co/d0zbYa2ePi”
- [WayBack] We are searching for some Automation Testing Framework, different from Test Complate… Any ideas? We are trying to select a tool for automating an ap… – Avatarx – Google+
- [WayBack] I’m using Tokyo here at work to develop our software. The rest of our team is still using XE5. Our build is still in XE5. A QA team member came to m… – Phillip Woon – Google+
–jeroen
