RaiMan’s SikuliX: Automate what you see on a computer monitor
Posted by jpluimers on 2018/09/05
On my research list:
Automate what you see on a computer monitor
Source: [WayBack] RaiMan’s SikuliX
Repositories:
- Stable: https://github.com/RaiMan/SikuliX-2014
- Development: https://github.com/RaiMan/SikuliX2
It is an evolution of [WayBack] Sikuli Script – Home that has an other fork that can be automated with PowerPoint slides:
I should play with it: [WayBack] SikuliX – QUICKSTART
Via: [WayBack] Any recommendations of automation tools for GUI testing.We tried AutoIT but it had some problems and way too technical… – Tommi Prami – Google+
–jeroen
SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI’s internals or the source code of the application or web page you want to act on.
SikuliX supports as scripting languages
- Python language level 2.7 (supported by Jython)
- running RobotFramework text-scripts is supported (see docs)
- Ruby language level 1.9 and 2.0 (supported by JRuby)
- JavaScript (supported by the Java Scripting Engine)
… and you can use it in Java programming and programming/scripting with any Java aware programming/scripting language (Jython, JRuby, Scala, Clojure, …).
Though SikuliX is currently not available on any mobile device, it can be used with the respective emulators on a desktop computer or based on VNC solutions. A solution for Android devices based on ADB (Android Debugging Bridge) is in an early experimental stage,
Besides locating images on a screen SikuliX can run the mouse and the keyboard to interact with the identified GUI elements. This is available for multi monitor environments and even for remote systems with some restrictions.
SikuliX comes with basic text recognition (OCR) and can be used to search text in images. This feature is powered by Tesseract.
SikuliX is a Java application, that works on Windows XP+, Mac 10.6+ and most Linux/Unix systems. For Windows, Mac and Ubuntu 12.04+ it is complete and should normally work out of the box. For Linux/Unix systems there are a few prerequisites to be setup.
SikuliX is the follow up of Sikuli Script – last version 2011 as Sikuli-X-1.0rc3. Though it is no longer supported: here you find valuable information for the installation.
The Wiert Corner - irregular stream of stuff 
Leave a Reply