The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 4,177 other subscribers

RaiMan’s SikuliX: Automate what you see on a computer monitor

Posted by jpluimers on 2018/09/05

On my research list:

Automate what you see on a computer monitor

Source: [WayBackRaiMan’s SikuliX


It is an evolution of [WayBackSikuli Script – Home that has an other fork that can be automated with PowerPoint slides:

I should play with it: [WayBackSikuliX – QUICKSTART

Via: [WayBack] Any recommendations of automation tools for GUI testing.We tried AutoIT but it had some problems and way too technical… – Tommi Prami – Google+


[Wayback] More details:

SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI’s internals or the source code of the application or web page you want to act on.

SikuliX supports as scripting languages

  • Python language level 2.7 (supported by Jython)
  • running RobotFramework text-scripts is supported (see docs)
  • Ruby language level 1.9 and 2.0 (supported by JRuby)
  • JavaScript (supported by the Java Scripting Engine)

… and you can use it in Java programming and programming/scripting with any Java aware programming/scripting language (Jython, JRuby, Scala, Clojure, …).

Though SikuliX is currently not available on any mobile device, it can be used with the respective emulators on a desktop computer or based on VNC solutions. A solution for Android devices based on ADB (Android Debugging Bridge) is in an early experimental stage,

Besides locating images on a screen SikuliX can run the mouse and the keyboard to interact with the identified GUI elements. This is available for multi monitor environments and even for remote systems with some restrictions.

SikuliX comes with basic text recognition (OCR) and can be used to search text in images. This feature is powered by Tesseract.

SikuliX is a Java application, that works on Windows XP+, Mac 10.6+ and most Linux/Unix systems. For Windows, Mac and Ubuntu 12.04+ it is complete and should normally work out of the box. For Linux/Unix systems there are a few prerequisites to be setup.

SikuliX is the follow up of Sikuli Script – last version 2011 as Sikuli-X-1.0rc3. Though it is no longer supported: here you find valuable information for the installation.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: