Tag Archives: windows

Automating the Windows 10 Desktop Calculator using PyAutoGui

Last year, I’ve tried automating a Windows 7 desktop calculator using a Java library called Winium. A year later, someone from the comments told me that the example didn’t work on Windows 10. I ignored the comment for a while because I didn’t have a Windows 10 machine to perform a test back then, but now that I’ve recently upgraded my home PC I decided to try it out.

What I found:

Running a Windows 7 desktop calculator automation example (using Winium) on a Windows 10 machine

The Windows 10 calculator opened up but the tests didn’t run properly. The error log told me that the program was unable to find the calculator elements it was supposed to click. Bummer. Maybe the names of the calculator elements were different, Windows 10 versus Windows 7? It wasn’t; the element names were still the same according to UI Spy. Perhaps there’s something from Winium that can point me to a clue? Oh, it seems that the library has not been updated in recent years.

If I can’t use Winium, how then can I automate the Windows 10 calculator? A Google search pointed me to PyAutoGui. Instead of Java, it says that I’ll need Python (and Pip) for this tool to work. And yes, being Windows, I also need to properly set the environment variables so I can use the Python and Pip commands on a terminal.

Let’s install PyAutoGui:

Installing PyAutoGui

And after writing some code, let’s see if we can automate the Windows 10 calculator with it:

Automating the Windows 10 desktop calculator with PyAutoGui. Click the image to view the GIF full-size on another browser tab 🙂

It works!

But here are some catches:

  • I had to rely on PyAutoGui’s keyboard control functions for performing the calculator actions, instead of finding elements via the user interface. Well, from what I’ve seen so far from the docs is that the only way to locate a UI element is by locating elements using screenshots. I tried that the first time and it was very flaky, so I opted for using the keyboard control functions instead.
  • The code introduces a time variable to wait for the calculator to appear on screen. The code also introduces a time variable to pause a portion of a second in-between each keyboard action so the actions don’t happen too fast for a person’s eye.
  • There are no assertions in the example code, because I couldn’t find any assertion functions I could use from the PyAutoGui docs. It is not a tool built for testing, only for automating desktop apps.

Source code for this experiment can be found on: Win-Calculator-PyAutoGui.

Running Makefile Tasks On Windows OS

In an ongoing software development project we are using Makefile tasks to make running long and repetitive commands as easy and as fun as possible to run. They’re like bash aliases, shortcuts to performing recurrent jobs we frequently have to do while writing new code or testing applications. For example, we could define a task that runs unit tests and code standard checks on an application running in a Docker container like so:

  test:
    echo "Checking application code with PSR2 standards ..."
    docker-compose exec -T php phpcs -v --standard=phpcs.xml ./app/src
    echo "Running unit tests ..."
    docker-compose exec -T php phpunit --colors=always --configuration ./app

and we would run the task with only the following command:

  make test

Cool, right? I don’t have to remember all the exact commands to do what I need to do. And even if I forget the right task name (in this case, make test) I can just run the make command in the CLI and I’ll be provided a list of the tasks that I can use for the project.

Now Makefile tasks will run on Unix terminals out of the box. For Windows however, we still have to do some setup before Makefile tasks can run. For my machine at work, I did the following:

  • Download and install GnuWin32
  • Go to the install folder C:\Program Files (x86)\GnuWin32\bin
  • Copy all files inside the bin folder to the root project directory (libiconv2.dll, libintl3.dll, make.exe)
  • Add the installation bin directory to the system environment variables Path

There are other tools that we can use to configure Makefile to run on Windows but this is a quick and easy way to do it. After that we can run make.exe test on the default cmd CLI but on some Unix-like terminals like the Docker Quickstart Terminal we can definitely use make test.

Favorite Talks from Selenium Conference Austin 2017

It’s amazing that the recently concluded Selenium Conference over at Austin Texas continues to live up to expectations, building up on the previous conferences and keeps delivering quality talks on automation and testing. And what’s more interesting is to know how they’ve been keeping up with everything with help from the Software Freedom Conservancy and the testing community. There’s even a European Selenium Conference happening on October 9-10, which I’m very much looking forward to.

Meanwhile, here are some of my favorite talks from the Austin conference:

  • Automate Windows and Mac Apps with the WebDriver Protocol (by Dan Cuellar, Yosef Durr, and Stuart Russell, about easily automating Windows and Mac apps using Appium)
  • Automating Restaurant POS with Selenium – A Case Study (by Jeffrey Payne, about automating a point-of-sale system for testing, including credit card readers, printers, cash drawers, and caller IDs)
  • Selenium State of the Union (by Simon Stewart, on an overview of the W3C spec process and about naming things in the Selenium project, how APIs should be correct and how people won’t write their own, how not everyone is a sophisticated developer, and how testing is under resourced)
  • Leverage your Load Tests with JMeter and Selenium Grid (by Christina Thalayasingam, on adding load tests for your system using JMeter and Selenium Grid in combination)
  • Selenium and the Software Freedom Conservancy (by Karen Sandler, on the safety and efficacy of proprietary medical devices and how often open-source software are more likely to be safer and better over time, and about what kind of organization the Software Freedom Conservancy is and how it helps open-source projects like Selenium continue to live on for the long-term)
  • Stop Inspecting, Start Glancing (by Dan Gilkerson, on automating web apps without looking at the DOM structure)
  • Transformative Culture (by Ashley Hunsberger, about moving from QA to Engineering Productivity and the culture changes necessary for getting better at testing)

Automating the Windows 7 Desktop Calculator’s UI using Winium

Ever since I learned how to use Selenium to automate browsers and web applications a few years back, I’ve wondered from time to time whether I can use the same process or technology to automate Windows desktop applications via its user interface. There’s no particular use case at work considering our apps are all in the web, but I challenged myself to find an answer only for curiosity’s sake. It didn’t turn out to be a thought-provoking experiment but at least it was somehow amusing.

Here’s what I got from the quick study:

Click the image to view the GIF full-size on another browser tab 🙂

And a few notes:

  • We can’t use Selenium to automate Windows desktop applications.
  • A quick search about automating Windows apps tells us that there’s actually a number of tools we can use for such purpose, some paid, others free. Other than Winium, I found out that we can also use Sikuli (or its successor SikuliX), WinTask, autoIT, TestStack.White, Automa, MacroScheduler, Cobra, or Pywinauto. There’s probably more tools out there. I just chose to use Winium because writing tests and automating the app with it is similar to how Selenium works, which means very little learning curve for me.
  • Source code for this short experiment can be found here: Win-Calculator. It uses Maven for handling installation of project dependencies while the test code is written in Java. The tests run locally, I’m not sure whether there’s a way to manipulate a Windows app on another machine using Winium. I have not researched that far.
  • Winium will actually take control of the machine’s mouse to move to the element locations set in the tests. This means that interrupting the position of the mouse while the tests are running would likely fail the test. This is annoying, and I’m not sure whether the other Windows app automation tools behave otherwise or the same.

Interesting Talks from Google Test Automation Conference 2016

There’s a lot of stuff going on in the software testing community at the moment, specifically in the field of automation, because of how software is now being deployed into various other platforms besides personal computers. Google needs to worry about testing their eyeglasses, virtual reality headsets, and cars. Others care about testing robots and televisions. This is why it is fun to watch talks from conferences, like the Selenium Conference or the recently concluded Google Test Automation Conference: I get to find out what problems they’re facing and see how they try to solve them, and maybe learn a thing or two. Sometimes I get to pick up a new tool to try for my own testing too, a great bonus.

Some favorite talks from the conference are: