Qat (Qt Application Tester)

Description

Qat is a testing framework for Qt-based applications.

Qat provides a Python API to interact with any existing Qt application by accessing QML/QtQuick/QWidget elements and simulating user manipulations.

It is also integrated to behave to support Behavior-Driven Development (BDD) with the Gherkin language.

Although Qat uses the GUI to interact with the tested application, it is oriented towards BDD and functional testing rather than pure UI or non-regression testing.

The main objective of Qat is to provide quick feedback to developers and easy integration to build systems.

Installation

Qat can be installed with Pip:

pip install qat

This will also install the following dependencies:

  • behave (for BDD testing)

  • sv_ttk (for GUI)

  • tkinter-tooltip (for GUI)

  • xmlschema (for test report)

Requirements

Qat requires Python >= 3.9.

Tested applications don’t need to be modified but they must be compiled in release mode with a compatible compiler and dynamically-linked Qt version:

Qt

Linux (gcc)

Windows (MSVC)

5.15 (LTS)

yes

yes

6.0

no

no

6.1

yes1

yes1

6.2 (LTS)

yes

yes

6.3

yes

yes

6.4

yes

yes

6.5 (LTS)

yes

yes

6.6

yes

yes

1 Touch screen events are only partially supported with Qt 6.1

Setup

The recommended approach is to install VSCode with the following extensions:

  • Python

  • Pylance

  • Test Explorer UI

  • Behave VSC (for BDD)

Any Python (+ Gherkin) environment can be used (PyCharm, Eclipse/Pydev, VSCode, …) but the present documentation will use VSCode as a tutorial.

Usage

Creating a test

Once Qat is installed and setup is completed, navigate to the folder where you want to store your tests. From there, open a command prompt and run:

qat-create-suite

That will generate all the files to run a demo BDD test.

If you prefer using a pure Python test, run:

qat-create-suite script

Then you need to register the application you want to test. The easiest way is to use the Qat GUI, but you can also use the Python API. In the current test folder, run the following command:

qat-gui

This will open the Application Manager window: "App Mgr"

Click on "Add app" and enter an application name in the top combobox, then the application path and arguments in the fields below. "App Mgr"

Click on "Save" to register the application.

Now you can close the window and launch VSCode:

code .

Running a test

The Python demo can be run like any other script: open demo.py in VSCode then click on Run Python file. For real tests, it is recommended to use a test framework such as Pytest or Unittest.

The BDD demo can be launched from the Test Explorer: "VSCode BDD demo"

In both cases, the demo will verify that an application has been registered (see previous steps), start it and verify that the main window is opened. An XML report will also be generated.

When the demo is successful, it confirms that Qat is properly working on your machine. Otherwise, please refer to the test itself for indications on how to solve the issue.

Configuring test execution

Test settings are available in testSettings.py. The easiest way to change these values is to add a testSettings.json file to the root of your test suite.

This file can contain the following parameters:

{
   "waitForObjectTimeout": 3000,
   "waitForAppStartTimeout": 60000,
   "waitForAppStopTimeout": 30000,
   "longOperationTimeout": 10000,
   "screenshotOnFail": true,
   "continueAfterFail": false,
   "lockUI": "auto"
}

If a key is missing, the default value will be used. During execution, values are available in the Settings class.

If you need to access those values from a script, you can add the following import to your script:

from qat import Settings

# Increase timeout value when finding objects
Settings.wait_for_object_timeout = 5000

All timeouts are in milliseconds.

screenshotOnFail determines whether a screenshot of the application is taken and added to the report after each failure. Default is True.

continueAfterFail determines whether the test execution should continue after a step has failed (BDD only). Default is False

lockUI determines when Qat will lock the application’s interface so that it ignores user inputs. Can be one of “always”, “never” or “auto” (default, application will not be locked when debugging a test script). Locking the UI allows users to continue working when a test is running, without affecting the test execution.

For more details on API functions please refer to the Python API reference.

You can also explore the tutorials.

Support

You can report any bug, question or feature request by creating a Gitlab issue from this page.

For details, see the issues workflow.

Contributing

Contributions are welcome and accepted through Merge Requests. Please refer to Contributing for detailed instructions.

Authors and acknowledgment

The complete list of authors and contributors is available here.

Qat is built on Gitlab using Docker images provided by Luca Carlon

License

Qat is licensed under the MIT License.

Qat contains a copy of Niels Lohmann’s json library which is licensed under the MIT License.

Qat uses components from the Qt framework which is licensed under the LGPLv3 License. Please note that Qat does not distribute any Qt binary: it uses DLL injection to dynamically link to the Qt binaries used by the target application.

Qat uses the Sun Valley Theme for ttk under the MIT License.