# Architecture ## Overview Qat utilizes DLL injection to load a library into the target application. Once injected, this library initiates a TCP server, exposing all Qt objects via custom requests. The Python client leverages this server to offer advanced features such as object discovery, property access, and event simulation. ### Table of contents - [Overview](#overview) - [DLL injection](#dll-injection) - [TCP server](#tcp-server) - [Plugins](#plugins) - [Project hierarchy](#project-hierarchy) - [Injector](#injector) - [DLL injector application (Windows only)](#dll-injector-application-windows-only) - [Injector library](#injector-library) - [Server](#server) - [_ActionCommandExecutor_](#actioncommandexecutor) - [_CallCommandExecutor_](#callcommandexecutor) - [_CommCommandExecutor_](#commcommandexecutor) - [_FindCommandExecutor_](#findcommandexecutor) - [_GestureCommandExecutor_](#gesturecommandexecutor) - [_GetCommandExecutor_](#getcommandexecutor) - [_KeyboardCommandExecutor_](#keyboardcommandexecutor) - [_ListCommandExecutor_](#listcommandexecutor) - [_MouseCommandExecutor_](#mousecommandexecutor) - [_SetCommandExecutor_](#setcommandexecutor) - [_TouchCommandExecutor_](#touchcommandexecutor) - [_IObjectPicker_](#iobjectpicker) - [_Images_](#images) - [Client](#client) - [Public API](#public-api) - [Internal implementation](#internal-implementation) - [GUI](#gui) - [Tests](#tests) - [test/QmlApp](#testqmlapp) - [test/WidgetApp](#testwidgetapp) - [test/pytests](#testpytests) - [test/behave](#testbehave) - [Configuration and build system](#configuration-and-build-system) - [Tool integration](#tool-integration) - [Documentation](#documentation) - [Packaging](#packaging) ### DLL injection DLL injection is a commonly used technique on various operating systems. For Windows, detailed information can be found on [this site](https://www.apriorit.com/dev-blog/679-windows-dll-injection-for-api-hooks). Essentially, a process can create a thread in another process, which can then load any DLL into the target process using the *LoadLibrary* function. On linux, Qat utilizes the ***LD_PRELOAD*** environment variable to load specified libraries before others when an application starts. In both cases, the injected library determines the Qt version of the application and dynamically loads the corresponding Qat server library using *LoadLibrary* or *dlopen*. The server library then initiates a TCP server on an available port, enabling any process to communicate with the tested application. ### TCP server The server employs a *QTcpServer*, initiated upon library loading. It listens on any available port on the local host, with each application instance utilizing a distinct port number. This enables the Qat client API to connect to multiple applications concurrently. Upon starting, the server writes the port number and current process ID to a text file for client identification. Leveraging the *Qt Meta Object* framework, the server exposes all objects and their properties. ### Plugins Qt applications may use different widget frameworks, such as *QWidgets* and *QtQuick (QML)*, and interact with native widgets. Depending on the application and platform, some libraries may not be available at runtime, potentially causing the Qat library to fail to load. For example a *QWidget*-based application may not redistribute any *QtQuick* library. To address this, Qat employs a plugin architecture that dynamically loads available frameworks at runtime. Each plugin provides a unified abstraction layer over the supported framework. Currently, there are three plugins: *QWidget*, *Qml*, and *WindowsNative*: - **QWidget**: Supports all *QWidgets* as well as virtual widgets allowing interaction with the Model/View framework and Menu elements. - **Qml**: Supports *QtQuick* widgets, including _Qt3d_ elements. - **WindowsNative**: Provides virtual widgets wrapping native Windows components using the *WinAPI*. Mostly used for native dialogs. Each plugin is also responsible to implement an *ObjectPicker* to be used with the **Spy** user interface. The *ObjectPicker* allows to highlight and select widgets from the application's UI hierarchy. ## Project hierarchy ### Injector #### DLL injector application (Windows only) The (small) process used to inject the Injector library into the tested application. It is composed of a single _main_exec.cpp_ file and uses the Windows API to find the main window of the application then creating a remote thread to inject the Qat libraries. > For easier packaging / distribution, it is important that this program does not depend on any other library. #### Injector library This library has one implementation for each supported platform. Once loaded by the *injector* application or the *LD_PRELOAD* mechanism, it calls the ***qVersion()*** function that is part of every Qt application. Based on this version, it will then load the corresponding **Server** library. ### Server The library containing the TCP server and all the code required to execute client's requests. The class hierarchy is as follows: !["Server"](../images/Server.png "Server") Requests are in Json format, with a small header containing the size of the request. The _ICommandExecutor_ interface is implemented by various classes that handle different types of requests. All _CommandExecutors_ inherit from _BaseCommandExecutor_ which provides functions to find objects and to store them in a cache when needed: !["Command Executors"](../images/CommandExecutors.png "Command executors") Since the Server depends on Qt libraries, it must be built for each supported Qt version. #### _ActionCommandExecutor_ Enables or disables the *ObjectPicker*, takes screenshots of applications and widgets, and locks or unlocks the application. #### _CallCommandExecutor_ Calls a method on the requested object through Qt's _MetaObject_ system. The logic of parsing and converting the arguments and calling the method is performed by the _MethodCaller_ class. The conversion between Json and QVariant is provided by functions in _QVariantToJson_. #### _CommCommandExecutor_ Initializes and terminates the TCP communication from the library to the Python API. Establishes or destroys connections between Qt's signals and Python callbacks (used for connections and bindings in the API). #### _FindCommandExecutor_ Uses the _findObject()_ method of _BaseCommandExecutor_ to find the object corresponding to the given definition. #### _GestureCommandExecutor_ Simulates native gesture events like pinch and flick. This is for advanced use as gestures are usually simulated with the *TouchCommandExecutor* (see below). #### _GetCommandExecutor_ Returns the value of the requested property of an object. Also handles "special" properties (i.e that are not part of the _MetaObject_ system) such as: parent, children, id, ... #### _KeyboardCommandExecutor_ Simulates typing and shortcut events and sends them to the Qt window containing the requested object. Object will be given the _Active Focus_ too. > Note: Some events will not be sent properly if the main window is in the background due to OS restrictions. > > Note: If an event was not accepted by any object, an error will be returned. #### _ListCommandExecutor_ Returns the list of all supported methods or properties of the requested object, along with their current values. Can also return other global elements such as the top-level windows or current Qt version. #### _MouseCommandExecutor_ Simulates mouse events and sends them to the Qt window containing the requested object. > Note: If an event was not accepted by any object, an error will be returned. #### _SetCommandExecutor_ Changes the value of the requested property of an object. Will return an error if the property could not be changed (e.g. property is read-only or final value does not correspond to the requested one). #### _TouchCommandExecutor_ Simulates touch events and sends them to the Qt window containing the requested object. > Note: If an event was not accepted by any object, an error will be returned. #### _IObjectPicker_ Interface for all *ObjectPicker* implementations. Implementations are provided by plugins for each supported widget framework. When enabled, an *ObjectPicker* will highlight the widget currently under the mouse cursor by displaying a semi-transparent rectangle overlay above it. It will also show the object's type and _objectName_ (if available) in a tooltip. The object under the mouse cursor is identified by using functions from _WidgetLocator_. When the highlighted object is clicked (without holding the CTRL key) it is stored in the *pickedObject* property of the *ObjectPicker*. The Python API can then access this object and all its properties. #### _Images_ When a screenshot of a widget is taken with the *grab_screenshot* API function, a *QImage* object is created that contains the image data of this screenshot. Since *QImage* does not inherit from *QObject*, it cannot be accessed through the *MetaObject* system. Qat uses custom wrappers to expose properties and functions of an Image: *ImageWrapper*. These wrappers are stored in a *static* queue with a maximum size of 10 to avoid using too much memory in the tested application. So the 11th screenshot will replace the 1st one, the 12th screenshot will replace the 2nd one, and so on. Old screenshots are not lost though, since their contents are saved to a temporary file and can be automatically reloaded when needed. ### Client The Python client communicating with the Qat server. It is the main API of Qat. It can be used for Python and BDD tests (with *Behave* for example). #### Public API The public API is composed of the following files: * __qat.py__: contains all the public API functions. The implementation of each function is delegated to multiple internal files (see below). * __qt_types.py__: contains classes corresponding to Qt common types such as *QColor*, *QFont*, *QRectangle*, etc. This allows Python scripts to use these types as method arguments or property values. * __report.py__: provides functions to add entries to the test report. * __test_settings.py__: Loads setting from the *testSettings.json* file in the current folder and make values available to Python scripts. * __bdd_hooks.py__: contains functions intended to be called by *environment.py* when using ***Behave***. The main role of these hooks is to generate the general structure of the XML test report: open/close XML tags for each test, feature, scenario and step, and add appropriate timestamps. #### Internal implementation The API functions defined in _qat.py_ are implemented by internal files. The calls are grouped by category in separate files in the ***internal*** subfolder: 1. __app_launcher.py__: Functions to start, close or attach to an application. 2. __application_context.py__: Defines the *ApplicationContext* class that handles the lifecycle of an application. 3. __binding.py__: Implements the *Binding* class that handles connections and callbacks between C++ properties and Python variables. 4. __communication_operations.py__: 5. __debug_operations__: Functions to manipulate the *ObjectPicker* from Python code. Intended for testing and debug purposes since the *ObjectPicker* is normally used with the corresponding Spy GUI. Also provides functions to lock and unlock the GUI. 6. __find_object.py__: All functions used to locate objects: _wait_for_object*()_, *find_all_objects()*, *list_top_windows()*, ... 7. __gesture_operations.py__: Provides high-level gesture functions such as *pinch()*, *flick()*, etc. 8. __keyboard_operations.py__: Contains the keyboard and shortcut functions. 9. __mouse_operations.py__: Contains the *click()* and *drag()* functions that are used by all mouse-related API calls. Also defines the *Button* and *Modifier* enums. 10. __screenshot_operations.py__: Contains the *takeScreenshot()* and *grabScreenshot()* functions. 11. __synchronization.py__: Contains the *wait_for_property()* function used by API functions such as *wait_for_property_value()* and *wait_for_property_change()*. Other internal features are implemented in various modules of the **internal** subfolder: * __tcp_client.py__: Connects to Qat server and sends json commands through TCP. * __tcp_server.py__: Opens a TCP server to handle callbacks from C++ bindings. * __global_constants.py__: Contains default constant values (e.g. timeouts) * __xml_report.py__: Asynchronously writes a test report in XML format. Mostly used by BDD hooks. * __preferences.py__: Reads and writes the *preferences.json* file from/to the user's personal folder. * __qt_object.py__: Defines the *QtObject* class. QtObjects are the ones returned by _wait_for_object\[exists]()_ for example. Each object contains a copy of its original definition and a reference to its *ApplicationContext*. The special functions \_\__getattr\_\__ and \_\__setattr\_\__ use the *TcpClient* to access remote "attributes". So if *object* is a QtObject, *object.property* will return the *property* value of this *object* in the tested application. The returned property can be a native value for simple types (e.g. bool, int, array...), a *QtCustomObject* for complex types (e.g QVector3D, QColor,...) or another *QtObject*. > Note that __\_\_getattr\_\___ is also called by Python when a *method* is used. In this case it returns a *QtMethod* object. * __qt_custom_object.py__: Defines the *QtCustomObject* class. This class represents Qt objects that have some members which are not available through the _MetaObject_ system. For example, *QColor* has __red__, __green__ and __blue__ members, *QVector3D* has __x__, __y__ and __z__ members, etc. For a complete list of supported types, refer to _QVariantToJson.cc_. *QtCustomObject* uses custom serialization and holds the values of the underlying object at the time of its construction. In other words, these objects are "_const_" and changes on the remote object will not be reflected automatically. For example if the __x__ component of a *QVector3D* changes in the tested application, calling *someVector.x* in Python will return the initial value, not the current one. To retrieve the current value, the entire *QtCustomObject* must be requested again. * __qt_method.py__: Defines the *QtMethod* class. The special method _\_\_call\_\__ is overridden so when any method is called on a *QtObject*, it will be redirected to the tested application. The method will then be executed by the *Qt MetaObject* system and the result returned as a native value(e.g. int, string, bool, float, list), a *QtCustomObject* or a *QtObject*. > Note: not all methods are supported: they must have at most 10 arguments, and each argument must be convertible to/from *QVariant*. ### GUI The GUI is based on [*CustomTkinter*](https://github.com/TomSchimansky/CustomTkinter). All source files are in the *client/qat/gui* folder. It is composed of the *ApplicationManager* and the *Spy* windows. The *launcher.py* file allows users to open the GUI by typing the "qat-gui" command in a terminal. ### Tests #### test/QmlApp QmlApp is a QML-based application containing various widgets and used by unit and BDD tests of the Qat project. #### test/WidgetApp WidgetApp is a QWidget-based application containing various widgets and used by unit and BDD tests of the Qat project. #### test/pytests Unit tests of this project, executed with Py.Test. Tests are executed with both *QmlApp* and *QWidgetApp* and for each supported Qt version. #### test/behave Some BDD tests examples to demonstrate the integration of QAT with the Behave framework. ### Configuration and build system The build system is hosted on ***Gitlab***. The pipelines are defined in *.gitlab-ci.yml* at the root of the project. Other specific configuration files are available in the *.gitlab* folder. #### Tool integration Configuration files used by other development tools (*Pycov*, *Pylint*, *Doxygen*, etc) are in the *config* folder. ### Documentation In addition to *Doxygen* and Python *docstrings*, all documentation can be found in the *doc* folder. During build, it is also published to [Readthedocs](https://qat.readthedocs.io/en/latest) by *Sphinx*. Related configuration is is the *sphinx* folder. ### Packaging Qat packages are generated and uploaded to [PyPI](https://pypi.org/) by a *Gitlab* pipeline when a tag is created in the repository. Packages include the *template* folder which contains example tests and configurations. These are used by the *qat-create* command to create new test suites on the user's computer.