Synchronization techniques

In automated testing, timing issues are common and can significantly affect the reliability of test suites. These issues arise because modern applications handle multiple tasks simultaneously, like managing the user interface and backend processes. This can lead to situations where the application isn’t ready when the test expects it to be, causing tests to fail unexpectedly.

Timing problems come in various forms, such as elements not being ready for interaction, delays in data processing, or conflicts between different parts of the application.

Attempting to address timing issues by introducing fixed delays in tests isn’t reliable and can lead to inefficient or failing tests.

Resolving timing issues is vital for ensuring the reliability of test results. In this tutorial, we’ll explore how Qat can help mitigate these timing challenges. We’ll discuss techniques for making tests wait until the application is ready, monitoring specific changes in the application, and reacting accordingly. By utilizing these strategies, testers can create more dependable test suites, thereby instilling confidence in the testing process.

Table of contents

Object lifecycle

Managing the lifecycle of objects within the application under test is crucial for ensuring that tests interact with objects at the appropriate times.

Ensuring object accessibility

Qat offers a set of wait_for_*() functions that allow testers to synchronize test execution with the lifecycle of objects. These functions enable tests to wait for objects to reach specific states before proceeding with interactions or verifications.

The wait_for_object_exists() function ensures that an object is created before performing any verification on it. It waits until the specified object is found in the application’s UI hierarchy.

Additionally, the wait_for_object() function guarantees object accessibility, ensuring that both its visible and enabled properties are set to True before interacting with it. Action-related API functions, such as mouse_click() and type_in(), automatically invoke wait_for_object() before execution to ensure object readiness.

While the default behavior of wait_for_object() checks the visible and enabled properties, in some cases, other properties may dictate the object’s state. For example, a test may need to wait for a text field to initialize after its widget creation. In such cases, the wait_for() function can be used to wait for any desired condition:

text_box_definition = {
   'objectName': 'searchField',
   'type': 'Text'
}
text_box = qat.wait_for_object_exists(text_box_definition)

if not qat.wait_for(lambda: text_box.text != ""):
   raise Exception('Text box was not initialized')

To handle cases where objects may not reach the expected state during testing, all these functions accept a timeout argument. If the object does not reach the expected state within the specified time, the functions raise a LookupError exception or return False.

Handling object removal

Tests may need to verify that objects are no longer present in the application’s UI, indicating removal or non-creation. Qat provides the wait_for_object_missing() function for this purpose. It waits until the specified object is no longer found in the UI hierarchy, signifying its absence. In other words, it does the exact opposite of wait_for_object_exists().

For example, suppose a test needs to verify that a dialog box is closed after clicking on a close button. The test can use the wait_for_object_missing() function to ensure that the dialog box is no longer accessible:

dialog = {
   'objectName': 'confirmationDlg'
}

close_button = {
   'objectName': 'closeButton',
   'type': 'Button'
}

qat.mouse_click(close_button)
qat.wait_for_object_missing(dialog)

By waiting for the dialog object to be missing from the UI, the test confirms that the dialog box has been closed successfully.

Accurate verifications

Qat executes events in a synchronous manner, ensuring that when a function such as mouse_click() returns, the corresponding event handler is guaranteed to have been called. While this synchronous behavior is sufficient in most cases, certain handlers may trigger asynchronous changes. For example, a QML binding may be updated in a later execution of the Qt event loop.

Also, it’s important to note that there are exceptions to Qat’s synchronous behavior, particularly when interacting with native widgets.

Consider the following example:

Button {
   id: counterButton
   property int counter: 0
   onClicked:
   {
      counter = counter + 1
      counterButton.text = counter
   }
}

Text {
   objectName: "counterValue"
   text: counterButton.counter
}

Now, let’s verify that clicking the button increments the value in the text box:

button_definition = {
   'id': 'counterButton'
}

text_definition = {
   'objectName': 'counterButton'
}

button = qat.wait_for_object(button_definition)
textbox = qat.wait_for_object(text_definition)

qat.mouse_click(button)

assert button.text == '1' # This is guaranteed to pass
assert textbox.text == '1' # Undefined result

In this example, the first assertion “button.text == ‘1’” is guaranteed to succeed because the instruction “counterButton.text = counter” is part of the click event handler.

However, the second assertion may pass or fail, depending on execution speed. The text property of the text box is updated through a binding, which means that Qt will generate a secondary event to update this binding in a later execution of its main event loop. Depending on the current computer performance and the number of events currently in the queue, the text box value may or may not be up-to-date when the test calls “textbox.text == ‘1’”.

Introducing wait_for_property_value() and wait_for_property_change()

To address such time-sensitive scenarios, Qat provides the wait_for_property_value() and wait_for_property_change() functions.

  • wait_for_property_value(): This function allows testers to wait for a property to reach an expected value. It continuously checks the specified property of the object until it matches the expected value or until the specified timeout is reached. This ensures that tests can accurately verify object properties before proceeding with further actions.

  • wait_for_property_change(): On the other hand, wait_for_property_change() enables testers to wait for a property to change its value from a known one. This function is particularly useful for scenarios where tests need to verify that an object’s property has been updated but the resulting value is not known.

By leveraging these functions, testers can ensure accurate verification of object properties, even in scenarios where asynchronous changes may occur. This enhances the reliability and effectiveness of automated tests, leading to more robust and stable test suites.

The previous example can then be made reliable as follows:

button_definition = {
   'id': 'counterButton'
}

text_definition = {
   'objectName': 'counterButton'
}

button = qat.wait_for_object(button_definition)
textbox = qat.wait_for_object(text_definition)

qat.mouse_click(button)

assert button.text == '1'

assert qat.wait_for_property_value(textbox, 'text', '1')

These two functions accept other arguments to customize their behavior; please refer to the Python API reference documentation for details.

Asynchronous events

Qat leverage the Qt’s Signal and Slot mechanism to provide connections and bindings, enabling testers to interact with and monitor properties of objects within the tested application.

Connections: Reacting to property changes

Connections allow testers to register a callback function that is triggered whenever a specified property of an object changes. This feature is useful for reacting to events, such as adding an entry to the log report or debugging a test.

For example, consider a scenario where a test needs to log changes in a button’s enabled property state. By establishing a connection to this property, the test can receive notifications whenever the enabled property changes, allowing it to log the state changes accordingly.

def on_button_enabled_changed(enabled):
    if enabled:
        report.log("Button is now enabled")
    else:
        report.log("Button is now disabled")

button_definition = {
    'objectName': 'submitButton',
    'type': 'Button'
}

button = qat.wait_for_object(button_definition)
qat.connect(button, 'enabled', on_button_enabled_changed)

In this example, the on_button_enabled_changed function is registered as a callback to the enabled property of the submit button. Whenever the enabled property changes, the corresponding callback function is invoked, adding an entry to the log report.

It is also possible to connect a callback to a Qt Signal. In this case, the callback will not receive any argument:

def on_action_triggered():
    report.log("Action has been triggered")

action_definition = {
    'objectName': 'saveAction',
    'type': 'QAction'
}

action = qat.wait_for_object(action_definition)
qat.connect(action, 'triggered', on_action_triggered)

Bindings: Synchronizing local variables with remote properties

Bindings in Qat automatically synchronize a local Python variable with a remote property in the tested application. This feature is particularly useful for monitoring values during debugging activities or maintaining global state readily available in tests without the need to repeatedly call wait_for_object_* functions.

For instance, suppose a test requires monitoring the current step of a state machine. By establishing a binding between a local Python variable and the state machine’s activeState property, the test can directly access the current state without explicitly querying the application’s UI hierarchy.

class State():
   def __init__(self):
      self.current_step = ''

state_machine = {
   'objectName': 'stateMachine'
}

local_state = State()
binding = qat.bind(state_machine, 'activeState', local_state, 'current_step')

def go_to_next_step():
   next_button = {
      'type': 'Button',
      'text': 'Next'
   }
   print(f'Current state is {local_state.current_step}')
   qat.mouse_click(next_button)
   qat.wait_for_property_change(state_machine, 'activeState', local_state.current_step)
   print(f'New state is {local_state.current_step}')

The object returned by the qat.bind() function is used to manage the binding lifecycle: calling binding.disconnect() will stop the synchronization while calling binding.connect() will re-enable it.

Also, keep in mind that the binding will be automatically disconnected when the binding object goes out of scope. Therefore, it is often desirable to store it in a global or class variable.

Conclusion

In this tutorial, we explored Qat’s synchronization mechanisms for robust and reliable test automation. By ensuring object accessibility and verifying object properties accurately, testers can enhance the reliability and stability of automated tests.

Timing issues are a common challenge in the testing world, but Qat’s synchronization mechanisms may offer an effective mitigation strategy for these issues.