Unit testing

Tests are setup using pythons default testing framework unittest. These tests are run each time new code is pushed to our Github repository using Travis. They can also be edited and run on your local machine as explained below.

In these explanations, we assume you have cloned a version of the qucat repository to your computer.

Running tests

Tests are located in the the tests directory. These can be run from the command line after navigating to the root directory of the qucat repository with the command

python -m unittest discover -v -s ./tests -p test_*.py

Each test file is dedicated to a specific part of the library, and is generally paired with a specific file of the source code, for example the file test/test_core.py tests the functionalities of the src/test_core.py file.

To run only run one of the test files, test_core.py for example, one can also run

python tests/test_core.py

Executing only a single test from a file is also possible by running for example

python tests/test_core.py SeriesRLC.test_frequency

Note

Some GUI tests have failed for a mysterious reason on certain operating systems, even thought the functionality they test works when tested manually.

Writing a basic test

Let us assume that a developer wishes to contribute a new module src/new.py which contains a function f which adds two numbers

One should then create a test/test_new.py file which test if the function works as expected. This is one way of doing it

# Import python test framework
import unittest

import os
import sys

# Add the ./src/ folder to the system path
sys.path.append(os.path.join(os.path.join(os.path.dirname(os.path.dirname(__file__)),'src')))

# Import function to be tested
from new import f

class TestCase(unittest.TestCase):

    def test_f_function(self):
        # Check if f(1,2) is indeed equal to 3
        self.assertEqual(f(1,2),3)

if __name__ == '__main__':
    unittest.main()

For more details on the unittest framework, check this link.

Writing a test for a GUI functionality

Testing the GUI functionalities i.e. what happens if I click here? is also possible in an automatic way by simulating mouse motion, clicks or keystrokes events. These test are located in test/test_gui.py

Creating these tests is also done through the GUI, and we present here a short tutorial on how to do so.

Let’s assume we want to test moving a resistor by clicking, dragging then dropping.

The correct location to add this test is in the test/test_gui.py file, with the following code

class TestMovingComponentsAround(AutomaticTesting):

    def test_moving_resistor(self):
        self.launch_gui_testing(force_build=False,run_slower=False)

The first time we run this file, an empty GUI will open. **Step 1: ** here we set the initial configuration for the test, in this example we create and place a resistor, then close the GUI. This circuit will be saved in the file tests/gui_testing_files/test_moving_resistor/initial_netlist.txt **Step 2: ** a second GUI will open in the initial configuration, and we now perform the task we are testing. Our actions will be recorded in the file tests/gui_testing_files/test_moving_resistor/events.txt later repeated in an automatic way when running this test. In this example, we drag and drop the resistor to another location, then close the GUI. Upon closing the GUI, the final configuration of the circuit is recorded in the file tests/gui_testing_files/test_moving_resistor/final_netlist

The subsequent times we run this test no human intervention is necessary. A GUI will be opened in the initial configuration, a sequence of events will be triggered, and the final configuration of the circuit will be stored in the file tests/gui_testing_files/test_moving_resistor/final_after_events_netlist.txt and compared to the configuration created manually tests/gui_testing_files/test_moving_resistor/final_netlist. If the files are identical, the test passes.

The function AutomaticTesting.launch_gui_testing which implements these functionalities takes two arguments. By setting force_build = True, the test is run as if for the first time. By setting run_slower = True, the test is run very slowly so that one can observe what the automatically generated sequence of events is doing to the GUI. This enables easy debugging of the test.

Writing a test based on a tutorial

All tutorials featured on the website are jupyter notebooks located in the folder docs_src/source/tutorials and are also run as tests. These tests are located in test/test_tutorials.py.

If one adds a new notebook, for example new_tutorial.ipynb in the tutorials folder, one can test it by adding the following code to the test/test_tutorials.py script

class TestNewTutorial(TestTutorials):

    def test_if_it_runs(self):
        '''Runs the notebook new_tutorial.ipynb.
            If an error is encountered the test fails
        '''
        self.run_tutorial('new_tutorial.ipynb')

    def test_if_it_produces_the_right_answer(self):
        '''Runs the notebook new_tutorial.ipynb
            and stores all the variables defined in the notebook
            as a dictionary called variables.
            We then check if the variable called
            "name_of_variable" is equal to 10, 
            the test will fail if it isnt.
        '''
        variables = self.run_tutorial('new_tutorial.ipynb')
        x = variables['name_of_variable']
        expected_value_of_variable = 10
        self.assertEqual(x,expected_value_of_variable)