# Writing Integration Tests with launch_testing https://autowarefoundation.github.io/autoware-documentation/main/contributing/testing-guidelines/integration-testing/ Goal: Creating an integration test that verifies output of a node This tutorial assumes you have some familiarity with using colcon, and have a package set up already Target audience: Familiar with ros and launch, familiar with testing in general, wanting to add integration test for existing package This concerns tests which involve larger amounts of functionality compared to unit tests. This tests entire systems, which could contain multiple ros nodes, ## Step By Step ros2 pkg create --build-type ament_cmake integration_testing_tutorial ## General idea Integration tests run either concurrently with the system under test (active tests) or after the system under test has shut down (post shutdown tests). The tight integration with launch gives the tests access to the console output and exit codes of the system under test. Packages are available for common ros agnostic (in launch_testing) and ros specific (in launch_testing_ros) test assertions. The tests may also interact with the system under test via ros mechanisms, such as checking if published messages are as expected. ## Writing test launch files Docs are here: https://github.com/ros2/launch/blob/rolling/launch_testing/README.md Examples are here: https://github.com/ros2/launch/tree/rolling/launch_testing/test/launch_testing/examples The minimum one looks simething like that: (TODO adapt example) Note that assertWaitFor allows selecting stdout and stderr. Default is stderr which is correct for ros log output, other tools might require stream="stdout" ### Launch Description The launch description is exactly like any other launch file. It may actually include your other launch files. Optionally, the function can return an additional context dict. Values will be passed to parameters corresponding to keys in the test cases. This is useful for referring to specific items of the launch description from a test case. * `generate_test_description()` returns launch description which contain special elements such as `launch_testing.actions.ReadyToTest` ### Tests The same file containing the launch description also contains tests. All member functions of test fixtures are considered tests. All classes defined in the file inheriting from TODO are test fixtures. test launch files * file also contains classes such as `class TestGoodProcess(unittest.TestCase)` with methods like `def test_expected_output(self, proc_output)` that assert stdout * all classes that inherit from `unittest.TestCase` are concurrently run test cases. If they are decorated with `post_shutdown_test` they are run after shutdown instead of concurrently to the dut ## Running tests In order of increasing integration with other tools: Tests can be run with the `launch_test` command: ```console launch_test <path> ``` * `ros2 test` cli command in `ros2test` package (seems to setup isolation with ROS_DOMAIN_ID and start launch_testing) The `ros2test` package provides a cli extention to the `ros2` command which extends the `launch_test` command by assigning a ROS_DOMAIN_ID not currently in use by another running test. ```console ros2 test <path> ``` * cli verb provided by `ros2test` package * command requires full path to python test launch file The third method is `colcon test`, which requires some integration as explained in the next section. ## Discovery, cmake, colcon * `ros_testing` package provides `add_ros_test` cmake function (which creates an ament test calling `ros2 test ...`) to integrate with `colcon test`, use the `add_ros_test` cmake function provided by the `ros_testing` package: ```cmake find_package(ros_testing REQUIRED) add_ros_test(launch/initial_localization_integration_test.launch.py) ``` This adds the specified launch test as a cmake test, which can be found and run by colcon. Now, the tests should run with `colcon test`. Unfortunately the output is not great for human consumption. Maybe the junit files are useful somewhere else. I think even junit is not that great since ctest already does aggregation, and then we have a bunch of individual and partially aggregated files? Does colcon do something intelligent here? I recall reading some issue that implied it does not... `colcon test --event-handlers console_cohesion+ --packages-up-to mock_liorf_package` seems to run the test from source file using `ros2 test`. ## More links * The 2019 roscon presentation "Launch Testing - Launch description and integration testing for ros2" (see https://docs.ros.org/en/rolling/The-ROS2-Project/ROSCon-Content.html for video and slides)