---
# System prepended metadata

title: '[GT4Py] Test Categorization'
tags: [cycle 18]

---

# [GT4Py] Test Categorization


- Shaped by: Hannes
- Appetite (FTEs, weeks): 2 FTE, 1 week
- Developers: 

## Problem
The GT4Py project’s test suite has grown significantly, resulting in a scattered structure of tests. Currently, tests are not consistently categorized, making it challenging to run specific subsets of tests, especially when focusing on particular features or types of tests. Additionally, the mechanism using pytest markers to exclude tests based on backend capabilities is not fully utilized, as not all tests are properly tagged, and the process of adding these markers is not standardized.

## Solution
The solution involves three main steps:
1. **Categorize Tests**: Go through the existing tests and categorize them into `unit_tests`, `feature_tests`, and `multi_feature_tests` based on the nature of the tests. One important candidate to split up is the `test_execution.py` file.
2. **Apply Pytest Markers**: Review each test and apply the necessary pytest markers based on the features or capabilities they are testing. The markers are defined in `tests/next_tests/exclusion_matrices.py`, and they can be extended as needed.

By the end of this process, we aim to have a well-organized test suite, making it easier for developers to run specific subsets of tests and ensuring tests are being excluded as expected based on backend capabilities.

## Rabbit holes
- **Over-Categorization of Tests**: Striking the right balance in categorizing tests is crucial. Having too many categories can make the process cumbersome.
- **Over-Tagging of Tests**: Similarly, applying too many markers to a test can lead to confusion about which features or capabilities are actually being tested.

## No-gos
- **Complete Test Rewrite**: The goal is to reorganize and retag existing tests, not to rewrite them from scratch.


## Progress
<!-- Don't fill during shaping. This area is for collecting TODOs during building. As first task during building add a preliminary list of coarse-grained tasks for the project and refine them with finer-grained items when it makes sense as you work on them. -->

- [ ] Task 1
    - [ ] Subtask A
    - [ ] Subtask X


