# Release Test Plan 020

This test plan is based on Pantavisor supported devices and features. It presents different levels of testing based on the presence of devices and features in the [Pantavisor documentation](https://docs.pantahub.com/). The objective is to determine what to test and to what extent.

After this document is validated, a [test plan spreadsheet](https://docs.google.com/spreadsheets/d/1D52qVSi9kuDyUbPsaDVvdkD178WOEWqutdmB_e1bYn4/edit?usp=sharing) will be generated to organize the testing process.

## Devices

This classification divides the [supported devices](https://docs.pantahub.com/initial-devices/) based on documentation level.

A device belonging to a tier does not mean that we want that in there, as the classification is based only on the current status of documentation. Dropping a board might be a possibility if we do not want to test it as thoroughly, the same way promoting it means we need to keep the documentation standard of its new tier before we test it.

### Tier 1

Full-featured devices (HDMI, TTY, OEM partition, etc). Used in the [get started guide](https://docs.pantahub.com/get-started/) for new Pantavisor users:

* **RPi4**
* **RPi3 b+**

### Tier 2

Boards with lighter documentation than tier 1 but still with content on [how-to bring up](https://docs.pantahub.com/choose-image/) Pantavisor:

* **Nvidia Tegra 4.9**
* **Rock64**
* **BPi-R64**
* **RPi0w**
* **Toradex Colibri iMX6dl**
* **Toradex Colibri iMX6ull**
* **x64-uefi (default)**
* **x64-uefi (installer)**
* **x64-ubuntu**
* **QEMU x64-uefi**
* **QEMU malta**
* **App Engine x64**
* **App Engine aarch64**

### Tier 3

Rest of [supported devices](https://docs.pantahub.com/initial-devices/) that does not have any documentation:

* **BPi-R2**
* **BeagleV**
* **Odroid C2**
* **App Engine mips**
* **App Engine arm**

## Features

This section enumerates the most important Pantavisor features, with a small description on how to test them, and divided by their documentation level.

### Basic Management

Documentation in [how-to manage your device](https://docs.pantahub.com/choose-way/) should suffice to perform basic management.

* **Pantacor Hub:** download image from hub with auto-claim (manually claim if not possible), get device metadata and logs, set user metadata, redeploy. Clone and post with pvr.
* **Pantabox:** download image from hub without auto-claim and with remote always on, ssh pvr-sdk, add marketplace app, add docker app, add \_config, deploy, manually claim.
* **pvtx**: ssh, open local, access pvtx, get device logs, deploy new transaction.
* **pvr:** locally clone, add app, add \_config, deploy, get logs.

### Advanced Features

Documentation is more general and can be found in [advanced feature](https://docs.pantahub.com/use-secureboot/), [tech overview](https://docs.pantahub.com/pantavisor-architecture/) and [reference pages](https://docs.pantahub.com/build-options/).

* **Configuration:** check log.level is 5 by default, then change it from cmdline to 4, then from trail to 3, then from metadata to 2. Check in every case the config has changed both in logs and from pvcontrol.
* **SSH:** connect to pvr-sdk using SSH. Disable SSH temporarily with pvcontrol. Check SSH no longer works. Reboot board. Check SSH is back again.
* **Policies:** set policy "prod" in cmdline and check debug.ssh is 0, debug.shell is 0 and secureboot.mode is strict.
* **Signatures:** post unsigned elements, check it fails, post again with all signed elements and check it works. Then, "prod" policy can be removed.
* **Rollback:** deploy empty update, remove awconnect and deploy. Check it rolls back to initial revision. Check progress message contains meaningful information with pvcontrol.
* **Persistence storage:** check persistence of pvr-sdk is permanent in /etc/dropbear/ but not in /etc/.
* **Garbage Collector:** make changes in any container and deploy, deploy empty update, ssh pvr-sdk and note down object, logs and rev list. Then run garbage collector, check object, logs rev list again and compare.
* **Checksum:** post new revision with debug console open, mess with object from new revision, exit console.
* **Logserver**: from debug console, change config to different combinations of singlefile, filetree, stdout, stdout.pantavisor, stdout.containers and nullsink outputs.
* **PH Logger**: disable/enable log push during runtime with user metadata. Check changes in Hub.
* **Network Configuration:** change in cmdline pvnet config to automatically connect to a new hotspot.
* **Cryptdisks**: check crypt disk is mounted, check metadata is using that disk and that new metadata can be saved.
* **Groups:** add new platform to group app, check platforms are started in the right order.
* **Restart policy:** post update to platform group container, post update to app group container, check transitions are done accordingly for each case.
* **Status goal:** set pvr-sdk to ready status goal. Check if deployment fails if ready is not sent. Check it works if sent manually with pvcontrol.
* **Roles:** check if pvcontrol ls can be used in pvr-sdk. Post a new revision with role nobody in pvr-sdk and check if it returns error in a non-mgmt one.