# Test all the things! http://goo.gl/Xcocrp
Last Working Group: https://wiki.freebsd.org/DevSummit/201706/Testing
## Current Status
* CI system:
* DTrace test pipeline: https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/
## What to test
* user space utilities
* kernel features
* in `src/tests/sys`, many skipped due to test environment (ex: no zfs)
* non-x86 architectures
* use qemu user-mode to test cross platform
* slow, some tests are racy
* there seems a arm-cloud provider willing to sponsor
* Tracing tools
* Possibly finding a way to test regressions while deploying security mitigations?
* How others do?
* has test for sys/
* has performance tests
* Microsoft, FreeBSD on Azure
## How to test
* Automated analysis of fuzzing results?
* Ideas to generate an analysis data structure (Crash Graph ideas)
* QEMU fuzzing?
* real hardware testing (arm/mips/... boards)
* unit testing of kernel functions/modules/units (requires framework)?
* user-space versions of kernel code (examples: SCTP library, TCP library, etc.)?
* Create "culture of testing"?
* Should reviewers ask for tests for big features just like they ask for man pages?
## Quality Assurance of FreeBSD
* criteria of "good to ship"
* no regression
* experimental feature should be marked explicitly
## TODO list
* go through cases in test suite
* need list of "skipped", "know to fail" cases
* can jenkins list them?
* integrate more test suites
* needs `WITH_DTRACE_TESTS=yes` in src.conf
* can be enabled after fixing all bugs (in DTrace or test cases)
* Using Packet Drill tests from Michael Tuexen (gnn@)
* `Test suite` is available here https://github.com/freebsd-net/tcp-testsuite
* ZFS Test suite (brd@)
* Have we converted all things under `tools/regression`?
* setup performance testing cluster
* location? NYI
* hardware requirements? (brd@ / gnn@)
* generalize freebsd-ci codes for downstream adoption
* a package to setup everything?
* Some form of automated testing for security mitigation deployment (W^X, ASLR, PAN, ...) (brooks@, Domagoj)
* Talk to Backtrace (gnn@)