# Testing improvements --- **WORK IN PROGRESS** -- this hackmd was for brainstorming and collaboration, and it will be distilled into a project plan/roadmap on github shortly --- TODO, things to include in project plan: - **what's the overall story here?** - clean house, create/encourage new patterns; fix up conformance tests and take them to the next level; work with build v-team to facilitate granular builds (longer-term: how can we take our tests to the next level?) - **summarize a few big items** and write a paragraph about why (keep the basic "what" as bullet points) near-term priorities - screener (Ling) [important, urgent] - custom proxy supports multiple projects - split v8/v9 - add a way to run only affected screener - ✅ ~~update testing-library [low-cost]~~ - very easy, already done - motivation: should be up to date before investing in enzyme/testing-library migration - enzyme to testing-library [important, urgent] - motivation - unblock react 17 or 18 in our repo - could we do this more quickly by using the community react 17 adapter, then porting any enzyme tests that break to testing library? - test environment a bit more closely resembles browser - steps - if you need to change a v8 test, switch to testing-library - remove from converged packages (already done?) - remove from conformance - remove from v8 (split work) - remove from v0? (up to n* team) - document testing best practices - link to testing-library docs - designate a component for demonstrating testing best practices? - don't test every single prop (perf concerns) - enable cypress for v8 - motivation: more realistic tests for focus etc; help with migration away from enzyme - most of the tests that break when we try to update to jest 27 or react 17 are focus-related; for these it makes more sense to move from enzyme to cypress - conformance tests [important, non-urgent] - remove enzyme (breaking change) - API improvements (breaking change) - maybe: presets and plugins/rules similar to babel - make the updates (1-2 weeks?) - split v8/9/0-specific parts - perf improvements (grouping tests that need type info) - write the new runner in separate package at first and update v9 to use it? - update jest [after we get off enzyme] - motivation: might want to be up to date before investing in enzyme/testing-library migration - additional later items - [switch to `@jest/globals`](https://jestjs.io/blog/2020/05/05/jest-26#a-new-way-to-consume-jest---jestglobals) and remove jest global types (remove type pollution in v8) - enable ES modules? - small pain points - fix ID churn issues in v8 snapshots [low-cost] - get someone else to help with this - eliminate example snapshots? [low-cost] - ask team if anyone thinks they provide value, and if not, remove them (can anyone remember being alerted to an actual bug by an example snapshot, which you might not have found some other way?) - cypress/"e2e" tests - can we improve the inner loop? - can we improve reliability? - check with Tristan and Ling about interest Longer-term work - Add more conformance tests - Find a new owner for perf and/or bundle size - Screener replacement? - motivation: - screener is okay but not great (doesn't seem to be getting a lot of investment, can be hard to get support, not integrated with our other tools) - inconsistent hover states - chromatic? (good integration with storybook+github; missing proper interaction support) - playwright? (in use by 1js, other teams) - vite shot? - percy? https://docs.percy.io/docs/storybook, https://docs.percy.io/docs/cypress - add memory tests - bundle size improvements - link to report to share out - other? - perf test - split v8/v9 so you don't have to build everything to run the tests (overlaps with things Martin is planning) - plan partner integration tests Taxonomy/layers of testing (feedback needed) - primary tests that dev runs locally (validate you didn't break your component) - unit tests of functions etc - component functional/unit tests - component conformance tests - snapshots - vnext: DOM snapshots - v8: DOM/style snapshots - runs in PR (ideally can also run locally) (validate your change didn't break something else) - component cypress "e2e" tests (functional tests using real browser) - snapshots of stories? (currently v8 only, not very valuable) - screener (visual tests including of basic interactions) - create-react-app template - typings back-compat - gates - bundle size - perf - not yet implemented - memory - partner integration (nightly?) Possible goals - improving feedback loops and having the right set of tests/coverage at different layers of testing - conformance: ability to test existing AND future components - tests add confidence and velocity for fixes - code sharing: validating our code with partners and reducing their inner loop to test with dev builds of our code Things to look at - tools we KNOW we want to use but are out of date and causing problems - cases where we might want to switch to better tools - cases where we need to improve tools we own - eventual goal, find ways where we can better leverage existing tools