# Driving Test Coverage ## A Play in Several Parts ### Mostly from the perspective of The Real Budget We Have To Do This Work --- ## Types of Test: - Unit testing --- ## Types of Test: - Integration testing --- ## Types of Test: - API Testing --- ## Types of Test: - Security Testing --- ## Types of Test: - Load Testing --- ## Types of Test: - Feature Testing --- ## Driving Test Coverage There are several roles involved in any medium scale effort to drive test coverage. The smaller the organization, the more of these roles will be given to a single person. Sometimes, all roles are a single person. --- ## The Roles - Test Coverage Prioritizer - Test Coverate Estimator - Test Framework Decider(s) - People Who Write Tests - The Testing Sage - The Anti-Tester --- ## Test Coverage Prioritizer Weigh Stakeholders _whose needs are most critical?_ --- ## Test Coverage Prioritizer Weighing Stakeholders Needs of the Market over Individual Economic Purchaser (all customers over one customer) --- ## Test Coverage Prioritizer Weighing Stakeholders Needs of the Individual Economic Purchaser over the Customer (the person who signs the checks is different than the chooser of the product) --- ## Test Coverage Prioritizer Weighing Stakeholders Needs of the Customer over the User (the chooser of the product is not the user of the product) --- ## Test Coverage Prioritizer Weighing Stakeholders Needs of the User over the Designer (the user isn't likely to be similar to the designer of the system) --- ## Test Coverage Prioritizer Weighing Stakeholders Needs of the Designer over the Developer (do it even if it's hard to make it a better design for the user, listen to the designer) --- ## Test Coverage Prioritizer Weighing Stakeholders Needs of the System over the Developer (maybe you don't like the syntax or tool because it's clunky, don't overcomplicate the effort because of this) --- ## Test Coverage Prioritizer Prioritize coverage- what parts of the system can take other parts down? what parts must stay available? --- ## Test Coverage Prioritizer Prioritize depth- how rigorous must tests be? Are they simply "down detectors" or are they formal proofs? What types of test will satisfy this depth need? --- ## Test Framework Decider Given the stakeholders and prioritized test coverage, decide on testing frameworks for the top prioritized systems based on their needs and depth. Assigns frameworks to systems. --- ## Test Coverage Estimator --- ## Test Coverage Estimator **Endpoints** are one of the units of measurement of tests. They could be functions, classes, or web server routes. They could be entire features or paths through the app. Lightly review the codebases to understand it's endpoints relative to the type of test you're going to implement. --- ## Test Coverage Estimator Find out this: - How many endpoints to test are there? --- ## Test Coverage Estimator Find out this: - How large is the range of expected input? (complex, high cardinality, or small) --- ## Test Coverage Estimator Find out this: - Does it require special knowledge or drive a mission-critical process upon which many things depend? (eg, billing or major reports, the emailer, etc) --- ## Test Coverage Estimator This person must be responsible for maintaining a **scoreboard**. Let's dive into [The Four Diciplines of Execution](https://slides.com/lizh/4dx) to discuss the scoreboard. --- ## People Who Write Tests These are people who look at the scoreboard of places to write tests for, then do that. They're heroes whose names will often be forgotten. The Scoreboard helps prevent this. --- ## The Testing Sage Someone with the internal clout to drive testing as a culture. This person is already respected and capable of driving innovation through their reputation. Be choosy about who this person is, if they have enemies, those people will oppose your efforts. They'll need to be a vocal proponent and recognize heroes from the **scoreboard**. --- ## The Anti-tester Someone must represent chaos and anarchy. This person's job is to defy the guidelines for testing in order to maintain space for greenfield development to happen quickly. They need to have equivalent clout to The Testing Sage. --- ## The Anti-tester This person is _not_ [**the devil's advocate**](https://www.slideshare.net/samanthaangel2/10-faces-of-innovation) but is instead simply part of the ecosystem that keeps your organization limber and agile. --- ## The Scale Resenter Another agent of chaos- due to the planning effort at a high-level leaving out some details and considerations (it always does), someone chooses a completely different testing framework and method of tracking their work. Reporting from this area becomes fraught and the project will stall. --- ## The Scale Resenter This player is a statistical reality of the complexity of large projects, due to some of their needs not being met. The Scoreboard Leader should carve out a special section for them, to represent the increased or more complex workload.