# Growth Hacking Success via High Tempo Testing

[toc]

---
Presentation by Sean Ellis
Notes by James Joseph
Link: https://www.youtube.com/watch?v=i0nxESR9sFs

---
# What is "High-Tempo Testing?"

## Growth is holistic
<b>Figure 1. AARRR = "Get > Keep > Grow". Strategies for User Growth span across the organization </b>

Acquisition Activation Revenue Retention Referral are all levers in the sales funnel.

## Test to Discover and Optimize
Test to discover a process
Test to optimize a process

## Optimize for the Number of Tests
Required to increase learning/innovation.

<b>Figure 2. When Twitter increased tests per week, their users per week also increased. </b>

# Building a "High-Tempo Testing" Growth Team

## PM of Growth
Product Manager/Project Manager/Portfolio Manager/Program MAnager of Growth
- Organizes the testing sprints
- target 3+ tests a weak
- the critical first hire

## Core Team

- Designers, growth engineers, analysts, etc may be utilized more often than not to hit High-Tempo Testing goals. Considering hiring full-time

# Executing Repeatable, High Tempo Testing

### First, Unbridled Ideation

#### Generate ideas internally

- Build a large backlog of ideas (Sean Ellis' team in 2015 had 500+)
- Invite participation from the whole company, require participation from the Growth Team
- Keep a leader board:

<b>Figure 3. Ideation Leaderboard</b>

#### Generate ideas externally
<b>Figure 4. External insights aide in understanding results and ideating new tests</b>

### Second, Prioritize the Backlog of Ideas

1. Formalize ideas:

- Include supporting <b>research</b>.
- Include your <b>hypothesis</b>
- Which Key Performance Indicator will be effected, by how much, and why?
- Define the <b>lever</b> you are targeting
- Acquisition? Activation? REvenue? Retention? Referral?
- Detail execution <b>plan</b>
- where in the funnel is the idea focused?
- detail the testing/production environment
<b>Figure 5. Prioritize ideas by formalizing them in order to score them</b>

2. Score ideas using ICE method
- Impact (1-10)
- Confidence (best practices elsewhere, customers?)
- Ease (how much effort to implement? You need more experiments!)
<b>Figure 6: Score each idea</b>

Scores help select an idea on the account of working conditions:
Difficult time? New hires? Choose easy project!
Not enough time? Healthy team? Choose impactful experiments with high confidence

### Third, Weekly Growth Meeting

- Meeting establishes a growth rhythm
- Meeting helps to improve growth process, enabling team to hit High Tempo Testing targets, maximizing learning
- Include core growth team and key executives, i.e CEO, VP Product, CTO)

1. Nominate ideas

- Use ICE score to select ideas to nominate

- Each growth team member has to nominate two ideas and offer a 30-second pitch. At least one idea cannot be there own.

2. Launch Tests

- Each idea has a project manager associated with it that answers to PM of Growth or Growth Master

3. Capture Learning

- EVERY tests leads to learning
- Learning is organized for easy access
- Publish a Weekly Growth Letter. Consider sharing it with customers to engender external feedback.

<b>Figure 7. Capture Ideation > Scoring > Nomination and Learning in a single, easy-to-explore knowledge base </b>

# Notes

Put a quantitative goal on testing, it will drive its process management
An alternative to the ICE scoring method is the PIE scoring method.