Impact, Confidence & Effort (I.C.E.) Prioritization

ICE force-ranks a growth backlog so the loudest voice in the room doesn't set the roadmap. Score each idea on impact (relative, 1-5), effort (person-weeks), and confidence (50/75/90%, based on the source), then rank by (impact ÷ effort) × confidence. Double down on the top quarter, take quick wins from the next, and skip the bottom half this quarter. Confidence is the factor almost everyone ignores, and it's the one that matters most.

Within a growth lever that we are focusing on, how do we determine what initiatives to focus on and which to not pay attention to?

One of the easiest ways is to apply a simple scoring method in order to force-rank your roadmap.

ICE stands for impact, confidence, and effort.

It is intentionally designed to be a ratio of Impact over Effort, surrounded by confidence.

This is because you want to be optimizing for maximum impact to your target metric, per amount of effort spent. Confidence underscores Impact and Confidence because with low confidence, the impact could be much lower than expected or the effort could be much higher than expected.

How to put I.C.E. into practice

In many cases, people only think of the impact, and not the required effort, and in even more cases, people don’t consider confidence at all.

Impact is the amount we believe we can improve the target metric.

• We grade impact on a simple 1-5 scale. What is important is that we are grading relative to other items on the list, since we don’t know what the actual impact will be, but that isn’t important for a relative-scoring exercise. 

Effort is the number of weeks of a person’s time.

• We grade up by half-weeks with no maximum. Effort estimation isn’t intended to be exact, but again, for relative-ranking, it is only important to be somewhat accurate.

Confidence is derived from the source of the information.

• Confidence can have a score of 50%, 75% or 90% (we're never 100% confident). A 90% means that we have hard data, strong comparables, or direct expertise which backs up our belief in the impact score. A 75% means that we have indirectly related data, or evidence to support our scoring. 50% means that we could be right or wrong, we don't have significant data to support or disprove our scoring.

Confidence truly is key

Confidence is the most overlooked of the factors, but is arguably the most important. An easy way to understand confidence is to think about information sources for things like scientific discoveries, news publications and others – your confidence in that information relies on the credibility of the source and how well it is corroborated. 

There are a number of idea sources of information, with differing levels of confidence.

Most people are familiar with brainstorming, which is actually the lowest confidence idea source. The equation that we consider is that confidence is made up of a combination of data and relevancy.

We at MasterClass ensure that we have a diverse backlog from a number of different information sources with high confidence.

• We do teardowns of 60 comparable companies cross Entertainment, Education, and Consumer Tech verticals. 
• We survey subject matter experts and give them free access to the product.
• We do on-site polls and ask users directly what feature in the backlog would be most beneficial towards their purchase decision.

These are just a few of our most practiced idea sources, but in reality we do them all.

Using I.C.E. to drive focus across initiatives

And so, the equation is the ratio of impact over effort, factored by confidence.

We can then use the value score to sort the backlog from high --> low score.

1. Then we take the top ¼ of the backlog and those are things we double down on.
2. The next ¼ we continue to evaluate and invest in quick wins.
3. The bottom half are things that we determine that we aren’t going to focus on for the quarter.

The techniques we've discussed are fairly simple, but have a huge impact on the strength of a backlog - 1.) Applying an impact/effort ratio, 2.) Leveraging idea sources that have high confidence, and 3.) Using a value score to determine what you should invest in, continue evaluating, and not do.

In the next post in this series I'll discuss core growth levers to evaluate and the basic structure of a Growth Model!

Common questions

What does ICE stand for?

Impact, Confidence, and Effort. The score is a ratio: (impact ÷ effort) × confidence. Impact and effort are scored relative to the other items on your list, and confidence scales the whole thing based on how much you trust the estimate.

Why is confidence the most important factor?

Because low confidence means your impact estimate could be far too high, or your effort far too low, and either one breaks the ranking. Confidence comes from the source of the idea: hard data, strong comparables, or direct expertise is about 90%; indirectly related evidence is about 75%; a hunch with little to back it is 50%. Most people skip this factor entirely, which is exactly why their roadmaps drift.

How do I turn ICE scores into a roadmap?

Sort the backlog high to low by value score, then split it. Double down on the top quarter, take the quick wins from the next quarter, and don't work the bottom half this quarter. Re-run the same scoring on any new idea or request so it competes on the same terms as everything already in the backlog.