Prioritization is essential for product managers because we seemingly work in a world where time is limited and demands are ostensibly unlimited. Our prioritization skills help us spend our time and resources wisely, freeing ourselves and team from less important tasks that evidently can be pushed out - or quietly dropped (isn't it funny how that happens).
In my last post, High, Medium & Low Don't Work for Agile, I suggested that traditional methods of product management prioritization, for example, P1, P2 and P3s, do both product managers and product teams, a disfavor. Such prioritization approaches do not link backlogs objectively to market demand nor profitability. When resources (time and money) are as scarce as they are, the risks are of not doing so are simply too great.
This week, my objective is to share our scoring and prioritization methodology in hopes of exposing its vulnerabilities. (disclosure: we develop, market and sell a product management platform designed to capture customer feedback, groom and prioritize your backlog). I'm looking to you, fellow product management practitioners, for feedback on how to improve it. Lastly, I hope our discussion (and if enough people are listening) will help others along the way (if so, please let us know who you are!).
To begin, I'd like to distinguish scoring from prioritizing. While they may seem interchangeable, they are, at least in our minds, very different. In this context, scoring is the process of objectively evaluating a specific item (e.g., backlog item or feature) against set criteria. The result of the exercise provides a score that can be used to sort and compare relative value. Items that score well are deemed more valuable.
On the other hand, prioritization is the process of evaluating the scored backlog to identify sequencing and/or sub-sequencing. A prioritized backlog depicts the ultimate order, be it a sequence or a series of sets (buckets), that will deliver the greatest value (superior ROI) in the least time.
Scoring, Prioritization and Sprint Readiness
Prioritization for us is a 3-step process. First, we score our backlog against business criteria (value). Second, we consider technical elements. Lastly, we select a subset of these items which is subsequently our release and sprint backlog (i.e., we iterate and release every month).
1. It's our contention that scoring features against business criteria is paramount and should be the focus of product management. To facilitate this activity, we use and evaluate each item against the following criteria:
- Market Evidence (traceable support from customers/market)
- Stakeholder or Persona Alignment (buyers, users, partners, etc.)
- Strategy Alignment (markets, technology, etc.)
- Business Impact (profitability - increase revenue, decrease cost, both)
It should be noted that our approach has evolved considerably since our early days. Similarly, I suspect you and your organization, like ours, continues to evolve and refine its methodologies. How do you measure business value today? How is your approach different?
2. While it is possible to prioritize features on business merits alone, it's not enough to drive release or sprint planning. Before ideas can be prioritized, they must first be estimated (roughly) by development. Our next step is to add technical attributes.
- Estimate (measurement of size/complexity)
- Risk (the probability or likelihood that the item won't be completed as designed or desired)
- Skill set (PM, UI, DBA, TW, BA, QA, Architect, etc.)
Note, the above technical items don't influence the score, but rather provide additional context to support prioritization, sprint and resource planning. This is a distinct step because it requires collaboration outside product management and with development. Armed with business context (score) and an estimate (hours or story points), we now have the main elements to prioritize our backlog to deliver maximum customer value.
3. The next step is release/sprint planning (remember, we release every month). Given that we now have relative estimates from development, we have all the necessary information to make informed decisions to drive business results. Interestingly, some items may be very valuable to the organization but may take weeks to develop. On the other hand, other requests may be half as valuable but may only take hours to deliver. From experience, this happens more often than you think. Evidently, estimates (costs) influence priority.
Note, we have some customers that use to apply estimates first, that is, before scoring for business value. When we dug into the reasoning, we found it was usually because they didn't have a formal approach to define and score business value. Our position is that you should score for business value first for two reasons:
- Business value should be your primary motivation
- Although your time is very precious, is not as expense as a team of developers cramped in a meeting room estimating or planning poker.
Rather than have developers waste time evaluating items that may never be necessary or "bubble-up to the top", we provide development with a backlog that has already been scored against business criteria. In doing this, we ensure they are working on the most important items first.
To conclude, prioritization is a key product management skill to make the best use of your time and that of your team. Backlogs are prioritized by product management but with the help and collaboration of development. With every release, our goal is to identify and prioritize the backlog items in a manner that produces the most customer and business value.
In my next post, I will dissect each of the above business criteria. I am very interested to learn what you do to prioritize your backlog. For example, do you use story points (e.g., Fibonacci), T-shirt sizes, or actual time (hours, days, etc.) to estimate your backlog? I'm also curious to know if skill set impacts your prioritization. Finally, and assuming you prioritize your backlog for business value, do you share your evidence with development? If so, how do they react?
Based on field discussions and my recent experience at ProductCamp Toronto, it appears most product managers have no formal methodology to score or rank customer feedback or feature requests. In fact, I suspect very few product managers actually score features against any relevant business criteria --- rather they simply apply a basic prioritization scale, for example, P1, P2, P3 or High, Medium and Low. Is this you? For these PMs, spreadsheets are usually the weapon of choice and are commonly initiated with each roadmap, release or sprint exercise. With every new prioritization, a new spreadsheet gets created, copied, revised and versioned --- over and over again. Sound familiar?
What evidently happens with these methods is that P1s (or Highs) become disproportionally represented. For example, on 50 features, poor prioritization will result is 30-40 P1s. Of course, in realty, this means nothing. Essentially, it amounts to an abdication of responsibility --- responsibility for making a product management decision.
As experienced product managers, we all know, "if we don't do our jobs ---someone else will". In this case, it's likely to default back to development. Note, developers make decisions based on technical fun factors --- and they don't consider UI to be fun, ever. Hmmm... I think I'd rather get my teeth pulled...
Few Product Managers Apply Business Attributes to their Backlog
Not sure why, but I imagine lack of training and tools have a lot to do with it. This is particularly disheartening because both are available online --- either for free or very affordable.
The problem with this approach, I believe, lies in its simplicity. Don't get me wrong, simple is good --- but only if it's effective and drives intended results. Truth be told, I'd like to think product management today is a little more sophisticated, intelligent and strategic. I'm curious, if you are a product manager that uses one of the aforementioned prioritization methodologies --- I'd like to hear from you. Specifically, what drives your decision making? What makes one item a P1 and not a P2? How does a simple ranking demonstrate or communicate your strategic objectives, be it short or long term?
The biggest issue with a simple priority system is that it neither supports nor encourages the product team to think strategically. Must Haves, Should Haves and Nice to Haves are fuelled by opinion and filled with conjecture. How does High/Medium/Low demonstrate value or congruency with product objectives? As product managers, aren't we always defending our strategic value? Isn't this an ideal opportunity to demonstrate our business aptitude and strategic thinking?
Problem is Amplified with Agile
Perhaps my position is skewed as an Agile (SCRUM) practitioner and advocate. In fact, I see this issue amplified in an Agile environment. How do P1s, P2s etc. support sprint planning? What do you tell development --- do all the P1s? How do they know where to start or focus?
Aren't we supposed to have a theme or goal at the root of each sprint? If so, how are these objectives reflected and supported in your backlog? The onus is on us. If Product Managers or Product Owners don't step up and effectively control and prioritize their backlog, the team will flounder, lose valuable velocity and become increasingly frustrated.
Certainly, I have some ideas to address this issue and will address them in a follow-up post. In the interim, I'd like to hear from you and learn about your prioritization practices. Are my assumptions way-off? Am I complicating a simple task? Trying to kill and ant with a bazooka?