How Can Agile Frameworks Foster Continuous Product Discovery

Why Are 70% of Features Rarely Used?

It's a common yet troubling statistic: 70% of features are rarely or never used. This reality is perplexing, especially considering the energy and passion that go into designing, developing, and launching these features.

‍

Our ultimate goal is to create a product that acts as a powerful vehicle of value exchange, where the organization addresses customer problems, jobs, pains, and gains through its product, and in return, the customer provides value to the organization in the form of money, data, or knowledge.

‍

So, what is going wrong? What can explain this low adoption rate of features? Let's identify some root causes:

1. Starting Without Framing the Business Needs
   • Without a clear understanding of the business needs, it's difficult to ensure that the solution aligns with organizational goals.
2. Unclear Contribution to the Organization
   • If we don't know how the solution contributes to the organization, it’s hard to measure its success or relevance.
3. Poorly Defined Target Customer
   • Not knowing who will use the solution leads to a disconnect between the product and its users.
4. Lack of Understanding of User Needs
   • We often fail to understand what users are trying to achieve, their success criteria, their pains, and what they value.
5. Assumptions Over Research
   • Assuming we know what customers and users want based on data alone, without spending time talking to and learning from them, leads to misguided solutions.
6. Overambitious Problem-Solving
   • Trying to solve too many problems for too many people results in large, unwieldy solutions that satisfy no one.
7. Ignoring Multiple Solutions
   • Focusing on a single idea because of sunk costs or initial excitement can blind us to better alternatives.
8. Roadmap-Driven Development
   • When the priority is merely to check off items on the roadmap, we lose sight of user adoption and satisfaction.

‍

Reflection and Improvement

Do you recognize yourself in any of these points? Admitting these issues is the first step towards addressing them. By framing business needs clearly, understanding our target customers deeply, and staying flexible and user-focused, we can develop features that truly resonate with users and provide significant value to the organization. Let's commit to a more thoughtful and user-centered approach to product development.

‍

What Do Successful Teams Do?

Successful teams take strategic steps to ensure their efforts are impactful and aligned with user needs and business goals. Here are some key practices:

‍

1. Address Risks Early
   • Before building anything, identify and mitigate the most critical risks. Risks can include:
     • Value: Will the user buy this?
     • Feasibility: Are we able to build this?
     • Usability: Will the user figure out how to use it?
     • Viability: Will it work for our business?
2. Collaborate Intensely
   • Product, design, and engineering teams work side by side, fostering close collaboration and integrated efforts.
3. Focus on Outcomes and Impact, Not Output
   • The goal is to build as little as possible while maximizing outcomes and impact (i.e., business results). Measure whether the intended outcomes are realized; if not, adapt and try again.
4. Embrace Product Discovery
   • Product discovery is a discipline that helps quickly identify the right thing to build, supported by sufficient evidence. It involves learning and validation to ensure the right solution is built for the right problem.

‍

Principles of Product Discovery

1. Customers Don’t Know What Is Possible
   • They can tell us about their problems but not necessarily what to build.
2. Value Is Key
   • Focus on creating value for the user and the business.
3. Functionality, Design, and Technology Are Intertwined
   • These elements must work together harmoniously.
4. Expect Iterations
   • Many ideas will not work initially or will require several iterations to succeed.
5. Avoid Assumptions
   • Do not assume you know the users and customers; always seek to validate these assumptions.
6. Validate Ideas Fast and Cheap
   • The most expensive way to test an idea is to build production-quality software. Instead, use rapid prototyping and testing to learn quickly and cost-effectively.
7. Commit to Learning
   • The essence of product discovery is continuous learning. Every experiment and iteration provides valuable insights that guide the development process.

‍

By adopting these practices, teams can navigate the complexities of product development more effectively, ensuring that they build solutions that truly meet user needs and drive business success.

‍

Why Is It So Difficult to Put Common Sense into Practice?

Despite the clear benefits of effective product discovery, implementing these practices often proves challenging. Here are some key reasons why:

‍

1. Time Constraints
   • People are constantly busy, jumping from one topic to another, making it difficult to fit in extra customer interviews or design workshops. Additionally, these activities take time, which can be scarce.
2. Organizational Barriers
   • Discovery-related activities may be conducted in different departments, but information often doesn't flow freely, leading to misalignment. Organizations tend to be output-driven due to the planning and individual goal-setting processes, limiting opportunities to engage with customers and users directly.
3. Knowledge Gaps
   • Effective discovery relies on various quantitative and qualitative research techniques and frameworks. However, people may lack knowledge of these methods and struggle to navigate the vast amount of information available.
4. Tooling Limitations
   • While certain tools can facilitate discovery activities, tooling constraints are often seen as a barrier. However, creative solutions can often be found to work around these limitations.

‍

Overcoming These Pain Points

To address these challenges, we can leverage existing agile frameworks and make some aspects more explicit. Here are some practical steps:

‍

1. Focus on Product and Sprint Goals
   • Establish clear product goals, sprint goals, and increments to ensure a strong focus on delivering value.
2. Make Time for Discovery
   • Incorporate discovery activities into sprint planning to ensure they are prioritized and time is allocated.
3. Structure and Visibility
   • Use tools like the product backlog, refinement sessions, and sprint backlog to structure the work and make it visible to the entire team.
4. Continuous Learning and Improvement
   • Utilize Scrum events to facilitate continuous learning and improvement, ensuring that feedback loops are in place.
5. Risk Management Through Experimentation
   • Address risks by conducting experiments, testing hypotheses, and iterating based on findings.

‍

Four Tweaks to Foster Continuous Discovery

Instead of introducing a whole new framework, we can make small, manageable changes to foster continuous discovery:

‍

Tweak 1: Reviving Your Product Backlog for Discovery

Why?

Most of us are familiar with the concept of a product backlog. According to the Scrum Guide, the product backlog is an emergent, ordered list of what is needed to improve the product. It serves as the single source of work for the Scrum team. Through refinement activities, we break down product backlog items into smaller tasks that can be completed within a sprint, ensuring they are ready for selection during sprint planning. This ongoing activity involves adding details such as descriptions, order, and size, which may vary with the domain of work.

‍

However, we often view the product backlog merely as a list of tasks to be delivered, assuming that everything should be built. This mindset does not support the product owner in effectively optimizing value, often leading to local rather than global optimization.

‍

The Solution: Flagging Discovery State

We suggest flagging your product backlog items to reflect their discovery state. While some advocate for a separate backlog for discovery, we prefer maintaining a single backlog that indicates the discovery status of each item. Some items may be well-understood and considered sufficiently discovered from both problem and solution perspectives, while others may still require significant discovery efforts to identify assumptions, assess, and test them.

‍

Note: Not all backlog items will require intense discovery work. The need for discovery will depend on the risk profile of the organization, keeping in mind that the most expensive way to test an idea is by building production-quality software.

‍

Benefits of an Effective Product Backlog

By integrating discovery state into your product backlog, you can achieve several key benefits:

• Increased Transparency and Shared Understanding
  • Clearly indicate which items need further discovery and which are ready for development, enhancing the team's and stakeholders' understanding of the work required to improve the product.
• Clear Stakeholder Expectations
  • Set clear expectations with stakeholders about what is included in the product backlog, what is not, and what is known versus unknown. This clarity helps manage stakeholder expectations and reduces misunderstandings.
• Focused Learning and Development Efforts
  • Concentrate the Scrum team's efforts on learning and delivering the next most important items of value, ensuring that development efforts are aligned with the highest priorities.
• Aligned Team on a Shared Product Goal
  • Foster alignment within the team around a shared product goal, which serves as a step toward realizing the product vision.
• Minimized Time Waste
  • Reduce time spent on items of low importance by maintaining a clear focus on the most valuable tasks, ensuring that the team's efforts are directed toward high-impact activities.

How?

1. Identify Opportunities
   • Step back from the solution/delivery-driven items typically added to the backlog and consider larger opportunities.
   • Use tools like the opportunity solution tree, story map, or impact map to identify opportunities. Ensure you include key elements:
     • Vision: Defines your intent and direction.
     • Objective: The business benefit or impact.
     • Outcome: The change in behavior measured by key results, leading to objectives.
     • Opportunity: A need, pain, desire, or job to be done better or more cheaply.
     • Solution: The output that creates the outcome.
2. Challenge Current Backlog Items
   • Reflect on your current product backlog and see how items fit into the picture drawn above. Disregard items that have become irrelevant.
3. Order Your Backlog
   • Prioritize opportunities according to upper-level priorities. Determine which objectives are key and how outcomes and opportunities contribute to these objectives. This helps identify which items to tackle first.
4. Start Refining
   • When gathering key information on your opportunities and ordering your backlog, identify assumptions. Assess the impact if assumptions turn out to be incorrect and design tests to start learning about your most important assumptions.

Toolbox

• Opportunity Solution Tree (OST)
• Impact Map
• Story Map
• Problem Identification Document (PID)
• Hypothesis Prioritization Canvas
• Product Backlog Sanity Check

‍

By implementing these steps, you can revive your product backlog for discovery, ensuring that it effectively supports continuous learning, value optimization, and alignment with your product vision.

‍

Tweak 2: Use MVPs for Learning

Why?

At this stage, our product backlog likely contains items we are confident about and can refine for delivery in the next sprint. However, there are often important backlog items that, while promising, remain uncertain. We need to address these uncertainties by setting up work to learn about our assumptions. To do this effectively, we suggest using Minimum Viable Product experiments (MVPes).

‍

What?

The term "Minimum Viable Product" (MVP) has been widely discussed, sometimes leading to confusion. Originally introduced by Frank Robinson in 2001, the MVP is described as:

‍

“The MVP is the right-sized product for your company and your customer. It is big enough to cause adoption, satisfaction, and sales, but not so big as to be bloated and risky. Technically, it is the product with maximum ROI divided by risk. The MVP is determined by revenue-weighting major features across your most relevant customers, not aggregating all requests for all features from all customers.”

‍

This definition emphasizes that the MVP is an actual product designed to be "just the right size" for current needs, performing well without being overbuilt.

‍

Eric Ries of Lean Startup later popularized the concept, highlighting its role in learning:

‍

“A version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.”

‍

This redefinition underscores learning as the main purpose, further clarified as:

‍

“The smallest thing you can make or do to test your hypothesis.”

‍

In the context of SAFe, MVPs are used in the lean startup cycle:

‍

“An MVP is an early and minimal version of a new product or business solution used to prove or disprove the epic hypothesis. Unlike storyboards, prototypes, mockups, wireframes, and other exploratory techniques, the MVP is an actual product that real customers can use to generate validated learning.”

‍

While this provides valuable insight into using MVPs, it also implies significant effort to create a product usable by real customers.

‍

We advocate for using the concept of MVPe (Minimum Viable Product Experiment), focusing on the experiment rather than the product. As Jeff Patton and others suggest, the goal is to create something—whether software or not—to learn effectively. This aligns with Kent Beck's additions to the Agile Manifesto, emphasizing validated learning and customer discovery.

‍

How?

To refine our top backlog items, we design MVP experiments for the most important assumptions. Tools like the Strategyzer's test and learning cards can structure this process:

‍

Test Card

1. Hypothesis: What needs to be true for your idea to work?
2. Test: How will you test if the hypothesis is true or false?
3. Metrics: What will you measure to validate or invalidate your hypothesis?
4. Success Criteria: What does success look like?

‍

Learning Card

1. Hypotheses Tested: Which hypotheses did you test?
2. Observations: What did you observe or learn?
3. Insights: What insights did you gain from the experiment?
4. Actions: How will you act on this learning?

‍

Conducting Tests

Testing is complex but improves with practice. Regular user interviews are a good start. Mixing qualitative and quantitative research provides a comprehensive understanding:

‍

• Qualitative Data: Helps understand why a problem occurs and the context behind it. It involves observing behaviors and accessing user thoughts. Interviews with five participants often reveal 85% of problems.
• Quantitative Data: Provides numerical insights and validation. It complements qualitative data, adding depth and breadth to findings.

‍

Toolbox

• Interviews: Validate assumptions about customers, their problems, and potential solutions.
• Prototyping: Create cost-effective prototypes to test ideas at varying fidelity levels.
• Concierge Tests: Manually perform tasks for users to understand their needs and validate solutions.

‍

Practice

Evaluate whether an MVPe is good or not by considering its alignment with learning objectives and feasibility.

‍

Food for Thought: How do MVPes align with Scrum?

The entire Scrum Team is accountable for creating a valuable, useful Increment every Sprint.

‍

The Product Owner proposes how the product could increase its value and utility in the current Sprint. The whole Scrum Team then collaborates to define a Sprint Goal that communicates why the Sprint is valuable to stakeholders.

‍

An Increment is a concrete stepping stone toward the Product Goal. Each Increment is additive to all prior Increments and thoroughly verified, ensuring that all Increments work together. In order to provide value, the Increment must be usable.

‍

Work cannot be considered part of an Increment unless it meets the Definition of Done.

‍

The learning from our experiments are concrete steps toward our product goal. Discovering the best ways to increase product value contributes significantly to achieving valuable and usable outcomes for the next steps, whether through further experimentation or delivery.

‍

Tweak 3: Make It Happen in the Dual World

Why?

Incorporating discovery work alongside development work within the sprint framework allows for continuous learning and adaptation toward the product goal. Acknowledging the dual nature of work—both development and discovery—is essential. Discovery should not be confined to a few individuals or a separate team but integrated into the entire team's workflow. Leveraging Scrum’s heartbeat, each sprint can serve as a structured opportunity to blend these efforts. Inspired by Jeff Patton, here’s how we can integrate discovery into Scrum ceremonies.

‍

How?

Sprint Planning - Laying Out the Work for the Sprint

• Prepare:
  • Select delivery stories a cycle or two in advance.
  • Workshop and refine stories ahead of time.
  • Review opportunities requiring discovery a day before planning. Identify items needing further learning and experimentation.
  • Invite the whole team to ensure collective ownership of both discovery and delivery work.
• Plan:
  • Start with a discussion of the big goals.
  • Review opportunities for discovery work.
  • Review stories to be developed into deliverable software.
  • Set a time-box for the team to plan independently.
  • In small groups, create plans for each story, including both discovery and delivery aspects.
  • Agree on a balanced plan to ensure preparedness for future work and avoid the pressure of "feeding the beast."
• "If you feel pressure to shorten discovery work to 'feed the beast,' it usually means you’re making decisions to build software before learning enough to be confident that you should."

‍

Daily Scrum - Inspect and Adapt Toward the Sprint Goal

• Share any team news or general announcements.
• Discuss development progress.
• Review preparation for the next cycle.
• Inspect progress on discovery work, adapt the sprint backlog based on new learnings, and plan any new experiments.

‍

Sprint Review - Inspect the Outcome and Plan Future Adaptations

• Evaluate the current product’s health and progress toward the product goal.
• Inspect discovery work by reviewing what was learned, the assumptions validated or invalidated, and insights gained from experiments.
  • Share test and learning cards to highlight key learnings.
• Evaluate development work and compare it against the original plan.
• Discuss any changes needed based on the review findings.

‍

Sprint Retrospective - Plan for Increased Quality and Effectiveness

• Reflect on the way the team worked, including discovery efforts.
  • Evaluate if enough time is allocated to discovery.
  • Assess the value and impact of discovery activities on initiatives.
  • Review touchpoints with customers and/or users for regular feedback.
  • Determine the quality and actionability of the learning.
  • Explore ways to improve whole-team participation in discovery.
  • Examine the efficiency of discovery activities and opportunities for industrialization.
    • Considerations include customer/user selection, interview kits, feedback processing, and insights sharing.
• Review changes made in the last cycle.
• Agree on specific changes for the next cycle, such as including a discovery story or conducting user interviews.

Toolbox

• Interviews:
  • Validate customer assumptions and problems.
  • Understand current solutions and requirements for switching.
• Prototyping:
  • Create cost-effective prototypes to test ideas.
  • Ensure prototypes are collaborative and appropriate for the learning objectives.
  • Examples include technical proofs of concept, Wizard of Oz tests, fake doors, landing pages, and mockups.
• Concierge Tests:
  • Manually perform tasks for users to understand needs and validate solutions.

‍

Practice

Evaluate whether an MVPe is good or not by considering its alignment with learning objectives and feasibility.

‍

Food for Thought: How Do MVPes Align with Scrum?

"The entire Scrum Team is accountable for creating a valuable, useful Increment every Sprint."

‍

"The Product Owner proposes how the product could increase its value and utility in the current Sprint. The whole Scrum Team then collaborates to define a Sprint Goal that communicates why the Sprint is valuable to stakeholders."

‍

"An Increment is a concrete stepping stone toward the Product Goal. Each Increment is additive to all prior Increments and thoroughly verified, ensuring that all Increments work together. In order to provide value, the Increment must be usable."

‍

"Work cannot be considered part of an Increment unless it meets the Definition of Done."

‍

Learning from our experiments are concrete steps toward the product goal. Discovering the best ways to increase product value contributes significantly to achieving valuable and usable outcomes for the next steps, whether through further experimentation or delivery.

‍

Tweak 4: Measure Discovery

Why?

During retrospectives, we aim to find ways to increase quality and effectiveness. However, assessing how well we are doing and whether we are improving can be challenging. To apply the experiment-measure-learn loop effectively to our process, having some key metrics is essential. While I generally caution against defining too many metrics, a few well-chosen ones can keep us on track with our continuous discovery habits.

‍

Key Considerations:

1. Finding Insights vs. Relevant Insights
   • Searching for something in numbers might lead you to find anything, but not necessarily the relevant thing.
2. Numbers as a Narrative Tool
   • Metrics can be used to support any narrative, so it’s important to use them wisely and objectively.
3. Balance
   • Avoid extremes. Either having too many metrics or none at all can be detrimental.
4. Qualitative Insights
   • Discussions with team members often reveal what’s working well and what needs improvement more effectively than numbers alone.

‍

Suggested Metrics:

1. Allocation to Delivery vs. Discovery
   • Measure the balance between delivery and discovery efforts. This can be done in various ways depending on your context:
     • Number of items in each category.
     • Number of story points allocated to each category.
     • Time spent on each category.
   • Keep it simple and avoid over-engineering.
2. Discovery Commitment Fulfillment
   • Track how well you are meeting your discovery goals:
     • Whether you met your discovery goals.
     • Number of assumptions "discovered" versus planned.
     • Story points related to "discovered" assumptions versus planned.
3. Product Success Metrics
   • Measure the success of your product goals. Failure to meet these goals may indicate issues with identifying the right problems or solutions.
4. Cost of Not Doing Discovery
   • Calculate the opportunity cost and sunk cost associated with not conducting adequate discovery.

‍

Implementation:

• Start Simple
  • The first two metrics are straightforward to implement and can act as triggers to initiate continuous discovery. Initially, these metrics will help establish the habit of discovery.
• Evolving Metrics
  • Over time, these initial metrics might become “vanity” metrics, simply measuring activity rather than value. Even so, they are valuable initially to help teams learn how to prioritize and conduct discovery work effectively.
• Ultimate Metrics
  • The final set of metrics, like product success and cost of not doing discovery, require more sophisticated tracking and instrumentation of your product. These metrics provide deeper insights into the true impact of your discovery efforts.

‍

By implementing these metrics, you can create a structured approach to measuring and improving your discovery processes. This will help your team maintain a balanced focus on both delivery and discovery, ultimately leading to better product outcomes and continuous improvement.

‍

Final Thoughts

To effectively integrate discovery work into your development process, follow these key steps:

1. Clarify Discovery Needs in Your Backlog
   • Ensure your product backlog clearly indicates which items require discovery work. This transparency helps prioritize learning and validates assumptions before investing in full-scale development.
2. Identify Key Experiments
   • Define and plan the experiments needed to validate assumptions and gather insights. Use tools like test and learning cards to structure your approach and ensure you are testing the right hypotheses.
3. Leverage Sprints for Discovery
   • Incorporate discovery activities into your sprint planning. Treat discovery work with the same importance as delivery tasks, ensuring the team allocates time and resources to both.
4. Measure and Improve Continuously
   • Implement metrics to track the effectiveness of your discovery efforts. Regularly review these metrics during retrospectives to identify areas for improvement and ensure your discovery practices are driving meaningful results.

‍

By following these steps, you can create a balanced approach that integrates discovery into your agile workflow, fostering continuous learning and delivering greater value to your customers and organization.