In my executive presentations I spend a fair amount of time on the topic of Do Less, talking about how we waste an incredible amount of time and money building features that are rarely or never used. Published studies put this number at far greater than 50%. I see audience members agreeing, but I can also see the little thought bubble floating slightly above their heads, “but not in MY organization.” Organizations worry about improving development productivity by 10% when what they should be worried about is improving customer demand effectiveness by 25%. I’ve always said that one of the biggest potential productivity improvements from Agile lies in all the features that we don’t do because of the constant attention to simplicity and highest value to the customer.
However, in most organizations, software development efficiencies are subjected to infinite analysis while customer demand effectiveness isn’t mentioned at all. We neither measure nor calculate feature value in the beginning nor do we measure whether or not value was actually captured as the customer or product manager had predicted. In many large companies project ROI might be calculated as part of the portfolio management process, but rarely does anyone follow up to see if that ROI was in fact attained. So development teams have feedback mechanisms while customer/product teams have few except possibly at a macro level (product sales for example). Product managers are left with gut feel for most of their feature-level decisions. No wonder 50%+ of developed features are rarely or never used. Product management teams aren’t the culprit here however, lack of adequate feedback mechanisms are.
If we look at a typical Agile project team, there is the development sub-team and the product/customer sub-team. Roles, responsibilities and feedback mechanisms have been defined, so for example, the product team identifies stories, writes stories, prioritizes them, and develops acceptance criteria (both automated acceptance tests and feature showcase evaluations). So there are micro-level (feature and story) feedback mechanisms to steer the development effort. But what about feedback from product management to the business? These are often much more tenuous, such as overall sales that tell how a product is doing at a macro level, but say nothing about individual features. Internal IT products have even less feedback in most cases. Lack of feedback leads to feature bloat since it’s always easy to succumb to customer requests and internal demands to improve the product.
At the Agile Brazil 2011 conference I was listening to Josh Keriesvky’s talk on his Lean Startup experience, and gravitated to thinking about this problem. Here’s a starter list of ideas about potential solutions:
- Develop Customer Demand Effectiveness measures for every product management organization and team.
- Calculate relative or monetary value for every feature.
- Use relative benefit dials such as increasing customer happiness, reducing customer risk, improving our internal collaboration culture, to evaluate feature value.
- Build feature usage information into software to provide feedback to product management. Product manages could thereby gain insight into what was actually used and what wasn’t.
- Develop feature evaluation experiments into your feature identification and prioritization process. For example do A/B testing on features.
- False feature analysis. Give users access to a feature, that when selected pops up a message something like “this feature is under development. When completed are you ‘very likely’, ‘somewhat likely,’ or ‘not likely,’ to use it. Josh talked about an expensive feature that his company decided not to implement because of the results from this type of test.
- Use short, focused surveys to take a measure of customer happiness. For example, “What was your experience using capability Y (capability being several features)? Awesome, OK, Not so Hot.
While there has been a lot alluded to in the Agile and Lean communities about value-based development, the actual practices to support this objective are not yet sufficient. There aren’t sufficient feedback mechanisms at the feature level to help mitigate the constant push towards feature bloat. Maybe taking a look at some of the ideas from Lean Startup and other sources can help.
These are some of my preliminary ideas. What are your ideas about this issue?
Hi,
That’s an interesting topic that is rarely discussed. Though not directly the target, I’d like to share an experiment a collegue told me about. It’s not about developing a feature or not, but about maintaining it, plus maintaining all interactions it has with other features/modules.
On an old product that he was maintaining, he wanted to know if a particular feature was used or not. He was almost sure this feature wasn’t used, and he knew it would be expensive to develop new ones while maintaining that one. So he introduced a voluntary bug, with a clear log message to make it easy to fix. He never had any feedback about this “bug”, so he removed that piece of code after some months/years. He used this technic again a couple of times afterwards.
I know, it’s not elegant at all, and I wouldn’t be proud using this trick (even if I’ve been very tempted a couple of times). But it worked at least once, so it deserves being shared. And it’s the occasion to talk of a side effect of developing unused features: maintaining them + maintaining interactions.
I’m still not proud sharing this hack ;o) BTW, I’m *not* this collegue.
Hi Jim,
I hope you have enjoyed your time here in Brasil!
At Locaweb we use some tools and techniques to get feedback from users to decide the features that will be built into a system:
- user interviews when we actively ask customers what they expect from our system, what are their goals and how we can help them.
- feedback tools like UserVoice to collect feedback.
- conjoint analysis is also a great methodology. Normally when customers get asked what they want, they want all. With conjoint analysis you present your customer (or prospect) with certain sets of features with different prices and they have to choose.
- usability tests provide a different view on customer needs because instead of asking what the customer needs, we watch the customer using our software to accomplish a certain task and get insights from this observation.
- also from observation, it is good to watch a user does 5 minutes after and 5 minutes before she used our software. Sometimes she does something that could easily be incorporated into the software to create great value to the user.
Thanks,
Joca.
Hi, Jim!
Actually, there is a very well documented method to do value-based development called EVO – Evolutionary Project Management, by Tom and Kai Gilb (http://gilb.com). Have you ever read it?
There is also a great paper focused on the Impact Estimation technique (a topic covered in EVO), that I linked in a recent blog post of mine: http://tiagomjorge.wordpress.com/2011/06/15/measurable-value-with-agile-the-impact-estimation-technique/
Best regards!
Tiago.
I have read Tom’s work. There are several approaches published to do this, including some I’ve developed that’s in my Agile PM book. We need more work and research in this area. However, we also need to keep the calculations simple so people will use them.
Jim
These principles apply to any business innovation. Good application to this particular business!