“Prediction is very difficult, especially if it’s about the future” Niels Bohr
I like the analogy of the lottery ticket (heard at an Agile user group, just don’t remember who said it, sorry), you know the cost of the ticket, you just don’t know how much it will pay, if anything.
The challenge of defining value is that you can only really measure it after the fact, indeed you cannot really measure the value of a software functionality before it gets to be used in production, by all your users (so you can analyze added revenues or decrease in cost). At best, before that, you can only “predict” value, and unfortunately decisions made around developing functionalities are still based on whom screams the loudest or a prioritization exercise that is based on perception. So predictions are more likely to be biased.
Recap of a recent discussion with a PO at the same company:
“Hey Mandy, how are you? Any more challenges with writing user stories? “
“Hey Fred, I’m great. Thanks. No, all is good. I have been working closely with the team to make sure they understand the requirements and what to build.”
“That’s great. I noticed your backlog was growing a lot after each sprint, I guess you are getting good feedback from the customers. Are they in the room during the demo?”
“Oh no, it’s because I typically find that I forgot to ask for specific components in the original user stories, so I create new ones with the missing acceptance criteria. Also, as I see the features working, I’m documenting all those nice things we could do to make our product more appealing to customers.”
“You mean new features?”
“Some yes, but mainly UI. As you know I have a background in UI design so I’m always thinking about how to make the product look nicer. I really think our customers will like it better with those changes.”
….
(Ctd)
“Are you delivering more often? Did you change your release schedule?”
“Not really. The business does not want to deliver too often. They don’t have the time to support a release to end users faster than every 3 months.”
“So you reviewed the priorities of the backlog with them?
“I tried, but they told me they did not want to tell me their priorities, because if they do they feel then we won’t do all those other things they want”.
…
Just a few examples of what can go wrong in measuring value. It implies, or could imply among other things, that:
Bottom line is that the software organization is not building the cheapest possible solution.
How do you measure if improving UI yields more paying customers? Or reduce the time it takes someone to perform an operation, thus reducing overall cost of the process or improving overall ROI?
Before the fact, and without added cost, you really can’t (8).
The best way to measure the real value of the features you are building is to deliver more often, frequently and faster. Not only because you get early feedback on what you developed but also, and potentially more importantly, because you can then change course if you realize that what you have next in your backlog won’t bring as much value ($) as predicted or if you get requests from customers to build something of higher value that they want more urgently and are committed on buying (no more prediction!).
In addition, you will drive the cost of the solution down by not working on those features that have less value.
Furthermore, delivering value more often not only provides value more quickly (faster ROI, better capitalization of projects, less financial investment up front keeping money in the bank, etc.), but it also increase the overall cumulative value of the project. Indeed by adapting quicker, changing course if required, you will, more than likely, end up delivering features of higher value.
That’s why you want to implement DevOps principles and practices. My colleagues and I have other blog posts on that critical topic. More to come on DevOps later.
In conclusion, the formal way of calculating ROI might not work anymore, you still need to understand cost and value obviously, but we need to look at them differently than in the past. And we need to implement tools like Visual Studio for technical metrics and principles and practices related to DevOps to increase overall value. We also need to look into other technologies and concepts, the Microsoft Cloud, Azure, for example, to help reduce cost and improve DevOps practices.
Spoiler alert: Future blog posts will take more about the people and culture components of DevOps, some additional tooling and predictive analysis. Stay tuned. In the meantime, if you want to know the value/ROI of ALM for you, check this: http://incycle-corpweb.azurewebsites.net/2014/06/doing-alm-right-real-world-evidence-alm-works-way-calculate-your-potential-roi-2/ - Free tool!
------
Additional reading:
Stop Trying to Predict Which New Products Will Succeed (HBR) -N. Taylor Thompson
http://pragmaticmarketing.com/resources/why-prioritizing-your-agile-backlog-for-roi-doesnt-work
How to Measure Anything: Finding the Value of Intangibles in Business-by Douglas W. Hubbard