Measure the quality of your decisions....
and not just your data
Data will always take a back seat in the drive towards better decision-making. As companies spend increasing amounts of time and effort in capturing organizational and market data, the return on these investments will continue to depend on our ability to transform the data into impactful decisions. Data alone doesn’t produce good decisions for decision-making is constantly handicapped by uncertainty, ambiguity, and complexity. Measuring the quality of our decision-making may well prove more important than improving the quality of our data. Let’s look both at why measuring decision-making is so difficult, and why it is so potentially rewarding.
One of the obstacles in improving decision-making comes from the diversity of managerial challenges today. On an “objective” level we refer to problems as simple when the data at hand is sufficient to identify the one best way, and complex when the data provides nothing more the best answer in a given context. On a “subjective” level, a manager gauges risk when he or she understands the probabilities and outcomes of the choices before them. A manager confronts “uncertainty” when for one reason or another the probabilities and outcomes cannot be determined with precision. The factor of “ambiguity” weighs into the equation when the decision-maker questions the clarity of the problem itself. As a result, the goal of management is rarely about finding the right answer, and largely about helping mangers and customers take better decisions through reducing the sources of risk, uncertainty, and ambiguity.
Measuring decision-making is complicated by the fact that the “best” choice depends as much on the manager’s state of mind as on the nature of the problem itself. In Decision Science, we refer to four mindsets that condition decision-makers.[i] The optimist, like the lotto player, is always betting on the largest payoff possible regardless of the probabilities. Inversely, visions of the worst haunt the pessimist who will religiously try to minimize potential losses. The opportunist will employ reasoning similar to Jeff Bezos’ regret minimization framework by focusing on outcomes that provide the greatest peace of mind.[ii] Finally, the realist calculates the potential return of each available outcome and tries to maximize expected value. Because there is no one right answer, it is fundamental to explore how each manager’s mindset corresponds the prevailing organizational culture.
The metrics used to measure decision-making process provide a third level of complexity. What exactly are we trying to optimize (or minimize) when looking for the better decision? We can focus on yield — what are the potential benefits of each of the alternatives? We might focus on the effort — how many resources, and much energy, must be mobilized to put the choice into practice? Perhaps we should focus instead on velocity, given our other responsibilities, how quickly can we find a suitable decision? Finally, we could evaluate the pertinence of the model itself, how squarely does the proposed solution address the problem at hand? The proper metrics depend upon how the organization, and the industry, measure value.[iii]
Finally, we can consider the operational challenges managers face when tring to take the best decision in a given context. Let’s take the self-serving example of improving the organization’s decision-making processes. The first complication comes from the problem itself — is there an organizational consensus of what we mean by decision making? There is the issue of the pertinence of the data at hand — what proof do we have today that we are taking poor decisions? There can be discussion on the objective we are trying to reach (the pertinence, the velocity, the effort, or the yield of each decision?). Finally, how can study what might be the best means of testing our proposals (sampling, simulation, a survey, face to face interviews)?
Like any skill, improving decision-making requires both analysis and practice. In the Business Analytics Institute you will study context specific challenges, mindsets and metrics of managerial decision making. In our Summer School in Bayonne, as well as in our Master Classes in Europe, we put data science to work for you and for your organization. The Institute focuses on five applications of of data science for managers: working in the digital age, data-driven decision making, machine learning, community management, and visual communications. Data-driven decision making can make difference in your future work and career.
Lee Schlenker is a Professor at ESC Pau, and a Principal in the Business Analytics Institute http://baieurope.com. His LinkedIn profile can be viewed atwww.linkedin.com/in/leeschlenker. You can follow us on Twitter athttps://twitter.com/DSign4Analytics
[i] Kasture, R. (2015), Decision Making, Tolerance to Ambiguity and Regret Theory, Vivify
[ii] Stillman, J. (2016), How Amazon’s Jeff Bezos Made One of the Toughest Decisions of His Career, Inc.
[iii] Blenko, M. and Mankins M. (2012), Measuring decision effectiveness, Bain and Co.