Review: The Tyranny of Metrics

The Tyranny of Metrics by Jerry Z. Muller
My rating: 3 of 5 stars

This approaches the role of metrics in running and in judging organizations. The authors illustrate the mismatches a focus on metrics typically provides within mission-driven organizations – think education, national defense, policing, and non-profits. This highlights the expenses and often questionable value of spending on optimizing metrics and rankings, as opposed to focusing on roles like “teaching” or “national security”. One point often repeated here is that schools are not for-profit companies, and will tend to keep increasing effort on metrics without concern for cost. And there is always gaming the system when the rules are well defined and accessible in a number or ranking format. So what is the alternative to these metrics? Professional judgement based on experience and reduced decision transparency while decisions are being made. This certainly harkens back to the days before spreadsheets, and that seems to be the point of this – not to throw out the good methods of the past just because measurement is easier now. The author simplifies his argument by disregarding the good that those changes have made. For instance, government transparency helps reduce corruption. Transparency is not just to expose decision points. The author also points out that the use of metrics for compensation purposes allows people to game the system to optimize their own returns at the expense of the organization as a whole or others in the organization. In my mind, metrics just sets down the rules. People in organizations without dependence on metrics also gamed the system (see corruption or “brown-nosing”). The author also inadvertently makes his own case that non-experts make too many assumptions when he describes jobs that can well be managed by metrics, including piecework assembly and, oddly, sales. The author seems to think these could be jobs replaced by robots. There can be different qualities of a sale that can be subjectively measured because they don’t appear until the future, but treating sales like one-size-fits-all doesn’t do justice to the profession and the value it provides to the organization. One is led to wonder if any job can truly be defined wholly through non-subjective numeric measures.

In summary I found this a good book for documenting some of the issues with over-reliance on metrics to run an organization and to provide guidance on individual’s compensation. The author provides guidance to eliminate some metrics, but the best advice is hinted at throughout the book – better understand what your metrics really measure and what they incent, and include the costs of gathering metrics in your analysis. Personally, I see the costs of gathering information, especially if technology is involved, is rapidly falling, and the use of technology tools for optimization such as AI and machine learning is greatly increasing. There is no escape from wider use of metrics.


View all my reviews