Thomas Davenport and Jeanne Harris’s Competing on Analytics: The New Science of Winning (Boston, MA: Harvard Business School Press, 2007) (hereafter COA) was a surprising read.  I found the foreword by Gary Loveman, the CEO of Harrah’s Entertainment, Inc., to be very engaging and insightful, and lifted my expectations for the chapters that followed.  Indeed, the conclusion of his introduction was exceptionally relevant to the problems that policymakers, and the analysts that support them, face in the international system:

The challenge for those of who attempt to employ analytic capabilities is to ensure that they are oriented forward, where the problems are least well defined and the data is scarce, rather than backward, where the work is easy and the risk is low.  Read ahead.  There is much to learn. (p. xi)

Unfortunately, Davenport and Harris were unable to meet the high expectations that were set for them.

COA’s problems exist on two levels:

1. First, as a book geared towards a business or management school audience, it is not tailored to a scientific audience.  This may be considered a fault of the reader and not the authors.
2. Second, the scientific content of the book is suspect at best, and a critical examination of the text and the authors’ arguments suggest it is promoting a particular approach to business management and decision-making as opposed to dispassionately assessing the merits, successes, and failures of analytics in business on the whole.

The book’s arguments are entirely based on anecdotes, and surveys of overworked CEOs, CIOs, and never provide a complete picture of what really works for organizations as a whole.  There is nothing to suggest that those companies that invested in analytic capabilities significantly outperform their competitors – COA never convinces the reader that those companies that are succeeding are doing anything different from those that are not.  Moreover, given the business and management focus of the publisher and authors, I’m unsure of the extent to which its defects are a result of editorial decisions motivated by the COA’s targeted readers, or a more serious lack of depth and understanding of tools and techniques that its authors don’t fully appreciate.  I suspect the authors are far more knowledgeable than their book suggests, yet, there are moments where the authors make claims that err in such fundamental ways that I have to wonder if there are deeper problems.  For example, the authors argue that statistical analysis can be employed to answer the question of “why is this happening?” (p. 8), thus conflating correlation and causation.

Despite my early disappointment, I worked through COA in its entirety and found the second half quite interesting, informative, and consistent with many of my own experiences using models to support analysts and policy-makers.  Whereas the first half of the book characterized analytic competition and provided a taxonomy for how firms use data and models to compete in markets, it never really established a solid theoretical or empirical foundation upon which to rest their arguments.  From a theoretical perspective, there wasn’t a clear or consistent picture of how firms or managers operated, particularly with respect to their use of information and analytical staff.  From an empirical perspective, the examples were too anecdotal, simple, and based on highly skeptical and limited data sources.  As a result, COA’s conceptual model of business competition was unconvincing, and further weakened by its apparently limited understanding of the precise kinds of knowledge different analytic methods produce (this was made more acute when the authors attempted to discuss novel techniques like simulations and “swarm intelligence” that vastly overpromised the character and quality of the predictions of these techniques).

COA’s largest weakness is subtle but persistent throughout the book, lurking in the shadow cast the foreword’s bright warning lights.  The author’s regularly promote the power of evidence, data and facts.  While this may appear to give power to scientific and objective decision-making, it actually dooms its followers to a very peculiar and limited understanding of the world, forever trapped by induction and the assumption that the future will be like the past.  Thus, while the foreword noted that the problems of the executive are in the uncertain future, beyond the facts available today, COA’s authors consistently pressed readers to rely on data above everything else.  Ultimately, their approach is too grounded in empiricism to cope with major uncertainties that organization’s face – after all, the toughest problems are those that haven’t yet been experienced.  A greater focus on the scientific aspects that undergird the analytic methods they are promoting would greatly improve COA’s value to readers, and done more to relate the promise, potential, and pitfalls of analytic methods to decision-making.

Ironically, the weaknesses of the first half of COA proved to be its strengths for the second half.  As the authors’ shifted the subject from general concepts of business competition to the development of analytic capabilities within firms, their practical-minded approach to building analytic capabilities within organizations provided many insights.  Interested readers would do well to read the foreword, and then skip to the second half of the book.  COA’s core argument is rather simple – analysis succeeds or fails based on the quality of the relationships between analysts, decision-makers and managers.  Factors such as technology, scientific rigor, objectivity, rationality, etc. may be important, but often constitute distractions or even threats to senior leaders and budding analytic units looking to prove their value.  As a result, analysis cannot be viewed as an isolated process of data collection and evaluation, but must be evaluated and embedded within the firm’s larger decision-making strategy and organizational structure.  Having worked with various government organizations on how to integrate Agent-Based Modeling into analytic processes, I found the arguments of COA to be consistent with my own experiences and elegantly and authoritatively stated.

There are three final points worth making about COA.

1. First, as a text targeted to commercial, private sector firms, it was rather interesting to see how vastly different the public and private sectors are in terms of their goals.  Much of COA is concerned with maximizing profitability, which, of course, is a perfectly fine pursuit for private-sector organizations.  For example, the authors note that banks and other service oriented firms should employ analytics to estimate the long-term value of individual customers and treat them accordingly.  Thus, a customer with a high potential to provide repeat business may be offered special deals or services, have various fees waived, and receive other special treatments in order to establish a long-term relationship.  Such practices might prove problematic for the provisioning of government services, e.g., should individuals that pay more in taxes or have higher tax paying potential receive special services or favorable treatment from the government?  Coming from a background in government and public policy, it was quite interesting to read a book that treated the consumers of an organization’s services as customers as opposed to citizens.
2. The second point follows from the first.  The major emphasis of the book was on using analytics to minimize risk and maximize profit.  This not only included pursuing long-term relationships with desirable customers, but also avoiding situations that were potentially risky or unprofitable.  Again, this provided a stark contrast to the role of government and governing institutions that are essentially in the business of absorbing risk.  I suspect buried deep in these differences are important implications for analysis and the suitability of different methods and models.  After all, it is one thing to use analysis to determine when to get out of a market because it is no longer profitable, and quite another to understand how to provide services efficiently as a government, knowing that they cannot be done profitably and equitably.
3. Finally, the very existence of this book is a testimony to the fallacy of treating companies as rational actors, finely tuned their environments and constantly and effectively calculating their best interests.  The authors’ anecdotes and surveys reveal firms to be rife with organizational rivalries, captured by their traditions, resistant to innovation, prone to the personal whims of their leaders, ignorant of their actual costs or critical market trends, etc.  The fact that COA’s authors argue that a small minority of companies have invested in sophisticated analytic capabilities and employ them well, while the majority of firms are only flirting with the simplest use of data is surprising and shatters the mythical world described by economists where firms deftly use all of the information available to them to make optimal decisions.  Simply put, if this was the case, there would be no need for books like COA.