As my dissertation is reaching it’s conclusion, I am working towards making more of its research accessible via the web.  This post continues the series of interviews that I performed as part of that research on Agent-Based Modeling and intelligence analysis.  In the interests of full-disclosure, Joe was my boss for more than three years (2001-2004) when I worked for him as a research associate in the Center for Technology and National Security Policy at National Defense University.  Under’s Joe’s and Desmond Saunders-Newton’s mentoring, I was introduced to Agent-Based Modeling and complexity theory.  I owe Joe a special debt professionally and intellectually and am proud to have benefited from his mentoring.  This interview was conducted on September 6, 2012.

Background:  Joseph Eash, III (JE) had a distinguished career in the US Air Force and Pentagon, where he led the development of advanced technologies for many decades.  He served at the National Reconnaissance Office, for which he received the Pioneer Award, served as the Deputy Undersecretary of Defense for Advanced Technology, and retired from government in 2005 after serving as a Chief Scientist for Computational Social Science Modeling at the Center for Technology and National Security Policy at National Defense University.

Discussion:  I asked JE several questions about his time in government, where he served in intelligence, technology, and policy positions.  As a result, he had a unique set of experiences upon to draw upon when considering the development and integration of new analytic technologies and methods into the intelligence community.

JE started by noting that we will never perfect information or intelligence, but that the ranges of what can happen can be understood by understanding technical things and the processes that people follow to develop artifacts or execute plans.  What we do is spend time developing templates or patterns of how activities are performed, and then sample from the world in an effort to see if we can fill in the templates.

JE noted that his characterization of intelligence introduced several challenges.  First, the problem of templates was largely a theoretical one.  Templates are essentially characterizations of how something might be accomplished, whether one seeks to develop a nuclear weapon or run a political campaign.  In a sense, they might be considered algorithms that constitute a set of steps to be followed in order to achieve a particular goal.  Collection then samples from the world and attempts to match observations against templates in order to understand what is happening and what might happen.  This means that intelligence requires both imaginative, theoretical work to develop libraries of templates, and also an understanding of collection systems and limitations in order to precisely understand and communicate what information is gathered and what is not.  Importantly, JE noted that in this context a single piece of information or  intelligence has very little meaning – what matters is how it relates with other information in order to fill in a template.

JE noted that in many cases we look for things with no template, or an unknown template.  This leads to problems regarding what to do with collected information.  We stretch it, exaggerate it, do a bunch of things to make sense out of it, but absent a general template reflecting a robust understanding of people, processes, etc. we are likely to misunderstand what we gather.

JE continued by noting the importance of cognitive factors in the analytic process.  He discussed the transition from film to digital images within the intelligence community.  He noted that one of the curious aspects of analysis was how the brain affected the perception of images.  When images were obscured, analysts often showed greater confidence in their ability to identify its contents than when it was clearer.  Moreover, once an analyst committed to particular identification or interpretation, it was very difficult to back them off their conclusion, even as the image was made sharper, the brain got stuck and attached to a particular interpretation.

JE believed that the cognitive challenges of interpreting imagery were representative of a larger range of analytic challenges.  He noted that in many cases, the available information is often of low quality, akin to the obscured image, but analysts must make judgments from them, and this process can lock errors into the analytic process.  He also noted how important good, high quality data is early in the analytic process.  Because analysts were more accurate when first shown a high quality image than a low quality image, but often resisted revising assessments from low quality images after being show a high quality image, an asymmetry existed where more information was needed to change an observers mind than to shape it in the first place.

JE argued that one of the major problems in intelligence and policy is that analysts and decision-makers are making assessments and policies with mental models or templates of unknown quality, and that the information going into them is also of poor or unknown quality.  As a result, everyone is vulnerable to seeing what they wish to see and unless challenged by high quality information, their perceptual biases are likely to carry the analysis.  He continued by noting that in many cases, the underlying theoretical work required to develop templates has not been done, and that the information we can collect cannot justify the conclusions we are trying to reach.

JE compared the problem of analysis to stealth in radar world.  He noted that collecting against a stealthy vehicle was a difficult problem – collection systems only get glimpses of the target, and these glimpses could be quite difficult to integrate into a common picture, e.g. were they of the same vehicle of or multiple vehicles?  In the abstract, the problem was the same – how to get the most out of low-resolution information?  Better collection was one possibility, which meant collecting with a better sample rate or resolution.  A second solution was to develop new templates that would allow analysis to proceed with less information.

I followed up by asking JE about the differences between intelligence problems in the technical domain and the development of new technologies compares with social science problems that became increasingly important during the latter years of his career.  JE noted that the social domain extended the range of considerations and duration of time to be concerned with, but also the opportunities to develop an understanding of the international system.  He noted that in a simple case, e.g. determining if an attack was occurring, one could look into the technical domain to look at the horizon using optical, infrared, acoustic, and other sensors.  However, extending into the social domain opened the problem up to provide new indicators for warning, many of which would be sooner than the technical ones, e.g. the execution of organizational procedures for attack preparation or social indicators such as communications between family members.

JE argued that the social dimension provided new dimensions across which signals could be correlated to gain a more complete and accurate understanding of current events and potential developments.  His turn towards the social sciences was then largely about finding new ways to task other technical collection systems by cuing off of social findings, e.g. if it was discovered a country’s economy was particular sensitive to a particular commodity price, then financial markets could provide indicators of impending political and social instability.  JE argued that by extending analysis beyond technical indicators and templates by incorporating social information, customized regional, national, and subnational models could be developed to provide customized warning for particular cases and improve the quality of warning available to consumers.  Importantly, JE noted that while these templates would draw upon social science and social indicators, each would be different and tied to particular cases and circumstances.

JE concluded by observing that one of the difficulties in integrating social science and indicators into the intelligence system was the entrenched institutional interests of expensive, technical collection systems that are the object of budget and corporate priorities.  Thus, investments in the social sciences and models faced difficult internal competition when paired against more expensive, traditional collection capabilities, even though the two necessarily enhanced the capabilities and effectiveness of the other.

My next question to JE was about his time as a consumer of intelligence rather than a producer.  I was particularly interested in his perspective on what made analysis effective or useful from the perspective of policy-makers.  JE answered that he always towards analysts that employed a clear and transparent process for discovering, explaining, and eliminating alternative explanations of events.  Whenever analysts did not consider, explore, or provide alternative explanation he noted that he became cautious and worried that analysts were engaged in advocacy for particular views rather than analysis.  He also paid great attention to the diversity and use of sources, noting that checks against foreign denial and deception and efforts to cross-validate information were crucial aspects of intelligence and establishing his confidence in the quality of their products.  Whenever collection and analysis relied on a single source, he became cautious and always wanted a way to trace back analytic judgments to the underlying source material.

Finally, I asked JE about the future of intelligence and what he thought would be most important going forward into the future.  He noted that our increasing understanding of cognition and the brain was key to improving analysis and understanding the limits, strengths, and failures of human perception and the use of information.  He believes that ongoing work in brain scanning and imaging as a way of understanding how beliefs develop and change will make important contributions to improving analysis.  Finally, JE noted that understanding cognition will necessarily need to include a better understanding of emotion and its role in analysis and decision-making.  Indeed, JE believed that many of the difficult intelligence problems are questions of understanding ‘why’ people behave as they do (individually or collectively), and that successfully addressing such concerns requires an understanding of their emotional states, needs, and predisposition as well as more traditional rational considerations.