It’s been several weeks since my last posting examining.  Over that time, I’ve been working towards the completion of a project that has taken a significant chunk of my time and forced me to think about the use of ABM in the study of international relations.  In doing so, I’ve begun to explore the philosophy of science and the extent to which theories of international relations are really amenable to empirical testing.

What captured my attention over this period was returning to Ken Waltz’s Theory of International Politics.  I have always admired Waltz’s book despite the fact that I completely find neorealism unconvincing.  Instead, it is his views on science that is of greatest interest, but also the most problematic when compared with the practice.

In order to understand this problem, consider Waltz’s description of neorealism.  Waltz’s theory rests on two major assumptions:

• All states are functionally identical and only differ with respect to their relative power, or the distribution of capabilities;
• Relations between states are characterized by the condition of anarchy, where no higher authority exists for states to appeal to when threatened, as a result, they can only rely on their own capabilities in order to preserve their independence as political units.

Together, these assumptions define a system where states are continuously seeking a balance of power, and generate different system structures based on the distribution of capabilities amongst the actors.

What is interesting about Waltz’s theory is not the particular assumptions or their implications, but rather his insistence on separating out his theory of international politics from that of international policy.  By attempting to isolate the effects of the international system’s structure on the behavior of states, Waltz sought out a pure and parsimonious representation of the international system freed from all other influences, such as domestic politics, culture, ideology, and psychology – the contents of 1^st and 2^nd images defined in his earlier book, Man, the State, and War.

The most fascinating part of Waltz’s work was the small role of empirical analysis.  Two of the most telling passages from Theory of International Politics are:

• (1) A theory contains at least one theoretical assumption.  Such assumptions are not factual.  One therefore cannot legitimately ask if they are true, but only if they are useful.  (2) Theories must be evaluated in terms of what they claim to explain.  Balance-of-power theory claims to explain the results of states’ actions, under given conditions, and those results may not be foreshadowed in any of the actors’ motives or be contained as objectives in their policies.  (3) Theory, as a general explanatory system, cannot account for particularities. – pp. 117-118.
• Structure, however, does not by any means explain everything.  I say this again because the charge of structural determinism is easy to make.  To explain outcomes one must look at the capabilities, the actions, and the interactions of states, as well as the structure of their systems.  States armed with nuclear weapons may have stronger incentives to avoid war than states armed conventionally.  The United States and the Soviet Union may have found it harder to learn to live with each other in the 1940s and ’50s than more experienced and less ideological nations would have.  Causes at both the national and the international level make the world more or less peaceful and stable.  I concentrate attention at the international level because the effects of structure are usually overlooked or misunderstood and because I am writing a theory of international politics, not of foreign policy. – pp. 174-175

These two passages identify the crux of the problem of empirical analysis in international relations.  On the one hand, Waltz sought to isolate the effects of the structure of the international system on the behavior of states.  On the other hand, he has dismissed the notion that a structural theory can predict the behavior of states in particular cases.  The problem is, however, that the empirical world provides collections of cases that are all particular.  Thus, Waltz has, correctly I believe, identified that problem where the situation he is most interested in is the characteristics and expectations of a general model of the international system, but this general model cannot be tested because the real world only provides collections of particular cases.

Nancy Cartwright’s How the Laws of Physics Lie provides a very good discussion of this problem.  Cartwright notes that modelers continuously face a dilemma where they are taught to search for and construct increasingly generalized models that can apply to an ever expanding number of cases, and that as models reflect increasingly fundamental laws that determine the behavior of systems, their predictive power should increase.  However, reality presents the opposite outcome.  As models become increasingly generalized, they rely increasingly on theoretical or unobservable variables, while losing track of the particular details of individual cases and it is these particular details that determine the outcomes of the cases that are observed.  Thus, as our models become more general, they lose, not gain, predictive and explanatory power because they rely in increasingly abstract entities and properties while losing their empirical grounding.

At this point, the unity of nature becomes an issue.  While general models isolate particular variables in search of the consequences of fundamental laws, the unity of nature argues that the world always presents itself in a unified fashion, where all forces, fundamental and incidental, operate simultaneously.  Thus, any model that emphasizes some laws over others necessarily presents users with an unreality—a representation of the problem this is partial, incomplete, and never specific.

Returning to Waltz, it becomes easy to understand why he believed that his theory could not account for the particular choices of individual states, and was therefore, unlikely to be affected by the outcomes of empirical tests.  The problem then becomes whether it is actually possible to test any theoretical argument in international relations from empirical cases.  This question is far larger than just Waltz’s theory, but essentially begs the question for all of the international relations as a whole, although I’m reluctant to extend this argument to all of social science, for reasons I’ll discuss later.

The standard practice in international relations scholarship is to propose a theory, identify what it predicts in a given set of circumstances, and then examine those predictions against a case or set of cases (this is a largely deductive approach, and there is an inductive counterpart as well).  The problem that Waltz identified, however, suggests that this approach to research design cannot actually falsify theoretical claims because intervening factors may simply drown out or obscure the information that is most important for evaluating the theory being examined—the observed outcome may simply not be determined by factors relevant to the terms embedded in the theory being tested.

Modeling and simulation may be more important for exploring the implications of theoretical claims and testing their predictions, precisely because artificial societies can be controlled experimentally.  These approaches can eliminate the problem of intervening variables and forces, but do so at a high price—shifting the evaluation of the theory from the empirical world to an artificial one.

The advantages of testing theories in an artificial society are obvious—control and isolation over variables that cannot be measured or otherwise managed empirically.  Alternatively, the problem with an artificial society is determining whether or not is has been suitably constructed and contains enough richness to provide a meaningful experiment.  This leads to the troubling conclusions of precisely how to test theories empirically, and if they cannot be tested, then what?

Again, Waltz correctly defended theories based on whether they are useful not truthful.  In pushing this argument forward, the importance of theory is based on its ability to shape expectations about future events by providing an intellectual aid for projecting forward contemporary circumstances into unknowable futures.  Yet, these projections should not be interpreted as predictions because they contain incomplete assessments of the real world situation, inevitably leaving out some particular details and amplifying others.

My belief, then, is that the gap between historians, journalists, and others deeply focused on the specifics of cases and theorists interested in theoretical generalizations may be far larger than I had previously believed.  I assume that if one places strong enough faith in induction, the collection of enough cases can be taken to provide a suitable test of theoretical claims, but that requires strong assumptions that the universe of cases sufficiently cancel out specific oddities and allow for insights beyond their specifics.  Perhaps this is true, but I suspect that such conclusions rest strongly on the collection of datasets and the coding of cases, both of which are not always straight forward.  For example, if one counted the number of democratic states in the Middle East, would Iran be included?  My sense, and fear, is that many of the tests run in the assessment of international relations theories are not robust to the gamut of available data and coding options, so the durability of their conclusions should not be assumed.

Earlier I suggested that this problem may not exist for all of the social sciences, even if it is persistent in international relations.  The reason for this claim has to do with the existence of natural units within hierarchical systems.  An essential problem of complex systems is that there are many different ways to examine them and their features and behavior can change based on the scale employed in analysis.  A couple of years ago I was able to ask Simon Levin how ecologists determined the scale at which they studied complex systems, and he pointed to the existence of natural units.  Simply put, it makes sense to study a watershed or forest based on the number of individual organisms or species, but probably not by the number of feet, eyes, or antenna in the system.

I never really considered this point until recently.  International relations theorists generally assume that the state is a natural unit within the international system—after all, if there was no distinction between international politics and domestic politics then the entire rationale for international relations as a distinct discipline evaporates.  However, the contemporary problems of international relations, security, and economics certainly don’t conform to this notion of settled, unified polities, and the blood sport that currently characterizes US politics suggest that national identities, even in the world’s most powerful state, are far from unified or “natural”—precisely what it means to be a good American is highly contested, particularly when supposedly responsible and senior politicians casually toss around terms such as “treasonous” and “impeachment” to express disagreements over policy choices.  Increasingly, international relations theories may rest on the study of theoretical units, not just theoretical forces, which further distances it from being an empirical science.  Alternatively, the study of more structured and formally organized political parties, civic groups, bowling clubs, etc. may in fact possess sufficiently stable identities and organization to be regarded as a natural unit and level of analysis.  Thus, it may be the case that the problems facing international relations do not necessarily affect other social sciences, although they too may encounter this or other difficulties.

The pessimistic nature of this post is not to disparage international relations as a discipline or its theories.  In fact, I’m far more interested in defending theorization in much the same way that Waltz did by noting that it does not provide us with a truthful understanding of the world, but does provide us with useful explanations as to what could happen and why.  In many ways, I think that empirical testing may actually provide a misleading standard for determining the value of international relations theories, and that those most interested in understanding the implications of theory would be better served working within artificial worlds, rather than test against historical data, precisely to eliminate the problems posed by the unity of nature.  The risk, however, is confusing the artificial world with the real one—what works in a lab does not necessarily work in the wild.  Likewise the problem of empirical science and the unity of nature leads to the well stated conclusion that “the devil is in the details,” and no amount of theorizing can change this.