Assessing the
Quality of Democracy:
Freedom,
Competitiveness, and Participation in 18 Latin American Countries*
David Altman
The Helen Kellogg
Institute for International Studies University of Notre Dame 217 O’Shaughnessy Hall Notre Dame, IN 46556-0368 daltman@nd.edu |
Aníbal Pérez-Liñán Department of Political ScienceUniversity of Pittsburgh4L01 Posvar Hall Pittsburgh, PA 15280 |
Version: June 1, 2001 (7:15)
Abstract
This paper explores the problem of conceptualizing and measuring the quality of democracy in Latin America. The first part discusses the use of the concept and the need for an operational measure. We explore three dimensions of the quality of democracy: civil rights, participation, and effective competition. In the second part we develop an indicator of effective competition, one of the key dimensions of the concept. The third part analyses the empirical relationship between all three dimensions in 18 Latin American countries between 1978 and 1996. We construct summary measures of the quality of democracy in several ways, and show that the ranking of the cases is highly consistent no matter the procedure applied. In the last section we test the validity of our measure and discuss its limitations.
Article length: 6,513 words.
* We are indebted to Ana María Bejarano, Daniel
Buquet, Miguel Centellas, Rossana Castiglioni, David Collier, Michael Coppedge,
Robert Fishman, Fito Garce, Kim Quaile Hill, Nestor Legnani, Scott Mainwaring,
Gerardo Munck, María Rosa Olivera-Williams, and Benjamin Radcliff for their
valuable comments. We want to thank the
Quality of Democracy Working Group of the Helen Kellogg Institute for
International Studies at the University of Notre Dame for its support. Replication datasets for this study are
available upon request.
Introduction
After the third wave of
democratization, students of comparative politics have noticed a decreasing
range of variance in their favorite dependent variable: the political
regime. Explaining the conditions for
the emergence, breakdown, or survival of different regimes has been a classic
goal of comparative studies. Over the
last decade, however, political democracy has survived in many
countries—meaning that the dependent variable has shown no significant
change. This situation has directed
scholars towards new and more subtle questions about preconditions for
democratic consolidation and the institutional features of new democracies. Moreover, it is breeding a growing interest
in the quality of democratic life, a factor that clearly varies from country to
country.
This paper explores the
problem of conceptualizing and measuring the quality of democracy in Latin
America. The first part discusses the
use of the concept and the need for an operational measure. We begin our quest by noting an important
distinction among polyarchies: even though all of them allow (quasi) universal
participation and legal opposition to the ruling party, effective participation and competition vary from country to
country. This suggests that countries
with similar levels of democratization might take advantage of their democratic
institutions to different degrees.
Following this observation, we explore three dimensions of the quality
of democracy: civil rights, participation, and effective competition. Following Hill, we conceive these three
dimensions as the extension of Dahl’s concept of polyarchy.[1] In the second part we develop an indicator
of effective competition, one of the key dimensions in our study. The third part analyses the empirical
relationship between all three dimensions in 18 Latin American countries
between 1978 and 1996. We construct
summary measures of the quality of democracy in several ways, and show that the
ranking of the cases is highly consistent no matter the procedure applied. In the last section we test the validity of
our measure and discuss its limitations.
Approaching the Quality of Democracy
By quality of democracy we
refer to the extent to which any given polyarchy actualizes its potential as a
political regime.[2] The assumption underlying this view is that
polyarchy is a necessary, yet not a sufficient, condition for a high quality of
democracy. We suspect that most
students of democracy would agree with this basic definition, if anything else
because it is broad enough to accommodate several perspectives on this issue.
Recent studies of
democratization have increasingly dealt with this question. Some authors have approached the topic as an
extension of the classic focus on regime change. A good deal of research has been done in order to measure levels
of democracy allowing scholars to trace fine distinctions among the cases
placed at the top of the scale.[3] For instance, Diamond and Coppedge conceived
the quality of democracy as the relative degree of democratization among
countries that we already label as polyarchies.[4]
We acknowledge the value of
this perspective but see two potential problems. The first one is that instruments designed to grade regimes in a
wide range between full authoritarianism and full democracy may lack
sensitivity to discriminate within the pool of polyarchies clustered at one
extreme of the range. Second, we
suspect that the relevant criteria to distinguish between authoritarianism and
democracy are not necessarily the same as the relevant ones to discriminate among polyarchies. For instance, marginal improvements in
political rights and civil liberties may be relevant, but not be the only key
to the quality of democratic life.
There is a substantial
difference between addressing the quality of democracy and the level
of democratization of a political regime.
Every analysis of the quality of democracy should assume a minimum
degree of democratization (i.e. Dahl’s procedural minimum). When we compare the quality of democracy
among countries we are not comparing which countries are more democratic (in
the sense that Freedom House scores or the Polity Index measure the level of
democracy as opposed to authoritarianism).
Rather, we are analyzing in which countries democracy performs better
given some normative standards. Much of
the debate about the Quality of Democracy is about the identification of these
normative standards.
For instance a second
approach to the quality of democracy has emphasized, often from a qualitative
perspective, substantive flaws that negatively affect democratic life in a
given country or set of countries.[5] To deal with these cases of “reserved
domains,” lack of “horizontal accountability,” or “electoralism,” among other
problems, scholars have developed a whole array of diminished sub-types of
democracy.[6] This perspective has been extremely lucid in
identifying challenges for (and flaws of) new polyarchies, but it has usually
avoided a comprehensive definition of the quality of democracy and has tended
to ignore problems of cross-national measurement.
In our view, some of the
most interesting studies of the quality of democracy have been done at the
local level.[7] Putnam identified the quality of democracy
with institutional performance—understood as some objective measure of
governmental responsiveness and output levels.[8] Some of Putnam’s indicators, however, are
related to the performance of local government per se, not necessarily local democratic
government (even though his universe of study was democratic). In addition, we suspect that his indicators
are too tailored to the Italian case (at the local level), and would not travel
well to Latin America (at the national level).
We pursue an operational definition of the quality of democracy that is
anchored in Dahl’s definition of polyarchy and that allows us to assess to what
extent different polyarchies transform legal opportunities for participation
and contestation into tangible patterns of citizen behavior. Democracy creates the potential for citizen
participation and opposition to elected officials, but in many countries
citizen apathy or weak party competition, among other possible reasons, hinder
the development of this potential.
Our approach is similar to Hill’s study of the United States. Hill proposed “…an empirical assessment of
the extent of representative democracy in the [American] states. That assessment is guided by empirical
democratic theory and its explication of the essential components of such a
governmental system: equal rights to vote in free and fair elections,
competitiveness among political parties contending to control government
through those elections, and the degree of mass participation in elections.”[9] Because these three dimensions were derived
from the Dahlian concept of polyarchy, we consider this approach particularly
useful for the purpose of cross-national comparison.[10]
Following Dahl and Hill we
therefore emphasize three dimensions of the quality of democracy in Latin
America:[11]
Effective civil rights. Dahl’s
definition of polyarchy hinges on a set of institutional conditions allowing
mass participation and free opposition to the ruling elite. The lack of such conditions (what we call
effective rights) determines the absence of polyarchy. But even if all conditions are present to a
good extent (making the country a member of the polyarchic set of regimes)
limited violations of civil rights may hinder the quality of democracy. Countries in which some specific regions or
social groups are affected by political violence or electoral manipulation are
clearly worse than democracies in which the whole population effectively
exercises its rights—of course, no country has a perfect record. We rely on
Freedom House scores as an indicator of this dimension —we normalized the 2-14
scale to range between 0 and 1 in order to facilitate comparison with the other
dimensions.[12] Since all cases in our sample are
polyarchies, the average score is high: .73, with a standard deviation of .15.
Effective Participation. The Dahlian
measure of participation reflected the right to participate, not the
actual rate of participation. Most
scholars contend (correctly, we think) that voter turnout should not be part of
a definition of democracy. But many
others have argued (also correctly, in our view), that turnout is an important
dimension of the quality of
democratic life.[13] Greater participation—whether it is
voluntary or encouraged by compulsory vote—makes democratic governments responsive
to a larger share of the population.[14] The health of a democratic regime is
particularly weak when some citizens are effectively disenfranchised as a
consequence of poverty, lack of basic education, or sheer apathy.[15] Because low turnout in Latin America is
typically related to low levels of voter registration we measure turnout as the
number of voters over the voting-age population (VAT).[16] Data were gathered from a single source.[17]
In Latin America, turnout varies from extremely low (15% in 1994 Guatemala) to
very high rates (95% in 1989 Uruguay).
The average for our sample was a VAT of .62, with a standard deviation
of .18.[18]
Effective Competition. Dahl’s
definition of polyarchy allowed for the free exercise of political contestation
in—and between—elections, but it never implied that effective competition had to occur. For instance, Japan between 1955 and 1993 is a classic example of
democracy with low inter-party competition.
We agree with Hill, however, that a more competitive democracy is a better democracy.[19] An indicator of competition useful for our
purposes must fulfill three requirements: 1) reflect the opposition’s access to
the legislative process, rather than mere electoral outcomes (which can be
distorted through disproportionality or fraud); 2) punish the excessive
dominance of the ruling party in policy making, but 3) without rewarding
excessive dominance of the opposition (which may create serious problems of
governability). For different reasons
that we explain in the following section, traditional measures of party
competition do not serve our purpose.
We therefore develop an original index for cross-national comparison.
Assessing Effective Competition
Students of democracy and
elections have developed different measures of competition. For example, Powell measured competition as
the frequency of alternation in power over a nineteen-year period, and Ranney
built a multidimensional index of competition in the American states over several
decades.[20] This long-term perspective is not very
useful for new democracies in which just a few elections may have taken
place. Other students have measured
competition as the winner's percentage of the votes, the percent margin of
victory, and the raw vote margin of victory in elections.[21] Such measures are closer to our purposes,
but they are heavily biased against two-party systems because margins of
victory tend to be smaller in multiparty democracies.
We measure the opposition’s
access to power as a weighted difference between the share of the seats of the
government and the opposition parties in the lower chamber.[22] Being aware that it is important to penalize
fragmentation, we designed a measure to find the “typical party” in the
opposition by weighting the shares of seats in favor of the largest parties: 
O is an indicator of the
leverage of the opposition, where oi is the share of seats for the i-th opposition party. We followed the same procedure to estimate
the size of the “typical party in government” in the case of electoral
coalitions winning office:
Based on our previous
assessment of the size of the “typical parties” in government and in
opposition, we developed and index of competitiveness:
The value of C tends to zero
whenever the government (or the opposition) controls the whole legislature, and
to one if there is balance between government and opposition.[23] For example, C equals .332 for 1984
Nicaragua, and .998 for 1990 Bolivia.
The performance of our indicator is well illustrated by the cases of
Ecuador (1988) and Venezuela (1979). In
both cases, the ruling party (Izquierda Democrática and Copei, respectively)
controlled 42.2 percent of the seats.
In Ecuador, however, the opposition was highly fragmented, therefore
C=.656. In contrast, Copei had to face
the powerful Acción Democrática in congress: C=.745.[24] The average value of C in our sample is .80
with a standard deviation of .14.[25]
Measuring the Quality of Democracy
In previous sections we have argued that effective civil rights, participation,
and competition are three dimensions of the quality of democracy inferred from
the Dahlian concept of polyarchy. Are
these three dimensions independent, or do they reflect a latent variable? If these variables are independent from each
other, we can only conceive the quality of democracy as a multidimensional
phenomenon. If, on the contrary, all
dimensions reflect an underlying construct, we could develop a summary measure
of the quality of democracy in Latin America.
In this section we address
this problem using factor analysis. We
include in our sample all Latin American countries that were polyarchies at
some point between 1978 and 1996. The
decision to include some cases in the sample is debatable because the
boundaries between democratic and non-democratic regimes are sometimes
contested—and, of course, it would not make sense to measure the quality of
democracy in non-democracies. One way
of dealing with this problem is to consider polyarchy a concept with uncertain
boundaries, not because the concept is ill-defined but because there is
uncertainty about some cases belonging to the polyarchic set.[26] If we adopt a “fuzzy set” approach, we can
think of countries as being part of this set with different degrees of
probability.[27] For instance, the probability of Cuba
belonging to the set of democracies is virtually zero, the probability of Costa
Rica, virtually one. But we suspect
that the probability of Mexico being a member of the set in the early 1990s was
lower than one. Acknowledging the
problem of uncertainty, we wanted to include in our sample all cases with a
high likelihood of being members of the polyarchic set.
We based our decisions in
two categorical measures of democracy, the ACLP classification, and the
Mainwaring classification.[28] We included in our sample all cases coded as
democracies or semi-democracies in the Mainwaring classification (that is,
cases coded as democratic by Alvarez et
al. but not coded as
authoritarian by Mainwaring).[29] Eighteen countries entered the sample at
different time points (see Table 1 below).
Cuba and Haiti were not classified as democracies at any point during
the 1978-96 period.
Our units of analysis are
democracies-after-each-election, a total of 77 observations were included in
the analysis (see Annex). For the few
cases in which presidential and legislative elections were held in different
years, we measured turnout at the presidential or legislative election and
calculated the competitiveness of the system based on the configuration of the
new government or legislature.
To the extent that our
three dimensions (civil rights, competitiveness, and turnout) reflect an
underlying construct like the “quality of democracy,” we expect them to be
partially correlated across cases—they all map the same latent variable. Indeed, the Pearson correlation between
Freedom House scores and turnout is .58, between turnout and the C index is
.29, and between Freedom House and C is .49 (all significant at .01 level). These are not strong correlations and therefore,
we cannot be sure whether they reflect a single latent variable without factor
analysis.
Following Hill we used
factor analysis to create a summary measure of the quality of democracy.[30] Factor analysis is useful to validate an
index by demonstrating that its constituent items load on the same factor. According to Marradi: “Factor analysis
allows one to use statistical relationships between several lower-level
variables as empirical evidence for or against the establishment of a semantic
relationship of indication between these variables and an abstract concept,
which may thus be measured and transformed into a variable with a high semantic
extension and theoretical importance.”[31] Factor scores for each country are presented
in Table 1 below.
Table 1 here
We also aggregated the three
dimensions by estimating the unweighted average of their standardized
scores. The values, presented in the
column labeled “Z-scores” are highly correlated with the values for the common
factor. The problem with those two
measures is that they are dependent on the cases included in the sample. If, for instance, we add or delete some
observations from the sample, the standardized scores may change and the
relative order of cases may be affected.
We therefore tested two forms of aggregation that are free of this
problem: the average value ([FH+C+VAT]/3) and the product (FH*C*VAT) of all
three dimensions. Our results are
extremely consistent regardless the aggregation procedure, the only exception
being the ranking of Chile and Costa Rica (we discuss Chile’s ranking in more
detail the conclusions below). Those
two countries change positions if countries are sorted by Average, Product or
Z-Scores. The rest of the countries
remain in the same position when sorted by Factor or by any other measure.
For graphic parsimony we
have included a map of Latin America in which each country is colored according
to its factor coefficient (Figure 1).
On average, countries in the Southern Cone have shown a better quality of
democracy than countries in Central America or Northern South America. But Costa Rica is of course the most
noticeable exception to this pattern.
Not only has Costa Rica being democratic for more than half century, it
is also a good democracy. We suspect that “white” countries (Colombia,
Guatemala, Mexico, Paraguay) are, or have been for most of the period under
study, borderline cases. In other
terms, they might be located under the “cross-over point” where membership in
the set of democracies is more than uncertain.[32]
Figure 1 here
Concept Validity
Besides the issue of aggregation, another important problem in concept building is validation. Simply put, validity implies that our measure reflects (our definition of) the quality of democracy, not just a small part of the concept or some other theoretical construct.[33] In this section we focus on criterion-related validity (the extent to which our measure correlates to other indicators) and construct validity (the ways in which our measure relates to broader theoretical assumptions about the quality of democracy). We are aware that this is modest, first step in the validation process. As Hubley and Zumbo noted, “the validation process is a form of disciplined inquiry in which plausible, alternative inferences from the test scores or observations are disproved.”[34]
Criterion-related validation
is difficult in this case because, to our knowledge, no other measures of the
quality of democratic life are available for Latin America. There are, however, available indicators of democracy like the Polity III, 20-point
scale.[35] We expect a positive correlation between our
indicator of democratic quality and the (upper range) of the Polity score. In fact, the average Polity III scores for
countries in our sample correlate at .70 with our measure (n=18). This correlation is not very surprising,
since Polity is itself correlated to Freedom House scores, our indicator of
civil rights. Because our sample
includes some cases (i.e., countries in given years) that Mainwaring classified
as “semi-democratic” we also looked at the mean factor score for this
group. Cases labeled as
semi-democracies (n=28) have an average factor score of –.92, while the ones
coded as democratic (n=49) have a mean of +.53.
For construct validation, we relied on what Adcock and Collier called the AHEM (Assume the Hypothesis, Evaluate the Measure) validity test.[36] This is also a difficult task because the literature on the quality of democracy is at an early stage, and there are no well-established hypotheses about the causes (and consequences) of a “good” democracy. We selected three ideas that we considered (almost) uncontroversial: 1) a stronger democratic tradition is correlated with a better democracy; 2) political violence has a negative impact on democratic life, and 3) public satisfaction with democracy is related to the quality of the regime (probably in a bi-directional way).
As an indicator of
“democratic tradition” we counted the number of years of democracy enjoyed by
each country in the second half of the century (i.e., between 1950 and 1996,
the last year in our sample).[37] The correlation between this indicator and
our factor score is .79 (n=18). We also
measured democratic tradition in the long run using the Polity III country
average between 1900 and 1977 (the last year prior to our period). The correlation between this indicator and
our index was .62. Countries with
strong guerrilla movements in the 1978-1996 period (Colombia, El Salvador,
Guatemala, Nicaragua, and Perú) have a mean score value of –.94, while the
average for the rest of the countries is +.33.
The inclusion of Mexico in the first group would only strengthen our
test.
Satisfaction with democracy
at the mass level was measured by the Latinbarometer (1997 wave) in all
countries except for the Dominican Republic.
We measured satisfaction as the difference between the percentage of
respondents saying that they are “very” or “somewhat” satisfied with the
regime, and the ones declaring some degree of dissatisfaction. The correlation between this indicator and
our score is .65 (n=17). Table 2 below
presents the relation between the three factors (tradition, violence, and
satisfaction) and the quality of democracy index. The regression model predicts 80 percent of the variance in the
dependent variable, and all coefficients are significant and of the expected
sign.
Table 2 here
Conclusions
The major question underlying any assessment of the quality of democracy is how to discriminate between better and worse democratic units. The number of normative criteria we can use to evaluate democracies is, however, virtually unlimited. In this case we selected three dimensions that are well-grounded in democratic theory: effective liberties, competitiveness, and participation.
These criteria have proved
to be instrumental for comparing democracies at different levels of
analysis. For instance, Hill compared
the US states at two historical moments, Centellas assessed the evolution of
democracy over time in one country (Bolivia), and Altman and Pérez-Liñán dealt
with a cross-section of countries worldwide.[38] In this paper we have compared 18 Latin
American countries during the “third wave,” between the late 1970s and the mid
1990s.
We are aware that those
three aspects are not enough to fully describe the complexities of democratic
life. For instance, our indicator is
not sensitive to the presence of “reserved domains”—as the ones Valenzuela bare
in mind when describing Chilean politics.[39] This explains why Chile ranks so high in our
study. Neither is our measure very
sensitive to the presence of “brown areas”; the degree of independence of the
Judiciary ; accountability, or institutional performance.[40]
Those limitations show that
we are far from settling the question of how to measure the quality of
democracy. Although our index performs
satisfactorily well, fitting our theoretical expectations, we consider this
measure a very modest contribution to the incipient debate on this issue. Future research shall address three topics:
the validity of this measure, its reliability in different contexts, and the
addition of new dimensions to this idea of democratic quality.