Fragmented policymaking in Mexico: the design of social programs in a subnational case

Formulación fragmentada de políticas en México: el diseño de programas sociales en un caso subnacional

Carlos Moreno-Jaimes

Instituto Tecnológico y de Estudios Superiores de Occidente (ITESO) (México)

ORCID: https://orcid.org/0000-0002-4928-218X

cmoreno@iteso.mx

NOTA BIOGRÁFICA

Carlos Moreno Jaimes es doctor en políticas públicas por la Universidad de Texas en Austin. Especialista en análisis y evaluación de políticas públicas, descentralización del gasto público y protección social en salud. Ha sido consultor para diversas instituciones nacionales e internacionales. Autor de los libros Democracia electoral y calidad gubernativa: el desempeño de los gobiernos municipales en México (2007) y Seguro Popular y Federalismo: un análisis de política pública (en co-autoría con Laura Flamand, publicado por el CIDE en 2014), entre otros capítulos y artículos en revistas con arbitraje internacional. Durante seis años fue Jefe del Departamento de Estudios Sociopolíticos y Jurídicos del ITESO, donde actualmente es profesor-investigador.

ABSTRACT

Subnational governments in Mexico have significantly increased their role as policymakers. As a result, they have contributed to the creation of a wide variety of social programs earmarked to different target populations. Although the effects of these interventions on poverty reduction or on other development indicators are uncertain, analyzing their design can provide valuable insights about how social policy is conceived. In this article, I use the case of Jalisco, one of the states in Mexico that has taken the lead in the development of evaluation and monitoring mechanism to manage policymaking, to analyze the internal consistency design of social programs (the logic between problem definition, pertinence of goals and instrument selection) and their degree of horizontal articulation (to what extent programs duplicate or complement with each other). Drawing on an original dataset with more than 100 variables for 339 social programs, I find that policy interventions have a poor level of internal consistency, particularly regarding the formulation of policy problems. In addition, programs are highly atomized, which means that too many interventions aim at particularized interests, not at broader social groups. I argue that, notwithstanding that they operate in context where the formulation and evaluation of public policy are highly institutionalized, social programs are considerably inconsistent and fragmented, suggesting that they are not a deliberative response to social problems demanding solutions.

KEYWORDS

Policy design; policy coherence; policy articulation; subnational policymaking; Mexico.

RESUMEN

Los gobiernos subnacionales en México han aumentado significativamente su papel como formuladores de políticas. Como resultado, han creado una amplia variedad de programas sociales. Aunque los efectos de estas intervenciones en la reducción de la pobreza o en otros indicadores son inciertos, el análisis de su diseño puede proporcionar información valiosa sobre cómo se concibe la política social. Utilizo el caso de Jalisco, uno de los estados del país que más han desarrollado sistemas de evaluación y monitoreo para la gestión de políticas públicas, para analizar el diseño de consistencia interna de sus programas sociales (la lógica entre definición del problema, la pertinencia de fines y la selección de instrumentos) y su grado de articulación horizontal (en qué medida los programas se duplican o complementan entre sí). A partir de una base de datos original con más de 100 variables de 339 programas, encuentro que éstos tienen un nivel deficiente de consistencia interna, particularmente en lo relativo a la formulación de los problemas. Además, los programas están muy atomizados, pues hay demasiadas intervenciones que atienden intereses particulares, no a grupos sociales más amplios. Sostengo que, pese a operar en un contexto donde la formulación y la evaluación se encuentran altamente institucionalizadas, los programas sociales son considerablemente inconsistentes y fragmentados, lo que sugiere que no son una respuesta deliberada a problemas que exigen soluciones.

PALABRAS CLAVE

Diseño de políticas; coherencia de políticas; articulación de políticas; formulación de políticas a nivel subnacional; México.

SUMMARY

1. INTRODUCTION. 2. WHY POLICY DESIGN MATTERS? 2.1. CAN PUBLIC POLICY BE LOGICALLY CONSISTENT? 2.2. THE IMPORTANCE OF POLICY ARTICULATION. 3. METHODS AND DATA. 4. SOCIAL POLICY DESIGN: A SUBNATIONAL CASE. 4.1. ILLOGICAL CAUSAL DESIGN. 4.2. ATOMIZATION. 5. DISCUSSION. 6. CONCLUSIONS. REFERENCES.

1. INTRODUCTION1

Notwithstanding Mexico’s economic relevance, and being the 11th largest economy by purchasing power parity, poverty and social inequality remain among the most enduring and challenging problems in the country (Esquivel, 2012). To deal with these issues, since the early 1990s Mexico’s central government created ambitious anti-poverty programs to mitigate the consequences that economic reforms –based on promoting free-trade, market deregulation, and the privatization of different public services– would produce on vulnerable populations. Some of those interventions consisted in creating transfer funds earmarked to the provision of basic public goods, giving priority to the most deprived regions of the country (Cornelius et al., 1994), while others entailed the creation of conditional cash transfers targeted to poor households with the aim of breaking the intergenerational transmission of poverty by requiring recipients to invest in children’s nutrition, health and education (De La O, 2015). Although the effectiveness of those programs on poverty reduction is still controversial, it has been argued that they represented a significant improvement in Mexico’s social policy, as programs became better targeted to the poor and they also reduced government discretion in the management of funds (Diaz-Cayeros, Estevez and Magaloni, 2016).

The design and funding of the most important poverty alleviation programs have been mostly centralized. However, as Mexico’s electoral landscape became more plural and competitive throughout the last two decades of the past century, subnational governments (state and local authorities) have significantly increased their political importance and their role as policymakers (Beer, 2003). To address the needs of populations that federal strategies do not always reach, local and state governments have launched programs providing benefits such as urban transportation for students, transfers for people with disabilities, scholarships for indigenous communities, and many others. According to CONEVAL (Mexico’s agency in charge of measuring poverty levels), in 2014 the total number of social programs funded and operated by Mexico’s 32 state governments was around 2,500. However, the consequences of these programs on poverty reduction and social development are still uncertain, although some contend that decentralizing the poverty relief function towards local government has not been effective (Hernandez-Trillo, 2016).

In this article, I use the case of Jalisco to analyze the internal logic in the design of social programs and their articulation with others, under the premise that a program design based on a weak conceptual basis can explain policy failure by itself, regardless of other problems that could emerge during its implementation. Jalisco is one of the states in Mexico that for years has taken the lead in developing an internationally recognized institutional framework to manage policy planning, monitoring, and evaluation. As a result of its relatively advanced level of institutionalization in program formulation, it would be reasonable to observe that social policymaking in Jalisco operates close to the paradigm in which problems are clearly defined, policy solutions are compatible with such definitions and there are not manifest contradictions between different interventions aiming at similar target populations. Drawing on an original dataset with more than 100 variables for 339 social programs, I perform two analyses: a) an “internal consistency” assessment of policy design which evaluates whether programs establish a logical structure of problems, goals, policy instruments and target populations, b) an “horizontal coherence” appraisal of different social programs, in order to identify potential synergies and duplications between them. Contrary to my initial expectations, I found that social programs in Jalisco exhibit a very low level of internal consistency, particularly regarding the formulation of policy problems, and that there is a high level of policy atomization, namely that a very large number of programs are aimed at very specific target populations, not to broader social groups, which prevents programs to interact with each other in providing more integral solutions to the complex problems people face.

These findings are in line with a stream of literature that emphasizes that policy solutions do not emerge from a well-ordered process guided by logical reasoning, but the product of political conflict, negotiation and compromise. In other terms, my results suggest that political factors play a fundamental role in the making of social policies, regardless of deliberate efforts to give order and rationality to their formulation process, even if this translates into incoherence and fragmentation.

2. WHY POLICY DESIGN MATTERS?

Generally speaking, policy design can be described as a process of creating a policy response to a policy problem, or, “the deliberate and conscious attempt to define policy goals and to connect them to instruments or tools expected to realize those objectives” (Howlett et al, 2015, p. 292). Given its instrumentalist quality, policy design has been widely studied both from a rationalistic perspective that emphasizes goals such as efficiency and effectiveness and also from a more political view that underscores other social values such as equity, equality and political feasibility (Schneider and Ingram, 1993). In any case, it is widely accepted that the attainment of those objectives cannot be left to chance, but requires a deliberative effort to design internally consistent interventions and articulate them with other existing policies.

2.1. Can public policy be logically consistent?

A standard assumption in the literature on public policy analysis is that government interventions are deliberated choices through which public authorities attempt to solve situations that society deems as problematic (Bardach and Patashnik, 2015). A crucial step in policymaking is the definition of the public problem that calls for government intervention, mainly because the consideration of alternative policy solutions strongly depends on the behavioral assumptions and normative goals used by policy designers in their formulation of policy problems (Dery, 2000; Weiss, 1989; Wildavsky, 2017). For some scholars, policy formulation is, primarily, an analytical endeavor that entails the use of scientific knowledge and data analysis to acquire a sound comprehension of a policy problem, identify its fundamental causes and develop the appropriate policy tools capable to transform those causes (Majone, Giandomenico and Quade, 1980; Stokey, E. and Zeckhauser, 1978). Therefore, policy formulation is often conceived as a highly technical task where experts play a crucial role in the definition of policy alternatives. From that view, the lack of internal consistency of policies would derive from a poor understanding of a problem, from the absence of suitable intervention tools or from factors that policy designers fail to anticipate, including the unexpected response of target populations to some policy inducements.

However, such a rationalistic view of policy design has been subject to different criticisms. For example, Lindblom emphasized that rational analysis has clear limits in pluralistic political systems: having highly-trained policy experts and sophisticated information systems cannot by themselves determine what public values should be prioritized or what social problems should deserve the greatest attention and resources for their solution (Lindblom, 1968). These questions, according to Lindblom, can only be processed through the logic of political interaction, that is, through inter-party competition, debate, negotiation, consensus or dissent. In other terms, the formulation of policy problems is not purely analytical endeavor, but a highly political one, as different players strive to impose their own problem definition in the policy agenda (Kingdon, 1995; Stone, 2002). Considering that policy design is never detached from the general political process and that experts are only part of the many players that participate in the making of public policy, consistency in policy design can never be taken for granted. In fact, John Kingdon’s “multiple stream” approach anticipates a mismatch between problems and solutions; his model assumes that, in pluralistic political systems, problems, solutions, and politics are streams that develop independently from each other, until a window of opportunity opens to allow participants (interest groups, experts, politicians, the media, among others) to push their policy preferences into the government agenda. Since this process does not take place in a rational or well-ordered way, we cannot expect that policy solutions will necessarily be a logical outcome, but the result of negotiations and mutual adjustment (Cairney and Jones, 2016; Kingdon, 1995). On a similar vein, Baumgartner and Jones have pointed out that the fragmentation characterizing policy systems in American politics produces a disjointed policymaking (Baumgartner and Jones, 2013). In general, we can argue that, in democratic regimes, political imperatives contribute to increase the discrepancy between problems and solutions, mainly because incumbent politicians are compelled to fulfill their campaign promises even if they were based on ideological positions or even on social prejudices, rather than on a systematic understanding of a problem. On the other, in a context of high political competition, policymakers have strong incentives to react responsively to social needs, but this often involves addressing the problem’s symptoms, not its causes. As a result, the outcomes of such decisions might provisionally alleviate social discomforts, but the structural causes of problems would remain intact.

In sum, although the internal consistency of public policies is a valuable goal in the policy design literature, its attainment confronts different obstacles, most of which have to do with the functioning of policymaking systems and the incentives facing political and bureaucratic actors in democratic setting. The interplay between politics and policymaking often produces interventions that fail to resolve important social or economic problems simply because these interventions were not intended to actually solve those problems in the first place. Furthermore, an additional difficulty is that, even if governments somehow manage to achieve the internal consistency of their policies, this might not be a sufficient condition to guarantee their success. Excluding failures of implementation (Pressman and Wildavsky, 1974), a policy might not be effective if the problem is produced by many other causal factors demanding other interventions beyond what the policy has to offer. But the existence of more than one policy intervention to confront the same policy problem requires “horizontal policy coherence” (Carbone, 2008), namely the capacity of policies to contribute to a similar policy goal through the use of different instruments and, possibly, focusing on different target populations.

2.2. The importance of policy articulation

In a fairly provocative article, Steven Teles declared that “America has chosen to govern itself through more indirect and incoherent policy mechanisms than can be found in any comparable country” (Teles, 2013, p. 98). Although it is debatable whether the US takes the lead on the matter, it highlights the fact that governments have become extremely complex and incoherent creatures. Such complexity, besides imposing significant administrative burdens to citizens when they interact with public organizations, it also hides the regressive redistribution of polices, while bolstering the gains of organized interests. This in part explains that policy articulation has become an important subject for public administration literature, but also a priority for governments that in the past moved toward policy and managerial specialization, disaggregation and devolution. These administrative reforms, deeply rooted in the New Public Management movement, engendered a situation of government fragmentation that increased the transaction costs borne by citizens to obtain public services (Bouckaert, Peters and Verhoest, 2010; Peters, 1998).

Another reason for the increasing importance of policy articulation is that traditional policymaking based on specific policy sectors (health, agriculture, environmental management) is no longer functional to deal with issues that transcend sectorial boundaries (Bornemann, 2016). In its place, the attainment of multiple goals requires the design of “policy portfolios” that combine different tools in a coherent way:

The articulation of principles of what constitutes a “good” design has evolved from thinking about relatively simple ‘one goal—one instrument’ situations to address issues related to the use of more complex policy mixes or bundles of tools that aim to unite multiple interconnected goals and the means to achieve them across multiple levels of government (Howlett, Mukherjee and Howlett, 2017, p. 4)

In the case of social policy, articulation has also gained relevance for two reasons. On the one hand, it has been widely acknowledged that poverty and inequality are multidimensional problems that require being addressed in an integrated way, but mainly through policies explicitly adopting a human rights perspective. The human rights approach to social policy posits that poverty and inequality are not unavoidable situations, as they derive from actions and omissions by government actors and other social and economic agents. Poverty is not merely the lack of access to material goods and services, but a situation where individuals are excluded from realizing their basic rights as a result of a multiplicity of social, economic and political processes. Therefore, the main role of social policy is to remove the obstacles people face to exercise their freedom and human dignity, which implies the adoption of a universal access to basic services such as health-care, education, social protection, and housing (De Schutter, 2021). The universal access principle diverges from the conventional welfare regimes established in Latin America during the second half of the past century, where access to social benefits was closely linked to formal employment and where social policy decisions where strongly discretionary and clientelist. Second, since the delivery of benefits and services of social programs operates in a decentralized way, this entails the risk of dispersion and administrative fragmentation, which deteriorates the quality of their social impacts (Cunill-Grau, 2014).

An important point to emphasize is that policy articulation is a deliberate endeavor that requires an appropriate mix of policy components (Howlett et al., 2017), but also a set of conditions regarding issues, interests and targeting. That is, coherence within policy domains requires attention concentrated on a small set of issues, the prevalence of few interests around those issues and a very specific targeting in the assignment of benefits and burdens (May, Sapotichne and Workman, 2006). That explains the relevance of the concept of “policy regimes”, understood as governing arrangements to address policy problems. From this view, policies “specify a set of intention or goals, specify a mix of instruments for accomplishing the intentions, and structure implementation” (May and Jochim, 2013, p. 5). May and colleagues argue that the strength of policy regimes is a condition for not only for the legitimacy and durability of public policies, but also for their coherence because it “binds interests and institutions to shared goals and actions” (May and Jochim, 2013, p. 9).

What should we understand as policy articulation? As Cejudo and Michel argue, (Cejudo and Michel, 2016, 2017), there is a need to clarify many of the concepts that have emerged as a reaction to the problem of government fragmentation –the dispersion of roles, administrative requirements, information and authority among numerous government units, which increases significantly the transaction costs for citizens to obtain public services. Solutions to government fragmentation include concepts such as coordination, integration, joined-up government and policy coherence. Consider, for example, the joined-up government definition by Pollitt:

‘Joined-up government’ is a phrase which denotes the aspiration to achieve horizontally and vertically co-ordinated thinking and action. Through this co-ordination it is hoped that a number of benefits can be achieved. First, situations in which different policies undermine each other can be eliminated. Second, better use can be made of scarce resources. Third, synergies may be created through the bringing together of different key stakeholders in a particular policy field or network. Fourth, it becomes possible to offer citizens seamless rather than fragmented access to a set of related services (Pollitt, 2003, p. 35).

The previous definition implies that any arrangement that allows policies either to duplicate or contradict with each other represents a fragmented or disjointed system, while an articulated one implies that policies interact with each in a constructive way. Nevertheless, we should go further to clarify what constitutes duplication, contradiction and synergy in policy design. In an attempt to elucidate what constitutes policy coherence, Cejudo and Michel distinguish three dimensions: coherence between policy goals, coherence between policy tools, and coherence between target populations. There will be coherence in policy goals if two or more policies contribute to a broader policy goal; there will not be coherence if the specific goals of those policies are contradictory. The argument is similar regarding policy tools: if two or more tools contribute to the accomplishment of a broad policy goal, there will be coherence. Finally, a policy domain (May et al., 2006) might have a comprehensive goal whose realization depends on different policies contributing separately to cover specific target populations. There will be duplication if the target population of two or more policies aiming at the same goal overlap (Cejudo and Michel, 2016). An additional and more ambitious form or policy articulation is what these same authors denote as “policy integration”, which refers, as pointed out by Trein, Meyer and Maggetti (2018), not to specific features of policy design, but to the decision-making process to face multifaceted policy problems:

[policy integration is] “the process of making strategic and administrative decisions aimed at solving a complex problem. Solving this complex problem is a goal that encompasses –but exceeds– the programs’ and agencies’ individual goals (Cejudo and Michel, 2017, sec. 3).

In summary, the public policy literature has emphasized for decades that coherence and articulation are desirable attributes of policy design. However, it acknowledges that attaining those attributes does not depend exclusively on technical expertise or on establishing comprehensive administrative frameworks. Instead, the challenge is mainly political: the logical consistency of a particular policy is hard to reach when political players struggle to impose their own definition of the public problem or when each of them champion divergent solution alternatives. In addition, articulating different policies is a big challenge in the absence of stable policy regimes that allow a tighter bond between institutions, goals and actions.

3. METHODS AND DATA

In this article I analyze the design of social programs in Jalisco, one of the most populated states in Mexico that has been nationally recognized by its accomplishments in the field of public policy monitoring and evaluation. My analysis focuses on their internal consistency and their horizontal coherence or articulation. I only focus on some aspects of policy design, but not on its actual implementation, thereby I leave out any reference to coordination issues. I closely follow Cejudo and Michel’s conceptualization of policy coherence regarding tools and target populations. However, I disregard coherence in policy goals because social programs and actions commonly express them in a very ambiguous way, which complicates a comparative analysis.

The empirical basis of the analysis is a dataset created by the author based on information from Jalisco’s government covering more than 100 variables regarding the design of government programs and actions in the state, such as their goals, benefits, target populations, budget, and many others2. The raw data (which consists of text descriptions of each variable) was codified in systematic categories for analytical purposes. In this article I focus only in the 2017 dataset, which included a total of 153 entries. Each entry represents a government program, which in turn can be divided into “modalities”, which means, for example, that a program for people with disabilities might, on the one hand, provide them with direct cash support and, on the other, subsidize clinics that offer rehabilitation services for disabled persons. Although the dataset considers each benefit as a modality of a same intervention, for the purposes of this paper they will be regarded as separate public interventions, since each apply different policy instruments to different target populations. Therefore, the universe of public interventions in the data equals 282. However, after reviewing the programs’ documentation, I identified 57 additional interventions that were not reported in the original database, thereby increasing the total number to 339.

The next crucial methodological decision was to define what should be understood as a “social policy intervention”. For that, I used the following criteria: first, the intervention should be clearly aligned with any of the 12 social rights established in Jalisco’s Social Development Act3. To confirm whether the intervention was actually associated with any of those rights, I carefully reviewed the intervention’s stated objectives and description. Second, if an intervention was not aligned to any social right, it was considered as a social policy intervention if its declared goal was to improve the economic wellbeing of any of the following vulnerable populations: women, children, people with one severe disability or more, the elderly, indigenous persons, and agricultural day laborers. After applying such selection criteria, I obtained a total of 256 social policy interventions.

The internal consistency analysis was based on three aspects of policy design: 1) the definition of the policy problem, 2) the pertinence of policy goals, and 3) the degree of coherence of the policy tools employed. For each aspect, I developed a set of standards to assess its quality. For the horizontal coherence dimension, I analyzed whether some of the 256 interventions duplicate or complement with each other, or whether they are completely independent. I provide more details of the procedures followed for the internal consistency and horizontal coherence analyses in the next section.

Although the characterization of all variables in the dataset is based on a careful review of the normative documentation of each program, I conducted four semi-structured interviews with mid-level officials and program managers to better understand some aspects related to their implementation, namely the involvement of other government agencies in the delivery of services and how those agencies exchange information regarding beneficiaries4. Although my analysis concentrates on policy design, some aspects regarding program implementation are discussed below.

4. SOCIAL POLICY DESIGN: A SUBNATIONAL CASE

Jalisco is the third most populated state in the country, with 8.3 million residents in 2020. Although it is not among the poorest regions in Mexico –it occupies the 23th position in level of poverty out of Mexico’s 32 states–, citizens live under considerable disparities in basic wellbeing indicators across social groups and localities. According to CONEVAL, 28.4 percent of Jalisco’s population was poor, namely about 2.3 million people in 2018 (CONEVAL, 2020). As shown in Table 1, Jalisco poverty rate is well below the national average (41.9 percent) and, since 2012, it has a declining trend. Extreme poverty in Jalisco represents only 3 percent of its population, while the national average is more than two times that rate.

Table 1. Poverty and extreme poverty in Jalisco and Mexico, 2010-2018

	Poverty (%)		Extreme poverty (%)
	Jalisco	Mexico	Jalisco	Mexico
2010	37.0	46.1	5.3	11.3
2012	39.8	45.5	5.8	9.8
2014	35.4	46.2	3.2	9.5
2016	31.8	43.6	1.8	7.6
2018	28.4	41.9	3.0	7.4

Source: (CONEVAL, 2020).

Despite Jalisco’s comparatively better wellbeing standards, it has important social inequalities. For example, in five of its 126 municipalities, the rate of poverty is over 85 percent, almost three times the state’s average, although this affects only a minority of Jalisco’s residents. This clearly indicates that rural poverty is still a significant challenge for social policy, given the complexity of improving access to basic services in localities characterized by high population dispersion. Social disparities are also evident by considering the living standards of social groups that have been marginalized from economic development or whose needs have been historically overlooked, such as indigenous communities (Gracia and Horbath, 2018) and people with severe disabilities (CEDHJ, 2017).

In order to cope with the problems of poverty and social inequality, Jalisco has developed a myriad of social programs and actions, but also an advanced institutional framework to regulate the planning, monitoring and evaluation of those interventions. In 2004, one year after the Mexican congress enacted for the first time a piece of legislation to organize Mexico’s social development policy5, the measurement of poverty and the evaluation of social programs, the congress of Jalisco proclaimed its own social development act, defining 12 basic social rights that would be the focus of social policies. The act also created a new social development ministry that would be responsible to operate any program or activity related to Jalisco’s social development, but also to coordinate any other public organization dealing with such policy field, for example the ministries of education, health, agriculture, and many others, including the social development agencies of municipal governments. In its article 4, the same act provides a definition of a social development program:

[it is] a direct public intervention, carried out in a systematic and articulated way, aiming to fulfill people’s social rights and the economic wellbeing of vulnerable social groups, through the distribution of resources, the provision of services, the allocation of subsidies, and the construction and operation of social infrastructure.

In addition, since 2013 Jalisco established a system to monitor and evaluate its own social programs, which requires them to comply with a set of quality standards regarding the register of program’s beneficiaries, their operational rules, managerial and results indicators, the evaluation of their processes, activities and results, among others. According to a national assessment by CONEVAL, Jalisco’s monitoring and evaluation system has ranked first between 2015 and 2019 compared to the other 32 states in the country6. The system has been awarded international awards at different moments and has been used as a reference for other countries trying to implement their own policy evaluation and monitoring systems7.

Although Jalisco had already some sort of social policy interventions since the early 1970, their real expansion took place at the outset of 2000, the same year that an opposition party won the Mexican presidency after a long period of single-party hegemony, and five years following the first experience of party alternation in a gubernatorial race in Jalisco. In 2017, official sources report a total of 172 social interventions funded either with state government resources only or with a combination of federal and state budgets, 47 of which were created during the 2000-2009 period and 91 afterwards (Figure 1)8. In the last six years, the budget allocated to those social interventions has been about 50 million USD per year, but the bulk comes mainly from federal resources (between 50 and 60 percent). A typical social program or action has an annual budget of 4 million USD, but there are huge variations in budget size across interventions, ranging from a minimum of six thousand USD per year to almost 15 million USD. The number of people receiving any benefit or service from a social policy intervention, according to the official beneficiary registry, was of almost 600 thousand individuals in 2017, less than eight percent of Jalisco’s population and less than one third of the total number of persons living in poverty in the state.

Figure 1. State-funded social programs in Jalisco by period of creation

 

Source: own elaboration based on SIMAPP.

At the present, it is uncertain what have been the outcomes of so many different interventions in terms of relevant policy goals such as the reduction of poverty or social inequalities. As discussed in the theoretical section, the capacity of public policies to achieve their goals depends on many factors beyond the control of public decision-makers and program managers. However, two conditions are essential: 1) the internal coherence of a particular intervention (i.e. whether there is a logical connection between the nature of a policy problem, the goals envisaged to correct that problem, and the specific instruments designed to accomplish those goals), and 2) the coherence between different policy interventions (i.e. whether different specific policies contribute to overall desirable objectives or if their individual goals and instruments conflict with each other). Considering the notable progress that the state of Jalisco has made in creating a widely recognized system of policy monitoring and evaluation, it would be very likely to observe a reasonable degree of coherence in the design of social programs. The following section analyzes if that premise has empirical support.

4.1. Illogical causal design

“In Support for the Elderly” is a social program in Jalisco that offers a small monthly allowance to people aged 65 or above who do not receive any retirement income. According to its operational rules, the underlying problem that justified the program is the fact that a significant proportion of the elderly (almost 10 percent of Jalisco’s population is 65 or older) do not have access to the benefits of social security, such as health care services and pensions, which makes them highly vulnerable to suffer a situation of poverty. Although the ideal solution to their exclusion problem would be a publicly funded social protection system with integral benefits (Levy and Schady, 2013), this would transcend the institutional capacities of a state government. The program, instead, offered a more modest response to the problem, consisting of a monthly pension equivalent 50 USD with the explicit goal of helping Jalisco’s senior population to reduce their economic vulnerability and dependence. Now consider another social policy intervention titled “Music for development” that provides music training to children and adolescents, and promotes their participation in ensembles, choral groups, and community orchestras. The program defines its policy problem as “the limited participation of children and adolescents in Jalisco’s cultural and artistic life”, and its goal as “to contribute to the integral development of children and adolescents who face social vulnerability”. These two social programs are only two examples of the 256 interventions that the government of Jalisco carried out in 2017 to improve the wellbeing of its residents, particularly those who face high social vulnerability. For that end, Jalisco developed numerous planning, decision-making and evaluation frameworks aiming to improve the quality of its actions and programs, but particularly to guarantee that public agencies ground their interventions on a rationalistic, evidence-based approach to public policymaking. To what extent do programs such as “In Support for the Elderly” and “Music for Development” suggest that such a framework has in fact served to improve the logical consistency of social policy?

To analyze the internal consistency of social programs in Jalisco, I carefully examined 107 social policy interventions in the dataset to assess the definition of the policy problem, the pertinence of policy goals, and the degree of coherence of the policy tools employed9. Regarding problem definition, I examined whether the intervention defined the policy problem as an undesirable social or economic situation requiring a specific policy action for its improvement, and offering arguments and empirical evidence to support such assessment. I regarded all interventions as exhibiting a high-quality problem definition if they completely satisfied such a standard. Conversely, an intervention that merely enunciated the problem as an undesirable situation, but without providing any evidence on its significance received a medium-quality score. Finally, low-quality interventions were those that failed to comply with the two requirements. Concerning the policy goals dimension, the highest standard demands a complete correspondence between the ways a problem was defined and the expression of the goals of the policy intervention (if the correspondence is partial, the intervention is rated as medium-quality; if it is null, it is considered low-quality). Finally, to evaluate the coherence of policy tools, the maximum standard requires that the benefits, services or actions delivered by the interventions could contribute to the achievement of their policy goals in a specific target population. In other words, it presupposes the existence of a causal link between what a program delivers to its beneficiaries and the policy change it attempts to fulfill among them. A medium-quality intervention is one whose benefits or services can reasonably contribute to achieve its goals, but not among the target population envisaged. A low-quality intervention would fail to establish a causal link between its tools, goals and target populations. Table 2 describes the criteria to assess the internal consistency of social programs.

Table 2. Internal coherence assessment criteria

Problem definition	Rating
The problem is defined as a negative social or economic situation that should be improved through the program or action.	High
The problem is defined as a negative social or economic situation, but its improvement through the program or action is not explained.	Medium
The problem is not well-defined.	Low
Relevance of policy goals
The objectives of the program or action are fully consistent with the definition of the problem.	High
The objectives of the program or action are partially consistent with the definition of the problem.	Medium
The objectives of the program or action are not consistent with the definition of the problem.	Low
Coherence of policy tools.
The benefits or services offered by the program are likely to accomplish the policy goals in its target population.	High
The benefits or services offered by the program are likely to accomplish the policy goals, but its target population is not well-defined.	Medium
Although the program’s target population is well-defined, the benefits or services offered are not likely to accomplish the policy goals.	Medium
Neither the benefits or services offered by the program are likely to accomplish the policy goals, nor its target population is well-defined.	Low

Source: author’s own elaboration.

To illustrate the strategy, consider again the two examples used in the beginning of this section. The first one, “In Support for the Elderly”, has a well-defined policy problem, as it highlights the negative economic situation that the elderly undergo. The goals of the program, although limited, are entirely consistent with the problem formulation. The policy tool created to contribute to address the problem (the monthly allowance) is also consistent with the program’s goal, even though it is very far from the ideal solution consisting of creating a universal social protection system, a policy strategy that not even the national government has already attempted to adopt. In contrast, “Music for Development” constitutes an illustrating example of a poor problem definition, since it never explains why children’s limited participation in culture and arts represents a social problem, what adverse consequences such situation has for society, and it does not either provide any evidence about its magnitude, evolution or distribution. Consequently, its goal is inconsistent with the problem formulation. Finally, the program does not explain how the provision of music training courses will reduce children’s vulnerability. These critical observations to the program do not imply that the provision of music training for socially deprived children is a worthless or mistaken policy intervention10; they only denote that the way it was conceived lacks internal consistency.

Table 3 shows the results of the internal consistency assessment. Clearly, the definition of policy problems is the weakest element, as more than 37 percent of interventions exhibit a low level of internal consistency, while only 7.5 percent present a high level. Although goal consistency is closely linked to problem definition, there are more interventions with high levels of goal consistency (56 percent). The coherence between policy tools and goal turns out to be the best-rated dimension, considering that 73 percent of all interventions exhibit a high degree of consistency.

Table 3. Internal consistency analysis

Degree of consistency	Problem definition		Relevance of goals		Coherence of policy tools
	Absolute frequency	%	Absolute frequency	%	Absolute frequency	%
Low	40	37.38	30	28.04	6	5.61
Medium	59	55.14	17	15.89	23	21.5
High	8	7.48	60	56.07	78	72.9
Total	107	100	107	100	107	100
Average consistency: 3.65 / Standard deviation: 1.48 / Minimum: 0 / Maximum: 6

Source: own elaboration based on SIMAPP 2017.

4.2. Atomization

Regarding the horizontal coherence between social policy interventions, I argue that two or more interventions can be considered as “coherent” if they comply with at least one of three conditions: a) they do not duplicate with each other, b) they complement with each other, or c) they are independent from each other. Therefore, I classified social policy interventions into three categories: duplicate interventions, complementary interventions, and independent interventions. Identifying each type of intervention implied analyzing two variables only: 1) the target population they aim to serve, and 2) the benefits or services they provide. Duplicate interventions are those that provide the same benefits or services to the same target populations; complementary interventions provide different benefits of services to the same target populations; independent interventions provide benefits or services to different target populations. Figure 2 describes the identification strategy. To illustrate it, consider the case of two different interventions providing cash support for the elderly. Since there is a complete coincidence in the benefit delivered and in the target population, we can say that those are duplicate interventions and that only one of them should subsist. Then, consider a third program that provides free transportation coupons to the elderly. Since this program provides a very different type of benefit, but it targets it to aging persons, we can say that the third program is a complementary intervention.

Figure 2. Identification of duplicate, complementary and independent interventions

		Target population
		Identical	Different
Benefits or services provided	Identical	Duplicate interventions	Independent interventions
	Different	Complementary interventions	Independent interventions

Source: author’s own elaboration.

To classify the 256 social policy interventions according to the policy coherence methodology outlined above, I had first to identify all the benefits and services provided by all interventions, which implied classifying them in meaningful categories. I generated 23 different groups. The most common type of benefit among the 256 interventions is cash allowances (94), followed by training and certification services (21), infrastructure projects (18), diverse aid such as food, health-care or shelter (17), and provision of material resources (13). The next step consisted in identifying the target population and the types of benefits and services of each intervention. The first task was not straightforward, since the target population of an intervention is usually defined as a combination of different attributes. A program’s beneficiary can be either a person (a woman, a child, an aging person) or a formal organization (a school, a company, a hospital). When its intended recipients are persons, the program can define them in terms of age group, economic activity or social vulnerability status. Furthermore, interventions might be intended to cover the whole territory of Jalisco or only some specific regions (for example Guadalajara’s metropolitan area or the rural municipalities). Therefore, I created unique target population categories based on the following variables: a) age group, b) geographical coverage, c) professional or economic activity, d) membership to any vulnerable group category, and e) type of organization (when interventions target organizations rather than persons). After applying such classification criteria, I discovered that the 256 interventions aim at 99 different population categories. This means that there are, on average, 2.6 social policy interventions directed to serve the needs of the same population category. The five population categories with more interventions targeted towards them are municipalities outside the Guadalajara metropolitan area (15 social policy interventions), the general population (13), children and adolescents living outside the Guadalajara metropolitan area (12), all municipalities of Jalisco (11), and indigenous persons aged 12 or older living outside the Guadalajara metro area (9). However, there are 53 target population groups for which there is only one intervention; these categories include, for example, elderly teachers residing in rural municipalities, NGO’s aiding persons with disabilities, basic education schools in Guadalajara’s metro area and schools outside it. Together, the 53 interventions serving only a specific target population represented 46 percent of the budget allocated towards social policy interventions in 2017.

Finally, to identify duplicate, complementary and independent interventions, the 99 target population categories were intersected with the 23 benefits and services typology, following the strategy outlined above. Table 4 reports the results. I found 108 interventions grouped in 41 clusters. Each cluster represents a set of interventions providing different types of benefits or services to the same target population. This implies that all interventions within each cluster are complementary, which means that they have the potential to interact with each other to serve the needs of the same population in a more comprehensive way. However, I must alert readers that the complementarity of all intervention within a cluster is only a hypothetical attribute: in order for those interventions to create synergies among them, it is necessary to have coordination mechanisms in place, mainly a system to share information and a well-defined set of responsibilities among those in charge of their management (Cejudo and Michel, 2017). Although the dataset does not allow verifying such information, it at least describes the organizational unit responsible for the execution of each intervention. Therefore, we can assume that, at least in principle, coordination will be less complex if a complete cluster of complementary interventions falls under the responsibility of the same managerial unit. In contrast, if two or more units share execution responsibility for interventions within the same cluster, coordination will be more difficult because all the units should agree to share information about their target populations and define a very clear division of functions to attend their beneficiaries. The data reveals that 42 interventions falling in one of the 41 complementary clusters are doomed to suffer coordination problems, as those clusters do not fall under the responsibility of the same organizational unit. The remaining 87 complementary interventions are, in principle, subject to less coordination challenges, as each of their clusters belongs to the same organizational unit. Nevertheless, they might be operating in an uncoordinated way: from interviews with program managers and frontline workers, I learned that many interventions implemented by the same agency tend to have their own information registries that administrators rarely share with their peers.

Table 4 also shows that there are only 25 duplicated interventions, which represents less that 10 percent of all social policy actions and programs. However, this finding is hardly surprising: this is a logical consequence of having so many different target populations for which there are very few interventions.

Table 4. Horizontal consistency analysis

Clusters with complementary interventions	41
Complementary interventions	108
Interventions per cluster
Average	2.6
Minimum	2
Maximum	9
Duplicate interventions	25
Independent interventions	123
Total social policy interventions	256

Source: own elaboration based on SIMAPP.

5. DISCUSSION

The case of Jalisco illustrates the contradictions between a rationalistic, evidence-based policymaking approach generally advocated as a standard worth following, and the actual way governments design their social policy interventions. The first relevant finding is that social policies in Jalisco exhibit a poor level of internal consistency, particularly regarding the formulation of policy problems, as these fail to be expressed as undesirable situations requiring specific actions for their improvement, supported by arguments and evidence to persuade about their social significance. From a rational-comprehensive perspective, that finding would be interpreted as a significant failure of policy design, casting serious doubts about the actual usefulness of Jalisco’s system of policy monitoring and evaluation. Furthermore, considering that very few state governments in the country have systems enjoying such a good reputation as in Jalisco, the finding would suggest that policy design failures are prevalent in Mexico. However, the finding should not necessarily be regarded as an anomaly if we consider the different criticisms towards policy rationalism discussed in section 2, particularly that defining a policy problem is not a merely technical task, but a political endeavor that entails negotiation and mutual adjustments between different players. In fact, it might well be the case that many social programs were created as reactions of governments to different short-term pressures or the approach elected authorities decide to convey their political compromises. Nevertheless, social policy interventions, no matter their real origins, once they are adopted, are officially required to be adjusted to the administrative protocols, particularly to the “management for results” framework, that prevails in Mexico and other Latin American countries (Arellano-Gault and Ramírez, 2008). In agreement with Kingdon’s multiple stream approach, the design of social policy in Jalisco is a good example of solutions in search of their problems.

The second finding has to do with how social programs are articulated with each other. Surprisingly, policy duplication is not a widespread difficulty in Jalisco, as only 10 % of all interventions exhibit such a drawback. The real problem is the atomization of social programs: there is a very large number of target populations that social interventions attempt to serve (99 different types of beneficiaries, most of them attended by custom-made programs or actions). In other terms, there are too many interventions aim at particularized interests, not at broader social groups. This does not mean that interventions have in fact high coverage rates (on the contrary, they reach a very small fraction of the potential number of beneficiaries, as I could observe in government registries), but only that they intend to satisfy the needs of many types of people and organizations. I claim that this feature is problematic, mainly because it might end up exacerbating social inequalities: targeting assigns people labels that can become social stigmata, which in turn leads to discrimination. As stated by Amartya Sen, “any system of subsidy that requires people to be identified as poor and that is seen as a special benefaction for those who cannot fend for themselves would tend to have some effects on their self-respect as well on the respect accorded them by others” (Sen, 1992, p. 13). In addition, the targeting of social policy might create a situation of subordination of beneficiaries to bureaucrats and local elites, in which the allocation of benefits does not respond to people’s need, but on their political loyalties or other considerations (Mosse, 2004), which evidently contradicts the human rights-based approach that contemporary governments in Mexico claim to be adopting.

Finally, although it is not the purpose of this article to identify the underlying causes of policy fragmentation in Jalisco, from the review of the literature on policy design we could hypothesize that the low degree of internal consistency of social programs and its high level of atomization is related to what May and Jochim (2013) describe as “policy regimes”, namely the existence of governing arrangements allowing policies to organize around a narrow set of actors, interests and institutions. The available evidence clearly suggests that social policy in Jalisco can hardly be regarded as a policy regime in a strict sense because of the high number of target populations it aims to serve, because the social and economic problems that those populations face are very different, and also because the operation of social programs falls under the responsibility of more than 20 government agencies. In sum, a fragmented social policy regime is very likely to generate a fragmented set of social programs. Further research is required to investigate this proposition.

6. CONCLUSIONS

This article provides evidence that social policymaking at the subnational level in Mexico lacks internal consistency and it is highly atomized, as too many programs and actions aim at particularized interests, instead of addressing the problems of broader populations. These features, so often regarded as negative attributes in policy design, persist in a subnational government setting in which the formulation, monitoring and evaluation of public policy have acquired a high degree of institutionalization and have been praised nationwide and overseas. However, the literature that serves as the theoretical basis for this article emphasizes that policy incoherence and fragmentation is an ordinary outcome explained by political factors, such as the struggle of competing interests to impose their own problem definitions and solutions in the policy agenda, the short-term pressures that elected authorities have to sort out to assure their political survival, and the strength of policy domains to organize the goals and actions of different actors, target populations and institutions. Considering these insights, it would not be unreasonable to presume that social policies are not the deliberative response of governments to social problems demanding solutions, but political reactions of many sorts whose goals, tools and target population are not logically aligned with each other and they neither intend to contribute to the goals of other existing policies.

REFERENCES

Arellano-Gault, D., & Ramírez, E. (2008). Performance-Based Budgeting in Latin and South America: Analyzing Recent Reforms in the Budgetary Systems of Brazil, Chile, Colombia, and Mexico. International handbook of practice-based performance management (pp. 297-320). SAGE Publications, Inc. https://www.doi.org/10.4135/9781412982719

Bardach, E., & Patashnik, E. M. (2015). A practical guide for policy analysis: The eightfold path to more effective problem solving. CQ press.

Baumgartner, F. R., & Jones, B. D. (2013). Agendas and Instability in American Politics, Second Edition. University of Chicago Press. https://doi.org/10.7208/chicago/9780226039534.001.0001

Beer, C. C. (2003). Electoral competition and institutional change in Mexico. University of Notre Dame Press.

Bornemann, B. (2016). Integrative Political Strategies—Conceptualizing and Analyzing a New Type of Policy Field. European Policy Analysis, 2(1), 168-195. https://doi.org/10.18278/epa.2.1.10

Bouckaert, G., Peters, B. G., & Verhoest, K. (2010). The Coordination of Public Sector Organizations. https://doi.org/10.1057/9780230275256

Cairney, P., & Jones, M. D. (2016). Kingdon’s Multiple Streams Approach: What Is the Empirical Impact of this Universal Theory? Policy Studies Journal, 44(1), 37-58. https://doi.org/10.1111/psj.12111

Carbone, M. (2008). Mission Impossible: the European Union and Policy Coherence for Development. Journal of European Integration, 30(3), 323-342. https://doi.org/10.1080/07036330802144992

Cejudo, G. M., & Michel, C. L. (2016). Coherencia y políticas públicas. metas, instrumentos y poblaciones objetivo. Gestión y Política Pública, 25(1), 3-31. http://www.gestionypoliticapublica.cide.edu/ojscide/index.php/gypp/article/view/149

Cejudo, G. M., & Michel, C. L. (2017). Addressing fragmented government action: coordination, coherence, and integration. Policy Sciences, 50(4), 745-767. https://doi.org/10.1007/s11077-017-9281-5

COMISIÓN ESTATAL DE LOS DERECHOS HUMANOS DE JALISCO – CEDHJ (2017). Diagnóstico e informe especial sobre la situación de las personas con discapacidad en Jalisco. Instituto de Investigación y Capacitación en Derechos Humanos. CEDHJ. https://bit.ly/3pHrluk

CONEVAL (2020). Informe de pobreza y evaluación: Jalisco 2020. Mexico. https://www.coneval.org.mx/coordinacion/entidades/Documents/Informes_de_pobreza_y_evaluacion_2020_Documentos/Informe_Jalisco_2020.pdf

Cunill-Grau, N. (2014). La Intersectorialidad en las Nuevas Políticas Sociales: un acercamiento analítico-conceptual. Gestion y Política Pública, 23(1), 5-46. http://www.gestionypoliticapublica.cide.edu/num_anteriores/Vol.XXIII_NoI/01%20GyPP%20Nuria%20Cunill%20(1-46).pdf

Cornelius, W. A., Craig, A. L., & Fox, J. (Eds.) (1994). Transforming state-society relations in Mexico: The national solidarity strategy. Center for US-Mexican Studies, University of California.

De La O, A. L. (2015). Crafting Policies to End Poverty in Latin America. Cambridge University Press. https://doi.org/10.1017/cbo9781316105528

Dery, D. (2000). Agenda setting and problem definition. Policy Studies, 21(1), 37-47. https://doi.org/10.1080/014428700114008

De Schutter, O. (2021). A human rights-based approach to measuring poverty. In M. F. Davis, M. Kjaerum, & A. Lyons (Eds.), Research Handbook on Human Rights and Poverty (pp. 2-20). https://doi.org/10.4337/9781788977517.00010

Diaz-Cayeros, A., Estevez, F., & Magaloni, B. (2016). Poverty Relief in Latin America. In The Political Logic of Poverty Relief (pp. 25-44). Cambridge University Press. https://doi.org/10.1017/cbo9781316492710.002

Esquivel, G. (2012). The Dynamics of Income Inequality in Mexico since NAFTA. Economía, 12(1), 155-179. https://doi.org/10.1353/eco.2011.0009

Gracia, M. A., & Horbath, J. E. (2018). Exclusión y discriminación de indígenas en Guadalajara, México. Revista Perfiles Latinoamericanos, 27(53). https://doi.org/10.18504/pl2753-011-2019

Hernandez-Trillo, F. (2016). Poverty Alleviation in Federal Systems: The Case of México. World Development, 87, 204-214. https://doi.org/10.1016/j.worlddev.2016.06.012

Howlett, M., Mukherjee, I., & Woo, J. J. (2015). From tools to toolkits in policy design studies: The new design orientation towards policy formulation research. Policy & Politics, 43(2), 291-311. https://doi.org/10.1332/147084414x13992869118596

Howlett, M., Mukherjee, I., & Howlett, M. (2017). The elements of effective program design: a two-level analysis. In M. Howlett, & I. Mukherjee (Eds.), Handbook of Policy Formulation (pp. 129-144). Edward Elgar Publishing. https://doi.org/10.4337/9781784719326.00014

Kingdon, J. W. (1995). Agendas, Alternatives, and Public Policies. Harper Collins College Publishers.

Levy, S., & Schady, N. (2013). Latin America’s Social Policy Challenge: Education, Social Insurance, Redistribution. Journal of Economic Perspectives, 27(2), 193-218. https://doi.org/10.1257/jep.27.2.193

Lindblom, C. E. (1968). The policy-making process. Prentice-Hall.

Majone, G., & Quade, E. S. (Eds.) (1980). Pitfalls of Analysis. John Wiley & Sons.

May, P. J., & Jochim, A. E. (2013). Policy Regime Perspectives: Policies, Politics, and Governing. Policy Studies Journal, 41(3), 426-452. https://doi.org/10.1111/psj.12024

May, P. J., Sapotichne, J., & Workman, S. (2006). Policy coherence and policy domains. Policy Studies Journal, 34(3), 381-403. https://doi.org/10.1111/j.1541-0072.2006.00178.x

Mosse, D. (2004). Power relations and poverty reduction. In R. Alsop (Ed.), Power, Rights, and Poverty: Concepts and Connections. World Bank. https://doi.org/10.1596/978-0-8213-6310-2

Peters, B. G. (1998). Managing Horizontal Government: The Politics of Co-Ordination. Public Administration, 76(2), 295-311. https://doi.org/10.1111/1467-9299.00102

Pollitt, C. (2003). Joined-up Government: a Survey. Political Studies Review, 1(1), 34-49. https://doi.org/10.1111/1478-9299.00004

Pressman, J., & Wildavsky, A. (1974). Implementation: How Great Expectations in Washington are Dashed in Oakland. University of California Press.

Sen, A. (1992). The Political Economy of Targeting. Annual Bank Conference on Development Economics. World Bank.

Schneider, A., & Ingram, H. (1993). Social construction of target populations: Implications for politics and policy. American political science review, 87(2), 334-347. https://doi.org/10.2307/2939044

Stokey, E., & Zeckhauser, R. (1978). A Primer for Policy Analysis. WW Norton.

Stone, D. (2002). Policy Paradox: the Art of Political Decision Making. WW Norton.

Teles, S. M. (2013). Kludgeocracy in America. National Affairs, 17, 97-114. https://www.nationalaffairs.com/publications/detail/kludgeocracy-in-america

Trein, P., Meyer, I., & Maggetti, M. (2019). The Integration and Coordination of Public Policies: A Systematic Comparative Review. Journal of Comparative Policy Analysis: Research and Practice, 21(4), 332-349, http://doi.org/10.1080/13876988.2018.1496667

Weiss, J. A. (1989). The Powers of Problem Definition: The Case of Government Paperwork. Policy Sciences, 22, 97-121. https://doi.org/10.1007/bf00141381

Wildavsky, A. (2017). The Art and Craft of Policy Analysis. Palgrave MacMillan. https://doi.org/10.1007/978-3-319-58619-9