Bridging the qualitative-quantitative divide in comparative migration studies: newspaper data, and political ethnography in mixed method research

Abstract

Newspaper data are popular in Comparative Migration Studies as they allow diachronic and cross-national comparison and are relatively easy and inexpensive to acquire. Critics, however, warn that newspaper data are hampered by selection, description and researcher bias. This article argues that research drawing on newspaper data can be improved by employing mixed methods to confirm and complement data and to analyze findings from different research paradigms. To demonstrate this claim, I ethnographically re-analyze part of the dataset of the Mobilisation on Ethnic Relations, Citizenship and Immigration (MERCI) project. I chose MERCI as publications based on its data, which I am well acquainted with, have inspired numerous researchers and European comparative research projects to pursue newspaper content analysis. This article shows how an ethnographic approach can address description bias and researcher unreliability, and reveal selection bias. It offers concrete suggestions for incorporating political ethnography into newspaper analysis. In doing so it advocates a new path for empirical research in Comparative Migration Studies, one that bridges the qualitative-quantitative divide. The conclusion encourages researchers on both sides of the quantitative-qualitative spectrum to reconsider habitual and safe research paradigms and move towards the middle to improve the quality of their work.

Introduction

Over the past decades, newspapers have emerged as prominent sources in the Comparative Migration Studies. Facilitating comparative, historical and quantitative research, newspapers are a recognized and legitimate source for scholars around the world. But as newspaper analysis has grown in popularity, researchers have pointed to selection, description and researcher bias, questioning the quality of the data and suggesting limitations to their usefulness. Critics argue that whatever measures are taken to overcome these hurdles, problems with the data remain.

Rather than offering an additional critique, this article assesses the merits of existing criticisms on the basis of empirical data. It advocates two mixed-methods approaches to newspaper data research: mixed data-collection (relying on at least two kinds of data) and mixed data-analysis (using multiple or cross techniques regardless of the type of data) (Small, 2011, p. 60). The article argues for combining political ethnography with content analysis within the ongoing discussion on how to bridge the quantitative-qualitative divide in the social sciences (Goertz & Mahoney, 2006; McBride & Mazur, 2010) and posits mixed-methods research as a third major research paradigm (Johnson, Onwuegbuzie, & Turner, 2007). Contrary to widespread perceptions, ethnographic research is well suited to this endeavor as it is already a mixed-methods approach entailing both qualitative and quantitative methods (Schensul, Schensul, & Lecompte, 2013, p. 8).

The suggestions in this article are based on an ethnographic re-analysis of Dutch data from the completed Mobilisation on Ethnic Relations, Citizenship and Immigration (MERCI) research project, in which newspapers were the primary source of data (Koopmans, Statham, Giugni & Passy, 2005). MERCI is chosen for two reasons. First, Koopmans and his co-authors developed and fine-tuned ‘claims-making’, this methodology has inspired numerous comparative European research projects.¹ Second, MERCI, and more specifically the data on the Netherlands, is chosen because of my familiarity with this case.

The following sections review established critiques of newspaper analyses and the political claims-making method based on newspaper data as developed by Koopmans et al. (2005). I then explain the underlying rationales for political ethnography, how political ethnography is a mixed-methods approach, and why the current project can be understood as political ethnography. These rationales – confirming and complementing data, analysis from the perspectives of different methods and paradigms – guide the re-analysis of the MERCI data. The first part of the analysis shows how ethnographic field-specific expertise can reveal biases in newspaper data. The second part shows what political ethnography can add to the interpretation of data. It places the claims in broader context and evaluates how these insights amend Koopmans et al.’s (2005) conclusions. Subsequently, I propose mixed-method tools for research building on newspaper content analysis through the integration of what I label validating and explorative political etnograhpy. The final sections reflect on these recommendations in light of ongoing challenges faced by researchers interested in mixed methods and encourage a new path for future empirical research in Comparative Migration Studies.

Newspaper analysis: critiques and approaches

The content analysis of newspapers to identify actors, events, and actions has been with us since the mid-1930s (Davenport, 2010, pp. 4–9). Their design has always been straightforward, with researchers developing ‘explicit coding rules that defined the behavior of interest as well as specific characteristics that were identified during the coding process’ (Davenport, 2010, p. 6). Such data collection allowed researchers to collect and compare data which could be examined in multiple ways.

Despite these advantages, the strategy’s limitations have repeatedly been pointed out over the past four decades. Critics stress that newspaper data is vulnerable to three biases (McCarthy, McPhail, & Smith, 1996). First, newspapers report only a fraction of what could potentially be reported, with violent, larger, or protracted events more likely to be covered (selection bias) (McCarthy, Titarenko, McPhail, Rafail, & Augustyn, 2005). What ends up being published largely depends on editors’ professional networks and their instincts about what will appeal to readers (Ortiz, Myers, Walls, & Diaz, 2005). Second, errors occur in the descriptions of reported events (description bias). Third, the reliability of choices made by researchers has been questioned; no matter how advanced their codebooks, the risk remains that researchers interpret codes differently depending on their personal knowledge (researcher bias) (Franzosi, 1987).

To address the problems outlined above, scholars have suggested the triangulation of media sources, the use of electronic archives, and methods employed in survey research to address non-random response rates (Earl, Martin, McCarthy, & Soule, 2004). Recently, scholars have suggested that description and selection bias can be reduced by drawing on multiple newspapers (Koopmans & Statham, 2010, Van der Brug, D’Amato, Berkhout, & Ruedin, 2015) others have relied on interviewing journalists (Fillieule & Jiménez, 2003). Davenport’s study of the Black Panther Party (2010) has shown that the frequency and type of reported events depend on one’s sources. To reveal which events are missed, he suggests using as many sources as possible. These remedies, however, have not put to rest criticisms of selection, description and researcher bias in claims-making analysis (Ortiz et al., 2005).

Political claims analysis: data and methodology

The political claims-making method was developed to fine-tune conventional protest event analysis (PEA), which tried to systematically map, analyze and interpret the occurrence and properties of large numbers of protests through newspaper content analysis (Koopmans & Rucht, 2002). PEA has been criticized for presenting a highly selective and biased view of contentious politics, for its rigid distinction between challengers and the establishment, and for failing to capture the discursive side of claims-making (Koopmans & Statham, 1999a, b). For Koopmans and Rucht, objections to PEA on the grounds of misrepresenting ‘reality’ are unwarranted: ‘It is based on a constructed reality… of extreme importance for both policy makers and the wider public.… In a certain sense, protests and other events that remain unreported are simply “non-existent”’ (2002, p. 252).

Political claims analysis takes individual instances of claims-making by public actors as the unit of analysis. In contrast to other types of media analysis, it is expected to provide information about who addresses whom on which issues (Koopmans, 2007, p.189). The demands of, for example, ethnic organizations are expressed in the form of collective identities such as ‘We Muslims have the right to housing’. Claims consist ‘of the purposive and public articulation of political demands, calls to action, proposals, criticisms, of physical attacks, which, actually or potentially, affect the interest or integrity of the claimants and/or other collective actors’ (Koopmans et al., 2005, p. 252).

Koopmans et al.’s (2005) study of ethnic mobilization was based on the MERCI project, which compared political claims-making in the Netherlands, France, Germany, Switzerland and Great Britain. The source for political claims in the Netherlands was the leading liberal daily, the NRC Handelsblad; claims were coded from the Monday, Wednesday and Friday issues for the period 1992–1999. The coding focused on events that had occurred in the 2 weeks prior to being mentioned and excluded retrospective coverage of older claims. Each claim was coded once – the first time it was encountered. Repetitions in subsequent articles were not coded separately, although additional information was used to complete the data on the origin claim.

Koopmans et al. (2005) explicitly address selection, description and researcher biases (pp. 254–65). First, they concede that newspaper coverage is neither undistorted nor a complete mirror of reality; only a small part of the claims made on a daily basis are actually covered. This, however, is precisely the part of claims-making that interests them: ‘the part that can have an impact on the perceptions of the public or on policy making’ (p. 25). Second, they hope to minimize description bias by coding only factual statements and events and excluding additional comments by editors and reporters. Third, the authors assure that inter-coder reliability has been enhanced by a codebook with numerous examples as well as by regular discussion of problematic cases. The resulting data set is claimed to be suitable for macro-level comparative analysis and for focusing on the qualitative aspects of particular actors and claims (Koopmans et al., 2005, p. 27).

These properties led me to use the MERCI data set for a study on transnational politics of Turkish, Kurdish and Surinamese immigrants in the Netherlands. Back then, in 2002, transnational scholarship was almost exclusively based on ethnographic case studies; systematic comparative work was lacking (Waldinger & Fitzgerald, 2004). To fill this void Koopmans et al. (2005) applied the method of political claims-making to examine immigrant political activities that focus on their countries of origin. They had concluded that transnationalism is primarily driven by exclusion from host country political opportunities. In their view, homeland influences are: ‘relatively marginal to understanding patterns of migrant claims-making. Our argument for the retention of national citizenship approaches… is not a normative one, but a factual one. Our evidence simply shows that national citizenship shapes transnationalism’ (Koopmans & Statham, 2001, p. 95).

More recent European comparative research projects have significantly improved data collection and interpretation of claims making. Projects have adapted and fine-tuned the MERCI codebook (Berkhout & Sudulich, 2011; Castelli Gattinara, Cinalli, Giugni, & Morales 2015; Hoksbergen & Tillie 2012), have combined newspaper data with interviews, surveys, focus groups, existing statistics and policy documents (Cinalli & Giugni, 2013; Morales & Giugni 2011; Statham & Tillie, 2016; Van Heelsum & Koomen, 2016), redefined claims and actors to account for cross-national differences (Berkhout, 2015) and applied other techniques, such as network analysis, on the same data (Cinalli & O’Flynn, 2014). While these improvements have been complementary two limitations remain. First, additional data has not been used within a mixed method research design or with the purpose to validate newspaper data (but see Helbling & Tresh, 2011). Second, none of these studies have included political ethnography in their methodological and analytical toolboxes. Some authors even outright dismiss qualitative approaches to content analysis (Vanparys, Jacobs, & Torrekens, 2013).

Political ethnography and mixed methods

This article sees ethnography as an approach rather than a uniform method which is guided by ‘its commitment to direct experience with a population or community of concern. It is this continuous exposure and engagement with a research setting, rather than the specifics of a methodology alone, that distinguishes ethnography from other research approaches’ (Schensul et al., 2013, p. 2; for such an approach to content analysis see Korteweg & Yurdakul, 2009). The approach when observing and talking to the actors in the MERCI sample was open to their views, interpretations and memories of the reported events, following the ethnographic tenet that the best way to understand a different cultural setting is to immerse oneself in it (Schensul & Lecompte, 2013, p. 1). Such immersion often includes fieldwork that is characterized by being present in a research setting to gain sufficient trust (Schensul et al., 2013, p. 3). Ethnographers continuously observe behavior, conduct informal interviews and participate in activities which they document in field notes (Schensul et al., 2013, p. 14). This is time consuming and does not necessarily produce accurate data; no researcher can be everywhere or know everyone in a site. Nevertheless, it is believed that constant exposure enables the ethnographer ‘to make accurate guesses about what might or what might not be an appropriate response to a question’ (Schensul et al., 2013).

Ethnography is both inductive and deductive and is by no means only qualitative. The process of testing and retesting to approximate consistency of observation can be applied to a variety of data, be it observational material or survey findings (Schensul et al., 2013, p.5). Ethnographers may use surveys and other instruments to test concepts and theories derived from other fields or from ‘outsider’ observation. But they argue that such instruments and the theories directing them should not be arbitrarily used without testing them locally for both practical and theoretical applicability (Lecompte & Schensul, 1999; Snow, Morrill, & Anderson, 2003).

Critics who believe that science is or should be ‘objective’ often argue that the personal characteristics of the researcher bias data collection and interpretation, making ethnography highly subjective and unable to meet standard conceptions of validity and reliability. However, these criticisms apply more to classic anthropological fieldwork conducted individually in remote areas than to ethnography conducted in urban areas to test or better understand survey or newspaper data. Maintaining objectivity requires that researchers select the best interview or observation and carefully justify their decisions (Schensul et al., 2013, p. 19).

The strength of ethnography is its internal validity, arising from ethnographers’ familiarity with the research field in all its complexity. As for external validity, ethnographers are able to adapt or create locally appropriate aids to data collection that can yield predictive theories and hypotheses that can be applied to the same situation or to other similar situations using the same research methods and data collection techniques (Lecompte & Schensul, 1999, p. 3).

The re-analysis of the claims – the evidence on which Koopmans et al.’s (2005) conclusions are based - draws on my own data collection which included a survey in the Netherlands (N = 101), 241 in-depth interviews and participant observation during numerous political events in the Netherlands, Turkey and Surinam. I learned Turkish and the Surinamese lingua franca, Sranantongo, closely followed the daily news in Turkey and Surinam through newspapers, television and radio and reported on my observations in research diaries – approximately 2–3 pages per day in the field between 2003 and 2005 (Nell, 2008a; Mügge, 2010; Mügge, 2013a, b). I gained in-depth knowledge of the cases and encountered most of the actors and events coded from newspaper articles in the MERCI sample. With one Kurdish actor who was most quoted in the newspapers, I systematically reviewed all claims collected for Turks and Kurds during a four-hour interview. Subsequent triangulation with other sources confirmed the interpretations that surfaced in this interview.

My ethnography was political and differs from traditional anthropological ethnography in that it does not focus on culture, but on politics (Baiochhi & Connor, 2008). Political ethnography is conducted by political scientists who are ‘open to simplify for analytic coherence, and to seek to produce generalizable results’ (Schatz, 2009, p. 306). This implies ‘attention to cross-case comparisons, to broadly occurring factors’ (Schatz, 2009). I studied political and ethnic groups located or moving in and between at least two national contexts; the events I participated in were those organized by a particular network of political groups (for a similar approach see Fitzgerald, 2006). In this sense I broke with the classical ethnographic approach that involves participation in one locality for a long period of time. The trust I gained was not so much because the same group of people saw me at the same site time and again, but because people had heard from others that I was there. In other cases, I gained trust by telling respondents about events and showing them my knowledge and willingness to participate in their activities. Sometimes only the fact that I had taken the effort to study the Turkish language was enough to break the ice.

My participation spanned geographical contexts; its length was often short and depended on the accessibility of groups. For instance, I spent more time with groups that adhered to extreme political ideologies including certain Kurdish groups, the extreme left and right, as it was usually more difficult to schedule interviews with them. Anthropologists have debated to what extent multi-sited fieldwork with short visits still counts as ethnography (Hendry, 2003; Marcus, 1995). As ethnography in this article is understood as an approach rather than a method, I argue that it does.

Mixed methods have increasingly become mainstream in ethnography and the social sciences and may be motivated by confirmation, complementarity (Small, 2011, p. 63) and initiation (Schensul et al., 2013, p. 158). The rationale behind confirmation is to verify or to ‘triangulate’ the findings from one type of data with those derived from others (Small, 2011, p. 63). Seeking agreement with findings derived through different methods is particularly important as it counteracts various sources of bias (Schensul et al., 2013, p. 158). Complementarity refers to the value of combining different types of data to compensate for their various weaknesses (Small, 2011, p. 63). It increases interpretability, meaningfulness and validity, and counteracts biases in other methods and sources (Schensul et al., 2013, p. 158). Finally, initiation interrogates one method with another; it increases depth by analyzing findings and interpretations from the perspective of different methods and paradigms.

The next section shows how political ethnography revealed biases in the newspaper data (confirmation) and the extent to which these biases can be corrected by complementary data; initiation will be covered in the subsequent section.

Biases

The quality of newspaper data can be improved with ethnographic tools to address selection, researcher and description biases. In total the MERCI sample included 54 Turkish, 41 Kurdish and 19 Surinamese claims in the Netherlands, contained in 40, 28 and 11 newspaper articles respectively (Mügge, 2012a). These 114 claims have been thoroughly studied, examining the MERCI codebook, the database with the codes, the summaries of the claims and the original newspaper articles in LexisNexis. For transnational claims-making the code ‘foreign: migrant homelands and exile’ referred to both issues and actors, the latter including homeland political parties as well as organizations of political exiles in the country of residence that wished to affect homeland politics (Koopmans et al., 1999). For example, a claim would be assigned this code if supporters of a Kurdish political organization demonstrated in front of the Turkish consulate in the Netherlands to draw attention to the legal position of Kurds in Turkey. MERCI coded 18 out of 54 Turkish claims as ‘foreign: national homelands and exile’. In the Kurdish case, 27 out of 41 claims were coded ‘foreign: national homelands and exile’. None of the Surinamese claims were coded as such. However, the scope of the actor could not be established for roughly half of the claims. MERCI was unable to establish whether the first actor was a ‘national’ or a ‘foreign national’ in 28 out of 54 Turkish claims, 20 out of 41 Kurdish claims, and 7 out of 19 Surinamese claims. This biased the sample as crucial information on the identity of many actors was missing.

Selection Bias

What and who is covered by newspapers depends on several factors: the interests of reporters and newspapers in particular events and actors as well as the networks at their disposal to get first-hand information. Reporters remain the ‘gate keepers’ of claims. This creates two kinds of selection bias.

The first concerns the coverage of events. The Kurdish case confirms that violent claims considered a threat to domestic security are more likely to be reported than non-violent claims. For example, most claims were recorded at the height of the Kurdish conflict in 1999; no claims were registered in 1994 and 1998. This does not necessarily mean that Kurdish actors were transnationally inactive in 1994 and 1998, but only that their activities went unreported. A Kurdish activist retrospectively explained in a 2004 interview that the media only paid attention to demonstrations that turned violent while many peaceful marches did not receive attention.² Multiple claims in a particular year were often made in the same week, even on the same day in the same article. This suggests that a reporter was following the issue that week, but not during the rest of the year. Selection bias explains why no claims were recorded for Turks in 1998 and for Kurds in 1994 and 1998, even though both groups in the Netherlands were transnationally active in those years.

Second, newspapers are biased in which actors are named and interviewed. Actors must be considered newsworthy and must be part of a network that extends to reporters. The claims of radical actors appear much more often than those of their more modest counterparts. A closer examination of the Kurdish actors showed that the four claimants – although formally representing different organizations – belonged to the same organization, the PKK. The PKK and its affiliated organizations were continuously monitored by the Dutch Intelligence Service in the period under study; the PKK had also been placed on the list of terrorist organizations in several European countries. Additional field research revealed that the Kurdish spokespersons mentioned in the MERCI sample could all be traced to a single address in Amsterdam. The key individual, whom I interviewed extensively in 2003 and 2004, had a personal network that extended to the Dutch Minister of Foreign Affairs at the time. It was hardly surprising that he commanded media attention. He had a prominent place in the address book of NRC reporters and was often the first to be contacted for reactions to specific issues.

The selection bias created by reporters’ address books is evident when we compare NRC coverage with articles on the same events written by a journalist of Turkish descent in the Dutch multicultural magazine Contrast. These articles cover a much broader spectrum of Turkish claimants. Cross-checking mass media coverage with the movement’s own media can be an effective way to see how reporters’ address books bias the selection of events and actors. The objection that such media ‘only preach to the converted’ (Koopmans, 2004, p. 385) is actually an advantage in this respect.

Researcher bias

Coder unreliability stemmed from the lack of in-depth knowledge of Kurdish, Turkish and Surinamese immigrant organizations in the Netherlands as well as of homeland political organizations. This led to systematic coding errors and inconsistencies. First, coding assistants were unable to identify many actors. Without knowledge of claimants’ identities, one cannot establish whether their scope is transnational or not. In other cases, full names were given by reporters but wrongly identified – ‘foreign national’ or otherwise – by the coders.³ Coding was also inconsistent: the same actors were coded ‘foreign national’ in one claim but not in others.⁴

To appreciate how coding errors and inconsistencies emerge, consider this Turkish claim summarized in the MERCI database as follows: ‘T. Cetinkaya of the Turkish political party nationalist movement MHP says that they ‘are not fascists and racists. We follow Dutch law; we are no terrorist organization. It seems we never get rid of this stigma’ (my translation).

The actor of the claim (T. Cetinkaya of the MHP) is coded ‘foreign national’. The issue of the claim (to change the image of the MHP in the Netherlands) is coded ‘national’. The original newspaper passage was:

Two to three thousand Turks were present at the semi-official meeting of the nationalist movement MHP in the Haagse Houtrusthallen. Rumor has it that the undisputed Alparslan Turkes [leader of the extreme right party MHP] would come over from Turkey to give a speech. This did not happen. The organizer, T. Cetinkaya, said in The Hague that ‘we are not fascists and racists. We follow Dutch law; we are no terrorist organization. It seems that we never get rid of this stigma’ (Turken samen, 1995, my translation).

In this extract, T. Cetinkaya is not explicitly described as a MHP member. Additional research reveals that T. Cetinkaya is the spokesman of the Turkish Islamic Association, a local immigrant organization in The Hague.⁵ The code ‘foreign national’ is incorrect.

The correct coding for the scope of the issue would have been ‘foreign national’. It falls under the type ‘transplanted homeland politics’, which occurs when homeland conflicts enter the receiving country (for the typology see Koopmans et al. 2005, pp. 126–7). MHP sympathizers complain that they are labeled as fascists by Turkish leftists in the Netherlands, a conflict imported from Turkey where leftists routinely characterize the MHP as fascist (see Nell, 2008b). As an extension of homeland conflict, it has a clear transnational dimension. This is how lack of familiarity with the data can create problems. There were many more such cases in the data that has been reanalyzed.

Description bias

In-depth interviews and the study of secondary literature showed that in a number of cases, reporters misinterpreted events. Sometimes organizations and events were mistakenly connected, as in the following example. In the first year under study, 1992, PKK followers protested in front of Turkish consulates and attacked Turkish companies. They wanted the Netherlands to oppose Turkish military action against the PKK and support an independent Kurdistan. The claims were a direct response to Turkey gathering its troops to fight Kurdish insurgents, the so-called ‘Spring Operation’ (White, 2000, pp. 164–66). The original newspaper article states that the ‘spring offensive’ was announced by Kurds in Turkey rather than by the Turkish government – a clear reporting mistake. In the following year, 1993, one of the Kurdish claims was made on 21 March: ‘The police in Rotterdam prevented a confrontation between Kurdish and Turkish protesters. Turkish Kurds participated in a demonstration on the day of anti-racism. They shouted anti-Turkish slogans and burned a Turkish flag’. The interview with a key informant reveals his organization had coordinated Kurdish participation in this demonstration:

We never burned a flag! Of course you can always find a lunatic somewhere, but not where I was. Turks also participated in this organization; some Grey Wolves [extreme nationalists] among them attacked a group of Kurds. Then the [anti-Turkish] slogans started….⁶

The additional information points to probable description error in the newspaper. First, the interviewee nuances the claim that Kurds in the demonstration collectively burned a flag; it may have been an individual act. Second, it suggests that the anti-Turkish slogans were a response to the Turkish nationalists present at the demonstration.

These examples show the value of political ethnography in confirming and complementing newspaper data, underscoring the importance of mixed data collection. The following section examines how newspaper data can benefit from mixed data analysis. The overarching question emerging from this exercise is whether political claims-making as reported in newspapers is a useful method once coding errors have been corrected. Driven by initiation, the following section ethnographically reanalyzes the MERCI findings. It presents general findings from the qualitative analysis of all claims in the sample (mixed data collection) and then focuses on the consequences for conclusions drawn from newspaper data (mixed data analysis).

Mixed-methods data collection and analysis

Not all the actors could be identified after reading the original newspaper articles. Journalists often referred to the Turkish or Kurdish community as whole. But through additional research on specific claims in other newspapers, I could identify more actors than MERCI. Knowledge gained through fieldwork enabled me to identify the organizations with which actors were affiliated. In the end, I identified around three-quarters of the Turkish (40 out of 54) and Kurdish (31 out of 41) collective claimants and nearly all of the Surinamese claimants (18 out of 19). The majority of unidentified claimants were individuals who did not act in the name of an organization.

For recoding of the claims I made use of the original articles, additional news coverage, my own interviews, research notes and secondary literature. In most instances where my codes differed from MERCI, the issue has been coded as transnational whereas MERCI did not. The actors which I identified as ‘foreign national: migrant homeland and exile’ were organizations or representatives of organizations that were formal extensions of political groups in Turkey. Immigrant organizations with informal or invisible ties to such political groups, for example Milli Görüş, have not been labeled as transnational. Even with this conservative coding, I found much more transnational activity than the MERCI study.

Most of the issues summarized in the MERCI database concern transnational activities visible to the Dutch public and of interest to the broader Dutch debate on immigrant integration. The same general themes emerged in the interviews that I conducted. For example, one of the red threads in Turkish and Kurdish claims-making is the tension between the two groups. In the interview both Turkish and Kurdish organizations have been asked to elaborate on their relations in the 1990s; the key actor in the majority of the Kurdish claims has also been extensively interviewed.

The re-coding revealed that when simply counting newspaper mentions of actors and events, it is impossible to know precisely who and what is being counted. This only became fully visible after the claims were placed in a broader – in this case transnational – context. This exercise led to three conclusions. The first is that without background information, the number of actors is likely to be over-estimated. For instance, the majority of Kurdish transnational claims in the period 1992–1999 were a response to political developments in Turkey. Yet, MERCI registered no transnational Kurdish claims for 1994 and 1998. More than half of the Kurdish claims were made by (adherents of) political parties in exile, such as the PKK (12), KIC (11) and ERNK (1), while the Kurdish migrant federation FED-KOM made six claims. But as mentioned above, these seemingly diverse actors all pointed to one person who at the time represented the PKK in the Netherlands. The remaining 10 transnational claims were made by individuals or collective actors whose identity is unknown.

The second conclusion is that in-depth analysis based on as many sources as possible enables researchers to explain concentrations of events in specific periods. For example, the Turkish transnational claims in the MERCI sample are concentrated in specific periods. This is hardly surprising since 16 out of 22 – often violent and therefore newsworthy – claims were responses to preceding Kurdish actions. For instance, the establishment of the Kurdish Parliament in Exile (PDKW) in 1995 led to unrest among Turkish organizations; in 1997 a Turkish concert in Schiedam was disturbed by Kurds; in 1999, Kurdish actions on Dutch soil inspired Turkish organizations to position themselves publicly. Although the six Turkish claimants were more varied than the Kurdish claimants, it was a very selective part of the Turkish community; five were connected through a Turkish advisory organ to the Dutch government. They were actors with strong political convictions about the Kurdish conflict in Turkey, thereby attracting the attention of Dutch newspaper reporters and editors.

A combination of data and expertise is crucial to reveal what is not covered in newspapers (Altheide, 1987; Davenport, 2010). This is best seen in the scarcity of Surinamese claims. After re-coding, I found only one Surinamese claimant to be transnational in scope, implying that Surinamese were not involved in transnational activities. Other work shows that they in fact were, but in non-violent ways (Nell, 2008c, d; Mügge, 2011). Surinamese living in the Netherlands had easy access to homeland media to express their claims. In this case, the sample was skewed because Surinamese did not turn to the Dutch media to advance their transnational claims. Researchers thus need more information than what is generally presented in newspaper articles to validate their data.

Mixed Data Analysis: the case of political claims analysis and transnationalism

Is political claims-making, after re-coding and qualitative analysis, useful in explaining the driving forces of migrant transnationalism? Re-analysis of the Kurdish case shows that for political opportunities catering specifically to immigrants – so-called field-specific opportunities – Kurds in the Netherlands indeed felt excluded by structures catering to (ethnic) Turks. But qualitative analysis of the claims showed that ‘exclusion’ from field-specific opportunities was hardly the driving force behind Kurdish diaspora politics. Instead, it was the very inclusiveness of the Dutch legal system that allowed the Kurdish Parliament in Exile (PDKW) to be installed in the Netherlands, as the following example illustrates. The Kurdish organization KIC made a claim on the day the PDKW was inaugurated in The Hague. I had asked the KIC spokesman why the PDKW had been established and why they had chosen the Netherlands for its inaugural congress: ‘In most countries there was a problem with the laws…. we said the Netherlands was suitable…. According to Dutch law it was just a conference, organized by the KIC. In fact it was the installation of the Parliament in Exile.’⁷ The passage underlines the comparably favorable political climate for Kurdish nationalists in the Netherlands, contradicting Koopmans et al.’s (2005) conclusion that transnationalism is above all a reaction to limited political opportunities in the country of residence. The openness or closure of host-country general or field-specific political opportunities is hardly the sole factor driving transnational claims-making: events in the homeland clearly matter.

Koopmans et al. (2005, p. 142) second main conclusion is that transnational claims are generally violent because they mirror homeland action repertoires: strong homeland orientations among immigrants are detrimental to their integration in countries of residence. When recoding the claims, I found only 5 out of the 22 Turkish, 10 out of the 40 Kurdish, and none of the Surinamese transnational claims to be violent in nature. While Dutch field-specific opportunities did not support Kurdish homeland claims, this did not deter Kurds from making use of the general political opportunities provided in the Netherlands. Furthermore, transnational Kurdish actions did not mirror action repertoires in the homeland; they instead reflected non-violent action repertoires in the country of residence – a sign of political integration in and of itself.

While the re-coding and qualitative analysis of the MERCI sample could correct researcher and description bias, it could not address selection bias. The latter creates a skewed picture of the transnationalism-integration nexus since only specific forms of transnational politics are included in the newspaper sample. Violent instances of transplanted homeland politics (when homeland conflicts are transplanted to the host country) were over-represented. Selection bias explains why none of the claims could be classified as homeland-directed politics (when immigrants try to influence developments in the homeland) or transplanted immigrant politics (when skills and organizations are transplanted to the homeland). Such influence on Turkish or Surinamese politics was of scant interest to the NRC and its readers. The picture would look radically different were Surinamese media included in the sample: Surinamese immigrants in the Netherlands are highly visible in Surinamese newspapers, with some dailies dedicating a whole page to Dutch news.

Claims-making can be usefully applied to that part of the political debate that is visible in the newspapers that one studies, and has proven to be a powerful method for the study of transnational activities that may have direct consequences for the host country. However, the method does not allow drawing general conclusions about transnationalism and the impact (or lack thereof) of the sending state (Mügge, 2012b; Mügge, 2013c; Mügge, 2016). Koopmans et al. (2005) find host-country domestic politics to be driving transnationalism, simply because they ignore countries of origin in their analysis. In this sense, they, too, sample on the dependent variable – precisely their criticism of scholarship on transnationalism. Findings based on newspaper data are more specific and less generalizable than is often suggested.

Towards mixed methods: incorporating political ethnography in content analysis

If scholars using content analysis are receptive to mixed methods, they will benefit from including them from the outset – in developing their research designs, in collecting their data, and in their contextualization (Brannen, 2005, pp. 177–181). The MERCI study chose the content analysis of newspapers to facilitate cross-national comparison. The quantitative data set it generated lends itself well to Lieberman’s (2005) ‘nested analysis’ – a ‘pure mixed’ methods study that complements, confirms and provides direction to our next steps (initiation). Nested analysis ‘combines statistical analysis of large sample cases with the in-depth investigation of one or more cases contained within the large sample’ (Lieberman, 2005, pp. 435–436).

Nested analysis can be seen as a ‘pure mixed’ method as it ‘assumes interest in both the exploration of general relationships and explanations and the specific explanations of individual cases and groups of cases’ (Lieberman, 2005, p. 436). A ‘pure mixed’ study starts with preliminary or formal large-N content analysis to test how far results are robust and satisfactory in light of background information on specific cases (Lieberman, 2005, p. 438). Content analysis as pursued by the MERCI project suits such nested analysis, which requires access to a quantitative data set with sufficient observations to allow statistical analysis. This enables cross-national analyses to detect general patterns as well as the analysis of specific countries (Lieberman, 2005, p. 450). As internet search engines have become much more sophisticated since the collection of the MERCI data, complementing newspaper articles with other types of data has become easier.

Following Lieberman one proceeds from large-N analysis to ‘model-testing small-N analysis’ if the model is well specified and robust, and if not, to ‘model-building small-N analysis’. Either way, the researcher must reassess the findings to decide whether additional iterations of small or large-N analysis are needed (Lieberman, 2005, p. 436). Small-N analysis should also be used to assess the quality of the newspaper data and codebook. Intensive small-N analysis, of for example actors in a specific country, can answer questions left unanswered by content analysis ‘either because there were insufficient data to assess statistical relationships or because the nature of causal order could not be confidently inferred’ (Lieberman, 2005, p. 440). This means that following large-N analysis, the researcher evaluates whether the research question can be answered with newspaper data. Ethnographic analysis of a partial sample can help us find out.

If research goals cannot be met with the data, scholars need to adapt their aims or opt for alternative approaches. This exercise can be incorporated in a pilot study. If such a pilot study shows that the research aims are compatible with newspaper data, political ethnography can be employed in two complementary steps. First, validating political ethnography can corroborate data from other sources and help identify biases. Second, ethnography can reveal the limitations of newspaper data and how these limitations influence analysis. In large N studies, scholars can do this by re-examining extreme cases (e.g. a very high or low number of claims in a particular period). Such re-analysis should address a range of questions as proposed by Fillieule and Jiménez (2003): how systematic is the bias? Does the rationale for selection vary over time and contexts? If so, how? International comparative projects need to be particularly sensitive to whether selection bias follows comparable patterns across countries (cf. Koenig, 2006).

Validating political ethnography is guided by large N analysis. The newspaper guides the ethnography by pointing to actors and events. Ethnography lays bare what is invisible to the general public and journalists but may be crucial to the research. Validating ethnography also provides depth to recorded phenomena and helps to understand and interpret findings, even if it does not reveal what newspaper reporting misses.

The next step is to use explorative political ethnography and statistical analysis together. Depending on the topic and the selected cases, ethnography can include online or real-world participant observation, open interviews and archival research. In contrast to validating ethnography, explorative ethnography can go beyond the boundaries of newspaper reporting to establish relevant factors unreported in the newspapers. Although political ethnography is time-consuming, the inferential strength added through the analysis of a small number of cases is likely to be worth the costs. Moreover, the suggested distinctive approach to political ethnography within a mixed methods research design can be readily incorporated in comparative research projects.

Concerns in mixed methods

The previous sections have focused on the advantages of mixed-methods research for content analysis based on newspaper data, but scholars have also raised concerns about the practices of mixed-method research. Two concerns discussed by Small (2011, pp. 77–79) – commensurability and specialization – apply to the suggestions above. Commensurability derives from the epistemological perspectives that drive methodological choices. Some ethnographers may view the suggestion to combine ethnography with large-N analysis to be incompatible with the central idea of ethnography, since they argue that the best ethnographic research comes from the very absence of a research design (Becker, 2009). Integrated mixed-methods analysis will be challenging as agreement will be difficult if researchers in a team come from different traditions (Small, 2011, pp. 77–79). Scholars interested in cross-national conclusions may find detailed case studies to be incompatible with their research paradigms. Nevertheless, recent debates in mixed-methods research show that we have moved past the ‘paradigm wars’ between qualitative and quantitative purists (Johnson et al., 2007).

Regarding specialization, Small (2011, pp. 79) argues that it is difficult for mixed-methods researchers to remain up to date on a broad range of methodological developments. As methodologists work within increasingly specialized areas of expertise, it becomes more difficult to convince grant evaluators, journal reviewers and editors with strong beliefs about foundational issues to entertain the radically different alternatives that a mixed-methods project may involve. It will take time to train ethnographers for studies that integrate ethnography and content analysis and which require team-based ethnography. Ethnographers traditionally do not leave their field notes for other researchers to use and protocols; procedures for anonymizing and sharing ethnographic data are in their infancy (Small, 2011, p. 75). Establishing, training and monitoring an ethnographic team will require ethnographers to move beyond their individually oriented training to agree on common themes, coding schemes and writing topics (Small, 2011). Interdisciplinary fields such as migration and ethnic studies are good starting points for developing standards for team-based ethnography since they already make use of theme-based networks with researchers employing both quantitative and qualitative methods.

Conclusion

Newspapers provide researchers across the social sciences with a wealth of opportunities to compile large-N data sets for diachronic and cross-national comparative study. Facilitated by search engines and online databases such as LexisNexis, it is a relatively inexpensive tool for conducting systematic research. Despite these advantages, scholars have criticized newspaper data for their presumed sensitivity to selection, description and researcher bias. This article has shown how a political ethnographic approach can clarify data, provide depth to better understand events, and reveal what has not been reported in the newspapers. Taking the MERCI data as its case, it has shown how political ethnography can enrich the methodological toolkit of ‘event catalogs’ or claims analysis.

The added value of political ethnography reaches far beyond the realm of ethnic minority claims-making. First, ethnographic case studies can be used to validate newspaper data to ascertain whether there are biases in the data or not. When political ethnography is pursued as part of a ‘nested analysis’ guided by large N studies, it can provide insight into extreme cases. Second, political ethnography can be used exploratively in combination with statistical analysis. The strength of this method is that it can show what is missing from the newspaper data.

Designing and implementing research projects that integrate newspaper-based cross-national content analysis and team-based political ethnography will be challenging, time-consuming and costly. It will be a demanding journey in which scholars on both sides of the quantitative-qualitative divide will have to move towards the middle. But ultimately, the quality of the research should guide methodological choices, even if this requires reconsidering habitual and safe research paradigms.