2  Introduction

“Sure, it’s hard to get started; remember learning to use knife and fork? Dig in: you’ll never reach bottom. It’s not like it’s the end of the world–just the world as you think you know it.” (Dove, 1999)

“Listening to the world. Well, I did that, and I still do it. I still do it.” (Oliver & Tippett, 2015)

2.1 Quantitative Methods and Social Justice

There is clearly need for both qualitative and quantitative methods. Central to the argument of this document is the idea that advanced quantitative methods can be core contributors to the agenda of understanding issues of diversity and social justice more fully and thoroughly (Cokley & Awad, 2013; Grogan-Kaylor et al., 2018). Quantitative methods, particularly in discussions comparing qualitative and quantitative methodologies, are sometimes labelled as inherently positivist methods. My argument regarding this point is twofold. First, there is nothing within the mathematics of quantitative methods that requires a positivist epistemology. Quantitative methodologies could as easily be conducted using a critical epistemology–that is aware of dynamics of power and privilege–as any other methodology (Stage & Wells, 2014). I note that one of the pioneers of liberation psychology, Martin-Baró (Aron & Corne, 1994), used both qualitative and quantitative methods (Martin-Baro, 1994a), including in the latter case, relatively sophisticated arguments about patterns of missing data across a survey data set (Aron & Corne, 1994).

Second, when we have samples of a hundred, several hundred, several thousand, or even hundreds of thousands of study participants distributed across multiple and diverse social contexts, it is difficult to imagine a methodology other than a multilevel quantitative methodology that could accomplish the following:

  1. Sift through thousands of responses, and determine the overall, or average, pattern of relationships between risk factors, protective factors, and outcomes.
  2. Explore the diversity and variation in these relationships across social contexts.
  3. Determine whether there is evidence that the relationships observed within the data are more than statistical noise.
  4. Adjudicate the complex multivariate relationships of risk factors, protective factors and outcomes.

Therefore, I consider multilevel modeling to be a principled quantitative method for listening to the voices of large numbers of study participants across social contexts. In Section 6.1, where I consider the estimation of p values, and Section 6.2, where I consider the signs of regression coefficients, I explore the ways that multilevel data can contribute substantially to the complexity of the analysis of data. I thus argue that advanced quantitative methods, like multilevel modeling, can play an important role in contributing to liberatory ideas.

There is thus an ethical argument that is embedded in this document. Many of us do research with the hope of better understanding the relationship of risk and protective factors with outcomes in diverse, and often disadvantaged or marginalized, populations. Many of us further hope that our work might be part of conversations about appropriate polices, programs, treatments or interventions. Given the frequent vulnerability and marginalization of the people with whom we work, when using quantitative methods, it is incumbent upon us to employ methods that adequately address the complexities of the data, that offer an appreciation of the variability and diversity within the data, that provide the most accurate estimates possible, and that increase the probability of obtaining correct answers to important substantive questions.

“It is hard to imagine that anyone with a humanitarian worldview would argue against the need for a more quantitatively literate citizenry. Informed political decision-making, retirement planning, active parenting, and the vast majority of choices we make in our personal, occupational, and civic lives can be better served by improved quantitative understanding and reasoning, as well as accompanying action-oriented dispositions.” (Wiest et al., 2007)

The idea of this document is that a deeper study of multilevel modeling can result in an advanced “quantitative literacy” (Wiest et al., 2007), or “principled argument” (Abelson, 1995), that is appropriate for drawing accurate conclusions from multilevel data.

2.2 Some Philosophy of Science

I am not much of a philosopher of science. However, I am very persuaded by Strevens’ (2020) minimalist criterion of the “iron rule”. In essence, this rule specifies that to count as “science”, investigations must engage in “performing an experiment or making an observation that generates relevant empirical evidence” against which competing hypotheses can be tested. A similar perspective is offered by Goldacre (2011) who argues that ideas about interventions should be scrutinized with a “fair test”. That is to say, they should be tested against evidence that can support or refute those ideas. I would argue all ideas about promoting human well-being should be able to be subjected to such a “fair test”.

I believe that our work—whether qualitative, or quantitative—should strive to be both critical and scientific, in the sense that: our research should gather evidence; that evidence should be assessed in order to support, refute, or modify our initial beliefs; and that evidence should be used to think critically about human wellbeing, including dynamics of power and privilege and disparities. With regard to this idea, Shrader-Frechette (2014) suggests that a “practical philosophy of science” can contribute both to “speaking truth to power” and to “seeking justice”.

2.3 A Pragmatic Approach

This document will discuss the ways in which a multilevel statistical perspective not only allows one to appropriately analyze cross cultural or international data, but also the ways in which a multilevel perspective affords the opportunity for more precise quantitative thinking about cross cultural phenomena. The document takes a very pragmatic and very advocacy oriented approach to improving research.

“It shouldn’t be theories that define the problems of our situation, but rather the problems that demand, and so to speak, select, their own theorisation.” (Martin-Baro (1998) in Burton & Kagan (2005)).

“What we see and how we see is of course determined by our perspective, by the place from which we begin our examination of history; but it is determined also by reality itself.” (Martin-Baro, 1994b)

Following from this pragmatic and advocacy oriented emphasis, the document is largely oriented to the doing of quantitative social research with multilevel (or multi-country) data, and is therefore mostly statistical in nature.

The document moves quickly into detailed statistical arguments. Some of these statistical discussions may seem very technical, or even overly technical. However, an overarching theme of the document is that multilevel data contains hidden complexities. A lack of awareness of the complexities of multilevel data—e.g. complexities of multi-country data—might lead to statistical analyses that point in the wrong direction: yielding false positives; false negatives; or substantively wrong conclusions.

2.4 Are Answers from Social Science “Obvious”?

Closely related, I think to the the idea that quantitative research can advance issues of social justice, is the question of whether answers from social science are “obvious”. If social science answers are obvious, then social science has limited abilities to make new discoveries, and to build scientific foundations for evidence.

I have been thinking a lot about the idea that Everything Is Obvious, Once You Know The Answer, as detailed in the book with this title by Duncan Watts (2011).

This seems to me especially true in social research. Arguably, some conclusions of social research may indeed be obvious. For example, it may be obvious that Adverse Childhood Experiences (ACEs) are associated with long term decreases in mental health. However, even obvious conclusions may need to be quantitatively documented, in order to legitimate programs and interventions, and to secure funding. I also observe that I think that there is often a historical dimension to what is considered “obvious”: conclusions that are at first considered to be unlikely to be true, or even counter-intuitive, require the weight of accumulating evidence over time for these connections to become “obvious”. It is likely that the “obviousness” of the relationship between ACEs and later physical and mental health problems did not become apparent until research began to document these relationships (e.g. Felitti et al. (1998)).

As another example, Proctor (2012) documents the way which smoking was first considered to be an unlikely cause of lung cancer; only over the course of several decades of research and discussion to become an obvious cause of lung cancer. A similar historical dynamic seems to be playing out in some research on parenting and child development. Despite decades of evidence indicating that corporal punishment has undesirable conequeqences for children (Gershoff & Grogan-Kaylor, 2016b), corporal punishment remains a disciplinary strategy endorsed by the majority of the American population (Hines et al., 2022).

In contrast sometimes the conclusions of social research may not always be obvious. For example:

  1. There has been an ongoing debate about whether corporal punishment is more or less harmful when used by parents in social contexts, or communities where it is more common, or normative, or in contexts that are disadvantaged. Eamon (2001) suggested that “when environmental risk is high, parenting practices that are firmer and higher in control result in lower levels of young adolescent antisocial behavior.” This echoes similar research by (Deater-Deckard et al., 1996) suggesting that physical punishment was harmful for European-American children, but not for African-American children. Later, larger sample research has found that this appears not to be the case: physical punishment is harmful for children in all groups (Gershoff & Grogan-Kaylor, 2016b, 2016a; Pace et al., 2019).
  2. Using MICS Data (UNICEF, 2021), we conducted a study of the link between gender inequality and physical child abuse (Ma et al., 2022). We expected to find that higher levels of gender inequality led to higher levels of physical abuse for female children, but not for male children. Instead, we found that higher levels of gender inequality were associated with higher levels of physical abuse for both male and female children. Additionally, there was some slight evidence that male children were at higher risk of being abused than female children. Equally interesting was that we found that gender inequality was predictive of levels of child abuse, while country level GDP was not.
  3. In a study of parenting during Covid-19 (Lee et al., 2022), we expected to find that households with children would experience higher levels of anxiety and depression than households without children. Instead, we found the opposite. Being in a household with children was generally protective against anxiety and depression.

In Section 5.3, Section 6.1 and Section 6.2, I provide specific examples of how multilevel data provides even more opportunity to present answers that are not obvious.

2.5 Presenting Advanced Statistical Ideas

In presenting advanced, statistical concepts, one is faced with a quandary. One can present statistical concepts in the most general terms, in terms of x and y. While perhaps the mathematically most general way to present ideas, a highly general (and abstract) presentation risks not being a good way of teaching the ideas, as it is sometimes difficult to apply abstract ideas to one’s own specific area of research.

Alternatively, one can present statistical ideas in terms of specific substantive concepts. The risk of making use of a specific substantive concept is that while concrete examples are always helpful, it may be difficult for the reader to generalize from a specific example to their own area of research.

I ground this presentation in research that we have conducted on parenting and child development in international context (Grogan-Kaylor et al., 2021; Ma et al., 2022; Pace et al., 2019; Ward, Grogan-Kaylor, Pace, et al., 2021; Ward, Grogan-Kaylor, Ma, et al., 2021; Ward et al., 2022). For the presentation in this document, I use simulated data on these issues.

Using the simulated data, I refer to predictors and outcomes, and explore the ways that the multilevel model can contribute to understanding how relationships between predictors and outcomes might be similar, or might be different, across social contexts. In the examples presented below, I focus on two predictors, parental warmth, and parental use of physical punishment and focus on the outcome of improved mental health. I use the social context of different countries in our example.

It is my belief that while I use this specific set of examples, that the idea of studying families in different countries is generalizable enough to a multiplicity of diverse contexts, such that the reader can apply these ideas to their own area of interest, whether that be children in schools; residents in neighborhoods; or people in different countries.

2.6 Research on Parenting and Child Development in International Context

Research on parenting and child development has identified robust associations between parenting behaviors and child developmental outcomes. Broadly speaking, physical punishment is associated with increases in child aggression, child anxiety and child mental health problems (Gershoff & Grogan-Kaylor, 2016b), while warm and supportive parenting is associated with decreases in these outcomes (Khaleque & Rohner, 2002; Rothenberg et al., 2022). However, much of this research is conducted on North American samples (Draper et al., 2022; Henrich et al., 2010).

Barth & Olsen (2020) have argued, that children constitute a class of oppressed persons. If children are oppressed, then it is imperative to empirically determine what factors are promotive of children’s well-being, and what factors constitute risk factors that contribute to decreases in children’s well-being. Equally imperative–given the North American focus of so much research on parenting and child development (Draper et al., 2022; Henrich et al., 2010)–would be efforts to extend the study of parenting and child development to a broader, more global context. As part of such a research agenda, it is necessary to have quantitative tools that are able to determine the consistency of relationships in parenting and child development. That is, are the relationships between certain forms of parenting and child developmental outcomes, largely consistent across countries, largely different across countries, or somewhere in between?

2.7 Universalism And Particularity

“My conception of the universal is that of a universal enriched by all that is particular, a universal enriched by every particular: the deepening and coexistence of all particulars.” (Cesaire, 1956)

The specific domain of cross-cultural research on parenting and child development raises more general questions in cross-cultural research of universalism and particularity. With regard to child development it is universal that all children need some amount of emotional and material care to grow into healthy youth and healthy adults (Kottak, 2021). Further it is broadly understood that children should be protected from violence (UNICEF, 2014). This broad consensus is manifested in such documents as the Convention on the Rights of the Child (United Nations General Assembly, 1989) and the United Nations Sustainable Development Goals (United Nations, 2022), representing global efforts to ensure the children are cared for, and are protected against violence.

At the same time, broad international efforts to improve children’s well-being must engage with important considerations of cultural uniqueness. Put simply, parenting practices may vary widely between cultural groups (Gottlieb, 2002). Further, what is considered to be beneficial for children in one country or culture may not be considered to be beneficial in all countries or cultures. Similarly, what is considered to be detrimental in one country or culture may not equally be considered to be detrimental in all. Within the area of parenting and child development, most of the debate has focused around the question of whether physical punishment is equally detrimental in all settings, particularly whether physical punishment is detrimental in countries where it is especially common, or normative (Gershoff et al., 2010). Much less attention has been focused on the study of positive parenting internationally, and the degree to which the outcomes of positive parenting are consistent across countries remains understudied (Ward, Grogan-Kaylor, Ma, et al., 2021).

However, as global initiatives to improve child well-being and family life move forward, it becomes increasingly important to continue to collect internationally relevant data about parenting and child outcomes. If recommendations are to be made for policies, interventions, or treatments, such recommendations must be based on accurate balancing of that which is universal against that which is unique to particular cultural contexts. Thus it is necessary to employ statistical methods that are able to adequately and accurately analyze data across countries.

As I will outline below–and is evident in the literature (Kreft & de Leeuw, 1998; Luke, 2004; Rabe-Hesketh & Skrondal, 2022; Raudenbush & Bryk, 2002; Singer & Willett, 2003)–multilevel models are eminently suited for cross-cultural research in that they are not only able to control for the clustering of study participants within countries, but are also able to explore the variation–or consistency–of patterns of social life across countries.