Inglês (pdf)
Artigo em XML
Referências do artigo
Tradução automática
Permalink
As an African American social justice advocate and professional living in the United States (U.S.), most of my scholarship focuses on the disruption of white supremacist and racist logics in the U.S. And, without question, racism operates as the foundation of nearly all our systems and structures. Still, the U.S. does not hold a monopoly on the importation and exportation of racist oppression. For example, according to the United Nations Department of Public Information (Gorelick, nd), Indigenous peoples in Canada are over-represented in the prison system accounting for only five percent of the population, but approximately a third of adults in provincial/territorial and federal custody (Statistics Canada, 2022). Indeed, due to racist and/or white supremacist logics, Indigenous offenders are more likely to receive imprisonment sentences when convicted of a crime than non-Indigenous persons (Dylan, Regehr & Alaggia, 2008); and Indigenous victims of crime are often perceived to be less credible than non-Indigenous people (McGlade, 2010). In fact, outside of the prison system (e.g., stores, banks, restaurants), both Indigenous (33%) and Black (46%) persons report experiencing higher levels of discrimination than their non-minoritized (16%) counterparts (Cotter, 2022).
We can also turn to the Roma, in Europe, as an example of the racialization then marginalization of an entire people. The European Roma population is estimated to be between 10 and 12 million (European Commission, 2020), and they are the most economically disadvantaged in the area, often living on the fringes of society. Amnesty International (2007) reports that they face extensive discrimination in accessing housing, employment, and education. In certain nations, they are barred from obtaining citizenship and the necessary personal documents for social insurance, healthcare, and other benefits. Additionally, they frequently endure police mistreatment, and their grievances are rarely addressed (not unlike the experiences of Indigenous citizens in Canada). Romani children are often inappropriately placed in "special" schools with limited curricula, hindering their potential; and both Romani children and women are among the groups most susceptible to trafficking.
Finally, if we consider the nation of Brazil, we find evidence from multiple sources that systemic racism against Black Brazilians manifests in nearly every meaningful aspect of social life: education, employment, housing, health care, and the criminal justice system. Black Brazilians often have less access to quality educational institutions as public schools (which predominantly serve Black students) are typically underfunded and lack resources compared to private schools, have higher rates of illiteracy, and are outnumbered in universities by a ratio of 4:1 by white Brazilians (Instituto Brasileiro de Geografica e Estastistica [IBGE], 2019; Paixao & Carvano, 2008). On average, Black Brazilians earn less than their white counterparts and are more likely to work in informal, low-wage jobs with little to no job security or benefits (IBGE, 2020). Black Brazilians often face discrimination when trying to rent or buy homes, being steered away from certain neighborhoods or given less favorable terms (Mitchell-Walthour, 2017). And Black Brazilians do not fare better with respect to health systems, as they have lower primary health care usage rates (Medeiros et al., 2023), experience higher rates of chronic diseases and lower life expectancies (Chor, 2005; Santana et al., 2007; Barata et al., 2007; Faerstein et al., 2004), and more likely to die from mental disorders (Medeiros et al., 2023) than white Brazilians.
Similarly to Indigenous populations in Canada, Black Brazilians, especially Black women (Eschberger, 2022), are overrepresented in the prison population. In fact, the incarceration rates for Black inmates in Brazil have reached historic highs (Anadolu Agency, 2023). Moreover, in Brazil, the Black people have a higher probability of being the victim of homicide (129%) and physical assault (39%) than White people (Truzzi et al., 2022). Moreover, Black victims of violence are less likely to report their victimization to police for fear of the police or that would not be believed (IBGE, 2010). In fact, in 2023 Truzzi and colleagues found that “compared to Whites, Blacks had a lower chance of assessing justice in Brazil; this difference is as substantial as their greater probability of suffering criminal violence, which is well known’ (p.12). Indeed, as Moura (2023) writes, with respect to Brazil, “Black people display the worst indicators for employment, income, education, and political participation compared to their white counterparts…” (para. 1).
My point is simple: racism is not an aberration. Indeed, it is a norm that we can neither escape nor separate ourselves from based on where we live in the world. Perhaps, anti-Black racism, specifically, is not a problem outside of the United States or Brazil; and, if that is the case, it is probably because there is not a large enough proportion of Black people in that locale to disenfranchise; so those systems and institutions have turned to the racialization and disenfranchisement of another peoples (e.g., Roma). Because no matter where you are in the world, people are racialized then marginalized. And that is the context in which we must do our work in assessment and measurement. In a 2015 survey, Kteily et al. found that on a scale of 1-100, U.S. Americans rated other Americans and Europeans as being highly evolved (average score in the 90s) and rated Arabs and Muslims in the 70s and low 80s. In fact, about 25% of participants rated Muslims with a score of 60 or below. This is the world in which we must do our work - a world in which Americans quite literally, rated people who are not membered white as considerably less human. And try as we might as a fieldespecially because so many of us work in education and psychology (spaces considered altruistic by nature) - to distance ourselves from the rest of the world, Dixon-Roman (2020) reminds us of our ghosts of measurement pasts and how those ghosts still haunt our profession:
“…even with the sociocultural and postmodern turns in educational assessment and measurement, there remains a haunting logic in the epistemology of psychometrics that maintains colonialist formations of reason (i.e., mental processes or embodied practices) and the subject (i.e., the test-taker or assessed). Ultimately, this haunting is based on a conceptualization of the human in the post-Enlightenment, a conceptualization that rendered black and brown bodies as primitive, partially human, or nonhuman suggesting that black and brown bodies do not matter” (p.94).
The History of Dehumanization by Assessment: The Mess We Made
What we saw at the turn of the 20th century was the de-evolution of an assessment system meant to inform instructional practices (i.e., Binet’s initial assessments for students with special needs) to a large-scale, pseudo-scientific process used to reify racist (IQ tests) and xenophobic stereotypes and fears (e.g., Yerkes’ U.S. Army and Beta tests) effectively further marginalizing already marginalized populations (see Ross, 2014; Williams, 1971 for examples). Put more plainly, what we have witnessed is the use of assessments, or assessment results, to justify the dehumanization of entire populations of people. Indeed, in the United States alone, tests have been used to (a) restrict reproductive rights; (b) promote xenophobic decision making and policies; (c) disenfranchise all Black and some immigrant citizens (Ross, 2014); and (d) promote racialized agendas and deficit narratives (see Sireci & Randall, 2022 for a more detailed history). And, again, the U.S. and most of Europe have managed to extend our use of assessments as tools of oppression to the rest of the world as well, especially in low-income countries today.
U.S. (e.g., USAID) and European (e.g., DFID) organizations show up with our standardized assessments in an effort to measure student progress after we inject some amount of funding into their educational systems, because these federal agencies demand objective evidence of their success. And success is routinely (even if not explicitly) defined as the ability of the funded program to turn Black and Brown people into white people; or put another way, the program’s ability to strip an entire population of all its cultural ethos and replace it with an ethos based in the supremacy of whiteness. Strangers, often referred to as enumerators, administer these tests relying on assessment practices that are often unfamiliar and (sometimes) ridiculous to the students receiving the assessments. For example, students who are accustomed to demonstrating their understanding through choral responses are, instead, expected to sit quietly alone and write their responses. In many other cases, students are expected to demonstrate what they know by writing/speaking in French or English when the language used primarily in their homes and schools is Swahili or Lingala, for example.
Thus, just as so-called IQ/Intelligence tests around the world perpetuated and reinforced racial and national stereotypes with prejudicial and unsubstantiated results in the past, so do assessments like the Early Grades Reading Assessment (EGRA) or Early Grades Mathematics Assessment (EGMA) (RT1, 2016, 2014), which are used nearly exclusively within the context of international development. They rely on U.S. American and European cultural ways of understanding and knowing as normal or superior with little regard for the varied ways of knowing, understanding, and doing anywhere else in the world.
All this to say “It’s a Mess.” A mess has been made.
Indeed, what we have created - rooted in the dehumanization of bodies that are not membered white and racist logics is a mess. It is a mess that hate built. But our profession can begin the process of fixing this particular mess across all of our systems of assessment. And this commentary provides suggestions for how we can actively clean up this mess. A former president of the National Council on Measurement in Education wrote:
“ …the goal of educational measurement is not to measure the students who are the easiest to measure and who conform to the most dominant culture associated with the measurement enterprise, but rather to obtain the best measure of each and every student’s proficiencies” (Sireci, 2020).
Justice Defined
We fix this mess by questioning the critical structures and assumptions that make up all our judgments and behavior with respect to assessment (Inoue, 2015). And this shift in thinking will require us to let go of theories of justice such as those proposed by Nozick (1974) who argued that moral principles appropriately apply only to individual behaviors and interactions. He argued that although individuals must not engage in deceit or coercion in their dealings with other people, even a bad result from a transaction that includes no intentional deceit, or coercion, is a moral result. We have to move away from such a selfish theory of justice that suggests that the outcomes are immaterial as long as individual inputs and intentions are not egregiously evil.
We must accept that the feelings in our individual hearts are not what is most important. It does not matter that you are a good person and that you wish Black and Brown children no harm. Focusing on our individual behaviors will never get us to justice. We can no longer tell ourselves that “I have developed an assessment system with no malice in my heart, that follows all of the rules and is not intended to marginalize anyoneso I have done the just thing.” We must, instead, move towards a theory of justice that acknowledges that we (a) cannot ignore the history of systemic oppression (Mills, 2017); (b) must refrain from making arguments about equal opportunity without taking into consideration that all are not beginning at the same starting place (Rawls, 1971; 1999), and (c) must understand that if an advantage must be given, in a just system, it should be given to the least advantaged members of society (Rawls, 1971;1999).
Debunking the Myth of Meritocracy
If we rely on the theories of justice articulated by Rawls (1971;1999) and Mills (2017) then the very notion of merit, or meritocracy, seem ridiculous. Crenshaw (Crenshaw et al., 1995) wrote: “Our critiques of racial power reveal how certain conceptions of merit function not as a neutral basis for distributing resources and opportunity, but rather as a repository of hidden, race-specific preference for those who have the power to determine the meaning and consequences of ‘merit’” (p.xxix). This notion of meritocracy allows people to think that they got into college, or to graduate school, or a professional school based on merit alone; and that those who did not do those things are simply less meritorious. Indeed, large scale assessments contribute to this tyranny of merit. In fact, one might argue that the tyranny relies on active employment of admissions assessments in particular - so assumed, objective gatekeepers. These assessments allow people to say, for example, that “I received a high score of 10; and you received a lower score of 8, so I am more deserving of this opportunity.” These isolated scores leave no room for people to acknowledge that they also received a long list of privileges, or advantages, and that large-scale (particularly admissions) assessments measure, to some extent, their access to all of those privileges - thus the higher score.
I acknowledge that assessment companies are unlikely (or unable) to intervene with respect to the uneven starting points that plague marginalized populations articulated by both Rawls (1971;1999) and Mills (2017), but they can disrupt, or rupture, the fantastical, whitesupremacist narrative that assessments are pure measures of meritocracy. And as an assessment professional, I know that profit margins (and jobs) rely on people believing this myth of meritocracy; but I am also painfully aware that this myth allows people to develop a false narrative (devoid of any reference to the historic and ongoing dynamics of settler colonialism, capitalism, imperialism, and structural racism) and make racist decisions without consequence or feeling guilty. I am not arguing that most large-scale assessment systems do not measure something related to their intended construct (e.g., math, reading, verbal reasoning). I am arguing, however, that if we had some measure for privilege, the association between that measure and the intended construct would be remarkable. The ways in which assessments have historically and consistently served as a barrier to access has been long documented (see Sireci & Randall, 2022 for examples); and in most cases, these oppressive practices have been cloaked inside a narrative of meritocracy. As Viera (2018) writes: “When we accept the myth that these tests are merit-based, we also accept the idea that race and class gaps in standardized-test results, which have remained essentially unchanged over the last 20 years, are due to individual and group shortcomings, not structural ones” (para 2). My point is that meritocracy is a myth - and it is tyrannical.
Assessment Justice
If we are to fix this mess, we must commit ourselves to an approach to justice that recognizes/acknowledges the full historical and contemporary context in which assessment professionals work:
A justice-oriented approach to assessment design and development: (a) acknowledges the historical structures of oppression (such as racism, sexism, and colonialism) deeply embedded within our current assessment processes; (b) actively seeks to understand their ongoing consequences on marginalized populations; and (c) intentionally seeks to disrupt these negative processes and outcomes by centering the needs of these populations. Justice-oriented approaches to assessment do not seek to serve the “greater good” of the many and powerful to the exclusion of the few and minoritized, but rather assertively prioritizes the most marginalized populations. A commitment to justice goes further than a commitment to equity, as equity-driven approaches to assessment seek merely to provide scaffolds that compensate for historical and contemporary barriers. Justice-oriented approaches, on the other hand, actively seek to remove those barriers and also make amends for the damage those barriers have already created (Randall, et al., 2024, p. 206).
More simply put, assessment justice repairs harm. It fixes the mess.
Fixing the Mess (Correcting Harm)
In this section I briefly describe a series of corrective actions that move us closer to a justice-oriented approach to assessment and measurement. No one organization, or individual, has the capacity and/or power to enact each of these actions, but - as a professional community - we can commit to repairing the decades of harm inflicted upon racially and ethnically minoritized populations around the globe.
Take Responsibility
The first step to fixing this mess is to make the decision that we are all, in fact, responsible. The Social Connection Model of Responsibility finds that “all those who contribute by their actions to structural processes with some unjust outcomes share responsibility for the injustice…Being responsible in relation to structural injustice means that one has an obligation to join with others who share that responsibility in order to transform the structural processes to make their outcomes less unjust” (Young, 2011, p. 96). No one person is (or one small group of people are) to blame, so we must all work together in collective action and take responsibility.
Establishing a Monitoring and Evaluation System
Drawing inspiration from South Africa’s postapartheid truth and reconciliation process, I have noted elsewhere (Randall, 2024) that an important step in fixing the mess involves actively listening to and engaging in dialogue with marginalized populations. This process would include documenting and analyzing the modern history of educational and psychological measurement to understand the impact the measurement community has had on educational policy, both positively and negatively, for marginalized populations. Moreover, it would necessitate funding historians of science, educational policy, and the social sciences to carefully investigate how educational and psychological measures have contributed to the marginalization, erasure, and dehumanization of minoritized persons.
The objective of this truth and reconciliation process is to uncover previously unknown or unacknowledged harms and to enable the affected communities to decide on the appropriate actions to address these harms (or the mess that has been made). Prioritizing empathy and care, the aim is to establish restorative action plans. As part of this initiative, the measurement community must work together with harmed populations to develop an ongoing monitoring and evaluation system. This system would assess how their work either advances or impedes social justice objectives. In fact, each recommendation that I make in this commentary would also be evaluated through this collaborative monitoring and evaluation system. In summary, this truth and reconciliation process would empower the measurement community’s shift away from a large state-of-not knowing (Emi Iwatani, personal communication, May 15, 2021) into a deep understanding of the harm it has facilitated, so that we can work diligently to undo that harm and ensure it never happens again.
Investment in the Development of the Critical Consciousness of Entire Field of Assessment
Freire (1973) defined critical consciousness as “the ability to recognize oppressive social forces shaping society and to take action against them.” And this articulation of critical consciousness is important because it will help us change the language of assessment. Specifically, we need to change the language that says our assessments are objective, because there is no such thing as objective; or change the language that says assessments are simply a measure of merit, because merit is a tyrannical myth. It is the field’s lack of critical consciousness that leads to its vice-like grip on language like “we are only highlighting the problem”: deficit-oriented language that implies that the problem is lazy teachers, ignorant parents, and students who are too stupid to know what is good for them. An elevated critical consciousness would allow us to see that we are a part ofand allow the maintenance of - oppressive social forces. It would allow us to see how, as assessment professionals, we are helping to maintain colonial, imperialistic, racist, capitalist systems of oppression.
An elevation in critical consciousness would help us move past our individualist perspectives on justice - those individualist perspectives that compel us to rely on one-off examples of how the results on a standardized test led to an opportunity for us, or a single friend/cousin. An elevated critical consciousness would help us to see that far more data (Ansell, 2011; Au, 2008; Hilliard, 1976, 1992; Strauss, 2021; Williams, 1971) support the narrative that large-scale standardized tests are barriers (not doors or windows) for marginalized peoples. For example, Roberts et al. (2021) found that the GRE, a popular graduate school admissions exam for U.S. universities, is more strongly correlated with race, gender, and socioeconomic status than with predictions of later academic success. An elevated critical consciousness would also implore us to stop looking for reasons to justify the marginalization of entire populations of people. We would focus our efforts and resources on creating a better system - more just - rather than focusing our resources on legitimizing the existence of the current unjust system.
Transparency
Although transparency is not sufficient in the pursuit of justice, it is, indeed, required. Many large-scale assessments (especially those employed to make admissions decisions) rely heavily on a cloak of secrecy with respect to the construct and specific content to be assessed. The test preparation industry, however, devotes considerable time and effort to uncovering the test blueprint. And they sell this information via their test preparation courses and materials to the tune of billion dollars a year (Byrne, 2020) in the United States alone. The industry, however, is ever extending across North America, Europe, Asia-Pacific, South America and the Middle East. Indeed, potential test-takers can spend hundreds, if not thousands of dollars/euros/reals to improve their scores on large-scale assessments (Wellemeyer, 2019). And this industry can charge these rates because they are successful - test-takers' scores do go up when they attend these preparatory sessions. In a recent study (Massar, 2020) of the impact of GMAT Test Prep, researchers found that taking a preparatory class increased scores by an average of 93.7 points. The problem, however, should be an obvious one: some populations will always have access to the secret sauce, whereas others (i.e., Black & Brown) will also be relegated to the margins of unknowing.
As Byrne (2020) writes, when referring to the benefits of test preparation courses: “It’s another sign that the competition to get into a great school is not exactly fair because people who have the money to get the best test prep are likely to have an advantage over those who can’t afford thousands of dollars for a test prep firm” (para. 10). I will write this plainly so that my position is clear: If an assessment organization does not make the test blueprint/table of specifications readily and openly available to all possible rightsholders and make aggressive efforts to minimize the monetization of this information, then the assessment system is inherently and irrevocably unjust.
Without question, transparency with respect to access to the content is important. Equally important, however, is transparency with respect to what kind of content is valued. Scholars have written elsewhere (Cushman, 2016; Randall, 2021) that mere transparency cannot correct for implicit biases baked into the assessment. As Cushman (2016) writes we often operate “as though transparency can correct for bias when it basically and simply says to those created by the colonial difference: ‘Let me show you how the making of this assessment will exclude you using our exclusive and exclusionary process of building this test’” (Validity & Fairness section, para 1). In other words, transparently articulating a construct that relies on white supremacist, racist, and/or colonial logics built on hate does not get us closer to justice. As Inoue (2015) writes, we must remember “that the construct that these scores allegedly measure are created by the tests and do not actually exist before those tests” (Inoue, 2015, p.26). Simply, we design assessment to operationalize constructs, which often reflect confirmatory biases (e.g., white supremacist logics) that do not allow us “to see outside of its own content and imperialist tenets” (Cushman, 2016; Validity & Fairness section, para 1). I encourage the field of assessment to imagine a transparent construct articulation process from a justice-orientation, that asks: “Does the test adequately get at the different ways of knowing and thinking and doing that Black stakeholders possess and value?” (Randall, 2021, p.85).
Respect, Value, Affirm and Seek to Sustain the Cultural Norms of the World’s Majority
Measures are administered all over the world in low-income countries, but especially in African nations and to peoples from the African diaspora, that fail to account for the interrelated dimensions of African, or Black, culture or the ways in which these cultural norms differ from European or U.S. American norms. For example, the West African ethos values movement which is an emphasis on the interweaving of movement, rhythm, percussiveness, music, and dance which are taken as central to psychological health; and one could reasonably argue that a standardized testing situation in which students are required to sit near perfectly still for hours would be in opposition with that norm. The African ethos also relies on an oral tradition which is a preference for oral/aural modes of communication in which both speaking and listening are treated as performance and in which oral virtuosity is emphasized and cultivated; so relying primarily on measures that require students to write (or select answers from someone else’s writing) might be in direct conflict with this ethos. The point I am attempting to make here is that we currently build measures-and make inferences derived from the results of those measures-based on white supremacist logics. We assume the qualities that are valued by whiteness are objective or correct and that test-takers must/should adjust accordingly.
A path forward - that respects the values and norms of the world’s majority - will require us to actively engage with these populations in a human-centered approach to assessment design and development. Members of minoritized populations can, and should be, centered and deeply involved in all stages of the process, from articulating the construct to score reporting and interpretation. For example, by setting up rights-holders accountability councils, we can codesign assessment experiences with marginalized populations (the rights-holders) in ways that represent their values and the knowledge and skills that matter in their contexts. These standing councils would be tasked with providing meaningful direction concerning intended outcomes as well as the measures used to evaluate them.
Seek Evidence of Justice
In educational and psychological measurement, so much of our focus has become trying to avoid the inclusion of egregiously racist, or egregiously classist content primarily using bias and sensitivity panels. Instead, I am suggesting that we shift our focus to identifying content that is unapologetically justice oriented. We can create assessment systems that identify all the many things students know (or don’t know) and understand, and that also operate as an opportunity to learn and enact change. It is not enough to set up a bias and sensitivity panel to make sure that items are not insensitive or biased (whatever bias means). We have been relying heavily on a bias and sensitivity review process for decades and the outcomes for our racially and ethnically minoritized students have remained unchanged.
It is not enough for item writers to craft items that do not imply that people living in poverty have too many babies and are terrible parents, they must show examples of people living in poverty, working hard, and engaging as amazing parents. It is not enough to avoid referring to refugees as cockroaches or menaces, but rather we must employ the reader to see them as resilient humans worthy of empathy and care. We can ask ourselves in the design and development process: (a) Have we disrupted a stereotype; (b) Have we told the full story without trying to elevate or protect whiteness; (c) Have we called out sociopolitical injustice; (d) Have we meaningfully included members from marginalized populations in the process of developing task ideas; and (e) Have we worked with members of marginalized populations to define the construct in a way that resonates with them? Put simply, we need to actively look for evidence of justice - not simply assure ourselves that there is no obvious injustice.
Move with Love
Finally, this commentary ends with the most important action of all. As a young Black woman from the American south, I grew up in the Christian church - so if a bible story exists, I know it. The moral of so many of these stories is to move with love (e.g., So now faith, hope, and love abide, these three; but the greatest of these is love, 1 Corinthians 13:13). And the story of love is not a story owned by the Christian faith. Love serves as the core to nearly every major religion - Hinduism (e.g., Hatred does not cease by hatred, but only love; this is the external rule, Siddhartha Gautama), Islam (e.g. Never will you attain the good reward until you spend from that which you love, Quran 3:92), and Judaism (e.g., Hatred stirs up strife, but love covers all offenses, Proverbs 10:12) - all profess the importance of love. And this is what I know about love. It does not make excuses. It does not skirt responsibility. Love sees hunger and it provides food. Love sees illness and it heals. Love sees harm and it corrects it. Loves sees a mess and it fixes it. Justice is love.
We must stop making excuses. We must stop pretending like we do not see the harm; and begin making decisions with the sole intent of correcting that harm - whatever those decisions are. So, I end by begging the readers to commit to repairing the house that hate built with love.
