Are global learning metrics the answer?

ASU’s Center for Advanced Studies in Global Education, a part of Mary Lou Fulton Teachers College, organized and hosted a symposium aimed at answering the many questions surrounding global learning metrics as a central component of the post-2015 United Nations’ Sustainable Development Goals.

The symposium, “Innovations in Global Learning Metrics: a focused debate among users, producers and researchers,” was organized by Iveta Silova, professor and director of CASGE and

Gustavo Fischman, professor at MLFTC. Its main goal, Silova says, “was to initiate a focused debate about global learning metrics to make them more culturally responsive, pedagogically innovative and contextually relevant for national education stakeholders, while contributing to a deeper understanding of education and sustainability at a global level.”

This year’s event addressed not only the issues of reliability, validity and relevance of international comparisons for education policymaking, but also the urgent task of understanding educational indicators in their complex relationship with political, social, cultural, economic, health and environmental dimensions.

Fischman added, “Two other important objectives of this symposium were, first to offer a space and create an opportunity for dialogue, reflection and planning with a group of key stakeholders who represent diverse perspectives among test developers, users, researchers and scholars. Second, to explore the challenges and affordances that spaces such as ASU’s Decision Theater offer to engage in collaborative computing and display technologies for data visualization to more comprehensively understand GLMs.”

In attendance were many education heavyweights. Aaron Benavot, professor at University at Albany-State University of New York and a former director of UNESCO’s Global Education Monitoring Report was one of them. He says the “old global education policy” focused on a narrow set of issues, such as “making sure there is universal access, all kids are in school, they complete a full cycle, they go on to secondary education, that people become literate, and that was it.”

Now, Benavot says, the new agenda focuses on: Are kids learning? Are they acquiring foundational skills? Is education contributing to sustainability? “So it’s not just getting to school. It’s also what’s happening while students are there,” he says.

New agenda, new issues at stake

Along with a new focus, the agenda aims to place every student on a single metric. It’s a contested idea, Benavot says. “If you’re talking about getting everyone in school, there is, for the most part, a universal consensus. But when it comes to learning even something as foundational as basic reading or mathematics, it’s not clear that everyone agrees what that means, especially when we have different people who speak different languages that are assessed. It’s not clear if that’s the only thing you should be measuring and what might be some of the unintended consequences.”

Benavot raises issues that may be at stake and why the process of developing GLMs should not be hastened. “It’s important for people to be informed. They should think more clearly if this is the direction they want to take. When some countries see themselves ranked on these metrics I’m sure they’ll take umbrage.” Benavot says he hopes the critiques continue. “There need to be different views that are articulated, including letting countries voice their own views about the merit in this.”

Assessing student learning for purposes of helping to improve within a country is great, says Benavot. “But you don’t need to compare yourselves to all of the other countries in the world in order to gain insight.”

Jeremy Rappleye, associate professor at Kyoto University, adds that placing all countries on a single metric “becomes very problematic and something we should challenge.” He asks, “Are GLMs serving the right purpose? And if so, whose purpose are they actually serving?”

Radhika Gorur, senior lecturer in education at Deakin University in Melbourne, Australia and a co-director of the Laboratory of International Assessment Studies, observes that some of the stakes are political, noting that GLMs are being used for purposes such as negotiating funding arrangements and as fodder for election campaigns. “I cannot see how they could usefully inform pedagogy — very different and specific and local data are needed for this,” she says. Global learning metrics have limited value even as information for national policy, says Gorur. “Even in nations like Australia, PISA rankings and the average scores on which these rankings are based have led to the ignoring of important within-nation differences and have resulted in costly and needless reforms,” she says.

Gorur is quick to point out that she is not “against data.” But she believes global comparisons are a costly distraction. “They make invisible all the structural, systemic, specific and local factors that affect learning outcomes in particular communities, and this means that these factors disappear from the conversation when seeking solutions and improvements,” says Gorur.

Zooming out: What’s really at stake?

Beyond the attainment of minimum levels of literacy and numeracy, the debate about global learning metrics flags larger issues. By focusing global attention on the technical aspects of measuring literacy and numeracy, other purposes of education become displaced.

Symposium participants voiced the concern that such an approach to education may threaten environmental sustainability of the planet and interfere with the implementation of sustainable development goals.

Rappleye and Hikaru Komatsu, also from Kyoto University, note that it’s critical to “understand interactions (including negative ones) among different Sustainable Development Goals, including the ones on education, economic development and environmental sustainability among others.” They argue that broader access to quality education alone would not automatically translate into higher environmental awareness and behavior change. Komatsu and Rappleye take the issue further, saying we need a fundamental reconceptualization of the education paradigm itself, in order to shift the discussion from knowledge and skills (curriculum), as well as access and equity (structure), to a more fundamental reevaluation of self and others including nature (culture).

Komatsu and Rappleye traveled from Japan for the symposium because they say currently, there is virtually no discussion of climate change or any environmental issues in the area of global learning metrics, and the symposium opened much-needed space for this important dialogue.

We need to “zoom out in order to place the discussion about global learning metrics in the broader context,” says Tinde Kovács Cerović, a former state secretary of education in Serbia and a current education board member of the Open Society Foundations. It is important to start a serious conversation about the question, “Literacy and numeracy for what?” she says.

Read about all of the participants who attended the symposium and find out more about what CASGE is doing.