Advancing equity, diversity and inclusion approaches for evaluation professionals

The challenge

Diversity, equity and inclusion in STEM education and workforce development are critical for U.S. economic vitality. Yet women and people of color drop out of pathways to STEM degrees and professions at higher rates than their male white counterparts. A plethora of education-based projects are underway to address these discrepancies, and evaluation professionals have a key role in ensuring these approaches are integrated and measured appropriately.  EvaluATE  — which is based at The Evaluation Center at Western Michigan University —  is the evaluation hub for the National Science Foundation’s Advanced Technological Education (ATE) program. They educate the ATE community, which includes evaluators, project leaders, staff, grant specialists and college administrators, about evaluation. However, more work is needed to better operationalize and measure the effectiveness of equity, diversity and inclusion as part of educational program design.

The approach

The EvaluATE project is led by Lyssa Becho, a principal research associate at the Evaluation Center at Western Michigan University, and Ayesha Boyce, who began this work while at the University of North Carolina at Greensboro. Boyce serves as co-principal investigator . Boyce is now an associate professor with ASU’s Mary Lou Fulton Teachers College, and she continues to be active with EvaluATE, which has received increasing requests for tools, training, consultations and presentations around DEI. 

EvaluATE – ATE Evaluation Resource Hub is a supplemental sub-award from Western Michigan University that allows Boyce to contribute toward developing practical guidance and tools for measuring equity, diversity and inclusion as part of ATE evaluations.

Boyce’s research through this grant utilizes EvaluATE’s surveys of ATE principal investigators and evaluators to learn how they are defining and measuring equity, diversity and inclusion. The project involves interviewing ATE principal investigators and evaluators to learn more about this aspect of their work, and to identify barriers and promising practices. The study focuses on three case studies. These methods provide evidence to answer five research questions:

1. How are ATE grantees currently defining and measuring equity, diversity and inclusion in their research and evaluation practices?
2. To what extent do the current practices and data collection methods align with the National Academy of Sciences objectives and indicators?
3. What conditions or resources are necessary in order for ATE projects to successfully gather and report data on the NAS indicators?
4. What conditions exist in the ATE context (community colleges) that impede or facilitate the collection of sound data on equity, diversity, and inclusion?
5. What is the perceived validity and utility of project-level data on equity, diversity and inclusion?

The study is slated to end in December 2024, although Boyce and the EvaluATE team have just been awarded a new grant to expand and continue the research.

Findings and impact

Thus far findings have been presented in a manuscript published with the American Journal of Evaluation: Boyce, A. S.; Tovey, T. L. S.; Onwuka, O.; Moller, J. R.; Clark, T.; and Smith, A. (2023).Exploring NSF-Funded Evaluators’ and Principal Investigators’ Definitions and Measurement of Diversity, Equity, and Inclusion. American Journal of Evaluation, 44(1), 50–73.

A second manuscript is in preparation, and findings have been presented at the following conferences: the American Evaluation Association Conference, the Center for Culturally Responsive Assessment and Evaluation Conference, the ATE Principal Investigators Conference and the upcoming April 2024 American Educational Research Association conference.