Evaluating the integration of DEI elements into STEM educational program design

The challenge

EvaluATE  — which is based at The Evaluation Center at Western Michigan University —  is the evaluation hub for the National Science Foundation’s Advanced Technological Education (ATE) program. They educate the ATE community, which includes evaluators, project leaders, and staff, grant specialists and college administrators, about evaluating the effectiveness of equity, diversity and inclusion as part of educational program design, particularly around STEM-related subjects. 

The focus of the EvaluATE project, was to understand the state of DEI in ATE-funded projects. The project received funding from the National Science Foundation in 2018 to expand and begin multiple research projects. Now there is an identified need to shift toward a more participatory action research case study methodology with three ATE projects. EvaluATE has received increasing requests for tools, training, consultations and presentations around DEI. This shifting emphasis will require an expansion of the scope of this previously funded project from its initial focus on understanding the state of DEI in ATE-funded projects towards a more proactive and collaborative discovery of how DEI can be operationalized better.

The approach

The EvaluATE 5.0 project is led by Lyssa Becho, a principal research associate at the Evaluation Center at Western Michigan University, and Ayesha Boyce, who began this work while at the University of North Carolina at Greensboro and serves as co-principal investigator. Boyce is now an associate professor with ASU’s Mary Lou Fulton Teachers College, and she continues to be active with the project.

Funding from the NSF for EvaluATE 5.0 totals $6 million, of which $340,613 will be managed through Boyce and MLFTC/ASU to support the project’s expanded focus from understanding the state of DEI in ATE-funded projects toward a proactive and collaborative discovery of how DEI can be operationalized and better engaged. The project leads will:

• Extend current participatory action research case studies to better understand how they enact and implement their DEI action plans.
• Recruit and select two additional projects to work with on participatory-action research case studies. 
• Use findings to develop evidence-informed DEI tool kits focused on specific issues faced by ATE projects.
• Collect responses and reactions to the toolkits.
• Continue to broadly disseminate project findings via webinars, website posts, blogs, conference presentations and peer-reviewed manuscripts.

Through this project, principal investigators and evaluators in ATE projects get an opportunity and facilitative space to reflect on the processes and practices of DEI in their project, plus data collection and evaluation. Benefits to the field of evaluation will include supporting the development of evaluators who are more engaged with culturally responsive, DEI-focused evaluations. The project will provide evaluators with greater information and perspective on how ATE projects operationalize and evaluate DEI, adding to the conversation with analysis on processes, dynamics, and specific actionable methods.