Psychometric Advisory for Maricopa Community Colleges Math Diagnostic Assessment Project

The challenge

How can we improve the quality of tests that assess math skills so that students are placed into the appropriate courses?There are many critical or auxiliary factors affecting the reliability, validity and accuracy of a test. Many parents, educators, students and others do not realize the expertise and effort required to create a good, rigorous and sustainable test. This work will improve the quality of the developed tests so that incoming students who take these tests can be placed into appropriate courses.

 Based on nearly a century of educational measurement theory development, the psychometric consultant provides theory-based insights and ready-to-use tools to help the team efficiently control the quality of the products. They jointly determine how to plan pilot tests, how to analyze pilot test data and how to set cut-off scores. The consultant attends the MCCC-MDA project team meetings, answers questions related to test design and development and leads pilot test data analysis.The goal of the Maricopa County Community Colleges Math Diagnostic Assessment project is to develop a set of tests to measure students’ mastery levels in seven modules recently created for precollege math remedial curricula. Students who pass a test will not be required to take a particular remedial module. Students can proceed with regular college math curricula after they test out of or successfully complete all seven remedial modules. The principal investigator on this grant serves as the psychometric consultant on the project and provides advice on designing tests to reliably measure the intended mastery levels of students.

These are significant tests because students with sufficient mastery of corresponding content should successfully test out of taking the modules. Otherwise, they would need to spend additional resources taking the modules. Also, students without sufficient mastery of the content should score at a level that places them into the modules, or they won’t be ready for the college math curriculum. It is crucial to make sure the tests are reliable, valid and accurate through rigorous procedures involving empirical evidence and logical arguments. 

The approach

Based on nearly a century of educational measurement theory development, the psychometric consultant provides theory-based insights and ready-to-use tools to help the team efficiently control the quality of the products. They jointly determine how to plan pilot tests, how to analyze pilot test data and how to set cut-off scores. The consultant attends the MCCC-MDA project team meetings, answers questions related to test design and development and leads pilot test data analysis.

Findings and impact

The work is in progress. The tests are currently being developed and pilot-tested and will be put into operation once completed.