Research update: Internal grant projects, 2016–17
The 2016 and 2017 rounds of the Mary Lou Fulton Teachers College internal grants program provided support for 15 research projects conducted by individuals and interdisciplinary teams comprising 20 MLFTC faculty members, three doctoral students and one staff member; and six faculty members of other colleges and institutions. Here’s a review of the projects and some of their results.
2017 awardees
“Toward Establishing Evidence of External Validity for the Intersectional Competence Measure”
When teachers first enter the education workforce they will encounter complex student populations comprising diverse backgrounds and experience, abilities and disabilities. “Intersectionality” is a concept that describes the interaction of these multiple areas of difference within and across groups. Mildred Boveda, assistant professor of special education and cultural and linguistic diversity, coined the term intersectional competence to describe an educator’s understanding of the interaction of these markers of difference within student groups, families, and their professional colleagues. In 2016, Boveda created the “Intersectional Competence Measure” to assess preservice teachers’ preparedness for working effectively with these populations. With this study, she is attempting to establish the validity of the ICM through interviews with focus groups in predominantly white institutions, along with cognitive pretesting interviews with preservice teachers in three regions across the U.S. The project will continue through April 2019.
“Teachers’ Perceptions of Instructional Significance for History”
For Lauren Harris, recent debates in the U.S. regarding the appropriateness of Confederate monuments raise important issues for history teaching. An associate professor of history education with joint appointments to ASU’s Mary Lou Fulton Teachers College and School of Historical, Philosophical and Religious Studies, Harris says the controversy resurrects questions about the definitions and purposes of history; particularly as that history is found in school curricula, state education standards and textbooks. She describes history teachers as “curricular-instructional gatekeepers” who maintain a degree of control over the topics and perspectives they present in their classrooms, despite the mandates of standards and curriculum. However, little is known, Harris says, about how those teachers make choices about the history they teach. She seeks to fill that knowledge gap with this study that asks, how do history teachers determine what is instructionally significant for their classrooms?
Harris and a colleague at The College of New Jersey, working through history and social studies professional organizations, surveyed 260 teachers in 29 states. Preliminary analysis of the survey results indicates that teachers rely on state standards for decision making more than on textbooks or their district curriculum. When state standards offer teachers a choice of what to teach, the teachers ranked historical significance (what historians deem to be important) above student interest or the potential relevance of content to students. Harris is following up the survey with interviews and expects to be able to submit publication proposals in July.
“Braincandy: Providing Students Authentic, Engaging, and SAFE Spaces to Articulate and Refine their Thinking with Others”
Assistant Professor Bryan Henderson developed Braincandy in 2010 while a PhD student at Stanford. The cloud-based app provides students the safety of anonymous participation while utilizing questions developed by scientists and educators to elicit differences of opinion. Teachers employ Braincandy in their classrooms, facilitating student discussions of their differences, improving their language and reasoning abilities, conceptual understanding and social skills. The app is coordinated with the Braincandy website, which provides a library of questions, provides technical and strategy support, assembles networks of users and offers professional development. With the 2017 grant, Henderson and PhD candidate Earl Aguilera are improving the Braincandy system through design-based research using data derived from previous student use of the app. Henderson proposed a pilot study to evaluate aspects of a Braincandy learning environment that might be adaptable to the broader, non-Braincandy literature.
Henderson and Aguilera used the grant to carry out classroom observations and interviews at multiple grade levels using the latest version of Braincandy. They examined how use of Braincandy varies among teachers and students, and how the use of the technology changes within individuals over time. Their goal is to use this pilot project as a step toward larger grants for research on classrooms utilizing Braincandy. Among the preliminary findings Henderson wants to follow up in greater depth is the development of a practical framework for teachers interested in implementing technology like Braincandy in their classrooms. Aguilera defended his dissertation in 2018 and will be joining the faculty of Fresno State University, but he will continue to collaborate with Henderson on the project.
“Induction Experiences for Early Childhood Special Educators: Using Dramatic Inquiry to Increase Student Engagement and Positive Social Interactions”
While many children begin school ready to interact socially with peers and teachers, some are acquiring these skills, and others enter early school experiences having developed challenging behavior patterns. Associate professor Wendy Oakes and Assistant Professor Kathleen Farrand believe early childhood educators have the opportunity to positively affect student outcomes by creating safe, positive, proactive learning environments, but they note few teacher-preparation programs offer coursework supporting social and behavioral learning of children with challenging behaviors. Oakes and Farrand created this project 1) to offer induction experiences for MLFTC early childhood special education graduates as they begin careers, and 2) to examine the effects of a highly engaging instruction practice — dramatic inquiry — on students’ social and behavioral outcomes. The researchers theorize that feelings of preparedness to support children’s prosocial behaviors may translate into teachers’ greater persistence in implementing positive practices when they encounter challenges; an important consideration given that feeling underprepared to address children’s learning and behavioral challenges is one reason special educators leave the field at higher rates than general educators.
The researchers will be working with early career teachers from two school districts in 2018–19, providing professional development and coaching of dramatic inquiry. Participants are teaching students in special education preschool, elementary special education and middle school STEM. Farrand and Oakes are also developing partnerships with community educators to support the teachers’ planning and content development. Their 2017 pilot program created dramatic inquiry units exploring oceans and entomology. An upcoming unit on paleontology includes videos on museum resources and “Dinosaurs 101,” as well as a video recruiting students as paleontologists.
“Public High School Physical Activity Facility Use During Non-school Hours”
America’s elementary and secondary schools have traditionally been important settings for promoting physical activity among young people. Among the advantages they offer is the availability of designated activity areas including athletic practice fields and tracks, tennis courts, wrestling and weight-training rooms and dance studios. The increased incidence of obesity in the U.S. led the Centers for Disease Control and Prevention, the National Academy of Medicine and other professional organizations to promote initiatives increasing opportunities for physical activity for all age groups. Schools are in a unique position, due to their locations and their athletic-space assets, to offer an environment for members of the community to be physically active during non-school hours. Professors Hans van der Mars and Pamela Kulinna are conducting an exploratory study of activity areas at select Maricopa County, Arizona, public high schools. Their focus areas are 1) types and environmental features of dedicated physical activity areas, and 2) physical activity levels of people present in these areas during off-hours (late afternoons and evenings on weekdays, and weekend days).
Van der Mars reports, “We are in the midst of data collection with our project. Some school districts have agreed to participate while others have opted not to provide us access to the high school campuses. It is too early to draw conclusions, but after just over 1,300 scans of designated physical areas on public high school campuses in Maricopa County, Arizona, during these after school hours, it appears the large majority are unused. In part, this is a consequence of lack of accessibility, and/or not being usable due to scheduled events, being locked, or irrigation. Even when they are accessible, the vast majority of these publicly-funded activity areas remain unused.”
2016 awardees
“Teacher Evaluation Systems as Based upon Growth and Value-Added Models after the Every Student Succeeds Act: A National Overview Revisited”
The 2016 passage of the Every Student Succeeds Act provided states with more control over their education accountability and assessment systems. One of the areas in which that control has manifested is a move away from using value-added models in teacher evaluation, which attempt to measure a teacher's impact on student achievement. Several states have de-emphasized VAMs, while others still assign them a significant, high-stakes role in teacher accountability. Professor Audrey Amrein-Beardsley, associate director of the Division of Educational Leadership and Innovation; along with PhD student Kevin Close; and Clarin Collins, director of scholarly initiatives in the Office of Scholarship and Innovation, surveyed the education departments of each state and the District of Columbia to assemble a resource of teacher evaluation practices nationwide in the wake of the ESSA. The survey requested detailed information about teacher evaluation systems including questions regarding perceived strengths and weaknesses, value-added models and consequences tied to evaluations. This study follows similar research Amrein-Beardsley and Collins conducted and published in Teachers College Record in 2014, providing an overview of state teacher evaluation plans as influenced by Race to the Top and waivers for No Child Left Behind.
The researchers found that most states have not made substantive changes to their policies, and a few have kept the same teacher evaluation systems in place, post-ESSA. The team suggests this is attributable to the significant resources states invested in developing and implementing evaluation systems under prior policies which had stronger federal mandates and incentives. Many states now allow local education authorities to select from menus of evaluation models, many of which include measurement components that were used before ESSA. As a result, states are requiring or endorsing evaluation systems that are the same or slightly altered formulations of systems they previously required. Amrein-Beardsley’s team wrote a policy brief, to be published later this month by the National Education Policy Center, and has an article in progress for Teachers College Record which will provide updates and changes since the 2014 article. The team also presented at the 2018 annual meeting of the American Education Research Association.
“Harnessing Interdisciplinary Research to Examine the Relationship between Student Voice and School Change toward Equity”
Melanie Bertrand partnered with three other assistant professors — Sybil Durand of the department of English, Saskias Casanova in the School of Transborder Studies, and Taucia Gonzalez at the University of Wisconsin-Madison — to take an interdisciplinary approach to the problem of inequity in education along lines of race and other forms of difference. They investigated the relationship between student voice (specifically that of students of color) and school change toward equity as related to discourse, multicultural young adult literature, psychology, and intersecting identities (including those of immigrant students and students with disabilities). The team agreed that research indicates student voice holds promise in addressing systemic inequities in schooling, but saw a need for further investigation. They utilized data collected from an after-school program in which youth of color read multicultural young adult literature, engaged in participatory action research and advocated for school change. Their analysis of the data was informed by the Change Laboratory, an application of cultural-historical activity theory in which participants strive to change their environment.
The research team video-recorded themselves in a series of meetings throughout the project year as they confronted contradictions between their disciplines in analyzing the data. Their findings shed light on the complex processes involved in facilitating and hindering equity in schooling through student voice. The research showed the value of co-constructing spaces in which hierarchies related to race, age, disability and ability, gender, sexuality, social class and more are challenged. Six co-authored or single-authored pieces by the research team (which included two graduate assistants) have been published or are under review, and members of the team have presented in Florida, California and Mexico. They plan to complete publications in progress and submit to present at AERA 2019 in Toronto.
“Supporting Preservice Teachers to Implement Argumentation in Science Classrooms”
Argumentation is a core practice for engaging students in learning science, yet there is little empirical data on preservice teachers’ preparedness for implementing successful scientific argumentation. With this project, Assistant Professor Ying-Chih Chen and instructor Jaclyn Hernandez sought to identify what PCK — pedagogical content knowledge — is important to allow preservice middle school science teachers to implement argumentation in their classrooms. Chen and Hernandez also explored how teacher candidates develop their PCK of argumentation when they implement an approach called “Science Talk Writing Heuristic.” The project focused on four preservice middle school science teachers completing their senior year student teaching assignments. Their mentors were also involved in the project to help develop curricula, strategies and assessments for implementing argumentation in their classrooms.
After preservice teachers implemented the STWH, results show they struggle more with the development of dynamic PCK of argumentation than they do with declarative PCK of argumentation. Teachers with weaker declarative PCK of argumentation had increased difficulty detecting students’ thought processes and were unable to implement strategies to support students in their social negotiation of ideas. This project suggests argumentation should be considered as an enterprise of uncertainty management and teachers should be supported through systematic training. Based on this study, future research areas could include critical resources for managing uncertainty when engaging students in argumentation, resources to create productive conditions that lead to knowledge development, and teachers’ knowledge of uncertainty management in argumentation when they implement argumentation in their classrooms.
“Building Learning Experiences with Design Thinking, Creativity, and Entrepreneurship: A Study of 3-Day Startup”
Many educators are interested in exploring opportunities for design-based environments for learning, yet existing research does not offer many empirical investigations around implementing design thinking or design workshops in ways that support creativity and entrepreneurial thinking. Assistant Professor Danah Henriksen, Associate Professor Michelle Jordan and graduate research assistant Mathew Evans conducted a qualitative study examining participants’ learning experiences in a design thinking-focused, entrepreneurial workshop called the 3-Day Startup. Students were observed working collaboratively, using multiple disciplines and design-thinking processes, to develop a startup company. The research team hoped to address these questions: 1) What is the student learning experience like in an intensive, design-focused workshop, based on the process of collaboratively creating, negotiating and building ideas? 2) In what ways do student ideas and experiences evolve creatively during and through the design thinking process?
The researchers collected an extensive amount of qualitative data, which Evans documented with video logs created throughout 2017. In the latter part of the year, the team began analyzing the data and identifying core ideas, which led to an October presentation for the biennial conference of the Southwest Consortium for Innovative Psychology in Education. Analysis continues, with the team discussing how best to disseminate the results. Based on this pilot study, they envision additional connections for the 3-Day Startup between faculty across disciplines at ASU (but focused on MLFTC) and education stakeholders interested in incorporating design thinking into their programs. They also hope the project will be a step toward a larger grant application built collaboratively with ASU STEM faculty.
“Improving Preservice Teachers’ Noticing Expertise and Beliefs Through Methods Courses Incorporated Technology”
Teachers design learning experiences based on what they pay attention to in students’ thinking and how they make sense of it. Mi Yeon Lee, an assistant professor of mathematics education, says this makes “noticing,” a form of professional vision, one of the highest-leverage core instructional practices, with a direct positive impact on the quality of teaching. Her study investigated an approach to improving preservice teachers’ noticing skills with the goals of 1) effectively integrating the Three Point Framework (key concept, possible point of difficulty, proposed course of action) and technology into an elementary mathematics methods course; 2) using these resources to foster preservice teachers’ development of professional noticing expertise; 3) preparing those teachers to use their noticing expertise in their future classrooms; 4) assessing the effects of the technology-based intervention on preservice teachers’ noticing skills and their beliefs about mathematics teaching.
Lee gathered data from spring 2017 to spring 2018, from 24-30 preservice teachers enrolled in a math methods course at ASU. Data were collected from a variety of sources, both quantitative (e.g., noticing expertise questionnaires, mathematics beliefs survey) and qualitative (e.g., written reflections and class products). She is currently analyzing quantitative data using statistical software. Qualitative data will be analyzed based on grounded theory and a noticing framework. Her preliminary analysis shows the majority of preservice teachers increased their noticing expertise.
“Long-Term Teacher-School Matching Algorithm”
Assistant Professor Margarita Pivovarova partnered with Jorge Sefair, assistant professor in ASU’s School of Computing, Informatics and Decision Systems Engineering, for this interdisciplinary study to develop and test a matching model and solution algorithm that generates long-term, stable teacher-school pairings. Such stable pairings could result in fewer teachers leaving schools due to dissatisfaction with their jobs by aligning their expectations and goals with their districts’ at the initial hiring stage. The researchers derived the idea from similar algorithms widely used to match new medical residents with hospitals, but factors unique to teaching — decentralized decision-making, asymmetric information, desirability (for both teachers and schools) of long-term matching, and multiple factors influencing teacher mobility, for example — prevent existing algorithms from being directly adapted. The researchers proposed an algorithm that borrows from theoretical advances in economic science and practical applications of these concepts in engineering and also incorporates information from educational research on factors affecting teacher mobility and retention. These include teachers’ individual characteristics (age, gender, race, experience) and school-level factors (student demographics, working conditions, location and infrastructure).
Pivovarova and Sefair completed the design of the mathematical model and began testing using simulated data, and also real data from the Arizona Department of Education. From these data, they generated predictions of stable assignment. Next, they will test their predictions to determine if they are consistent with the teacher moves they observe in the ADE data. Pivovarova and Sefair presented their algorithm at the 2017 MLFTC internal grant poster session and the 2018 conference of INFORMS Business Analytics. In addition to disseminating their findings through academic journal publications and conference presentations, they intend to publish a user-friendly algorithm that school districts can use to inform hiring decisions.
“Advancing Computerized Adaptive Testing through Participant-Based Research and Open Source Platform Tools”
Assistant Professor Yi Zheng proposed a small-scale, human-subject study investigating effects of different designs of computerized adaptive testing. CAT is a technology-enhanced test delivery mode in which computer algorithms are created to tailor a test in real time to the
individual test-taker; for example, displaying harder or simpler questions based on correctness of prior answers. Despite the increasing popularity of CAT, most published research has been conducted using only computer simulations without human-derived data. Zheng postulated that simulated data may deviate significantly from the real cognitive and behavioral processes of humans. Her objective with this project was the foundation for a long-term research program to build a robust, sustainable laboratory carrying out a diverse set of human-subject experiments to better inform applications of CAT in practice.
The project began in December 2016 and has spanned three semesters. Zheng built the technical and logistical foundation of her virtual lab and hired three student workers to assist with item bank construction, platform configuration and experiment administration. Participants in the pilot study, recruited among undergraduate students at ASU who have taken a pertinent math course, took the test in a computer lab on campus. Administered through a web browser, the test comprised introduction and consent, a practice section, the experimental test section and a questionnaire that asking for demographics, anxiety level, attitudes and experience in the experiments. Zheng has finished data collection and has given three presentations, including for the annual meeting of the National Council on Measurement in Education in April, which will be submitted for publication. Zheng says her experience gained from installing and configuring the CAT platform, recruiting participants, obtaining informed consent, the testing and survey components and data collection, has laid the foundation for future projects.