Over the last year the ACRES (Afterschool Coaching for Reflective Educators in STEM) research team has piloted and deployed an innovative new assessment tool utilizing animation software to mimic an afterschool program. This tool allows participants an opportunity to give feedback about an educator’s use of questioning skills. Built off of the idea of virtual training and courses in other sectors (this idea came after a member of the research team participated in a virtual first aid course), this assessment is designed to evaluate how participants’ abilities to identify and provide feedback on effective questioning strategies changes from before to after they take the ACRES course.
Participants are asked to watch a series of short animated videos of an educator working with students on a slime activity in an afterschool setting (see example from first video clip below). After each short clip, the participant is asked to give feedback on the animated educator’s strengths and weaknesses related to their questioning strategies. This animated assessment overcomes many of the challenges of using live video clips. First, it avoids any confidentiality and privacy issues; and second it allows the research team to script out specific points in the video to demonstrate effective questioning or an opportunity for improvement.
To date we have captured around 40 participants who have taken the Purposeful Questions Module with the instrument. Initial analysis indicates that ACRES participants get better at noticing and being able to talk about purposeful questioning strategies—especially the more nuanced skills of waiting after questions for students to answer, using open-ended questions more strategically, and following up with students to clarify their thinking. Beyond capturing skills acquisition the research team can relate each participant’s increase or decrease in their ability to other measures such as increases in confidence and the self-reported impact that various course elements had on their learning. Findings here indicate that participants that value social interaction with their peers; viewing each other’s videos of themselves working with youth; and the opportunity to reflect on their practice show greater increases in skill acquisition. A really powerful finding given participants only have just a few hours of virtual face-to-face time in the 3 session course with their coach and peers.
