Boston Youth Arts Evaluation Project
Tools
The project developed and tested five different sets of evaluation tools:
- Beginning Self-Evaluations and Final Self-Evaluation: This is the basic pre- and post-test model where most of the questions are the same in the beginning and final evaluations. Youth fill out Self-Evaluations in the first three weeks of the program and in the last two weeks of the program in order to measure the short-term program impact. Inspired by: Search Institute (2004), Lerner, R. M., & Israeloff, R. (2007), Forum for Youth Investment (2008), Oyserman, D., Bybee, D. & Terry, K. (2006), Oyserman, D. (2007), Brooks, McCarthy Ondaatje, & Zakaras (2005), and Massachusetts Board of Elementary and Secondary Education (2008).
- Program Evaluations: Feedback from Youth: Youth evaluate the programs in which they are involved at the end of their program cycle. We used some elements of the retrospective evaluation when we designed our Program Evaluations to both measure the program effectiveness and to encourage youth to think back on what they were like before the program. In doing so, they could compare differences between their past and current states and consider whether changes resulted from their involvement in the program. Inspired by: National Research Council and Institute of Medicine (2002), The Forum for Youth Investment (2007), Little, Dupree & Deich (2002), Bouffard, Little & Weiss (2006), Smith & Hohmann (2005).
- Alumni Surveys: Our Alumni Evaluation was designed as both a survey of current lives and as a retrospective evaluation, inviting young adults to reflect on what they experienced and gained in their time with our organizations. Program graduates complete an extensive survey online or in hard-copy form. Inspired by: Gittleman (2007), Csikszentmihalyi & Schneider (2000), Catterall (2009), and Gambone, Klem, & Connell (2002) .
- Teacher Evaluations: These were designed as a combination of observations and case studies on individual youth where teaching staff completes an evaluation on each youth in the beginning and at the end of the program. Inspired by: Massachusetts Arts Curriculum Framework (1999), Seidel et al (2009), Hetland, Sheridan, Veenema, & Winner (2007), and the “5 Processes” by Larry Scripp in Rabkin & Redmond (2004, p. 42).
- Artistic Response and “This program is like!”: These creative, open-ended responses offer the widest range of expressive opportunity. Youth respond through drawing and creative statements at the end of the program to what they feel has changed due to their experience in their program. This incorporates the power and voice of youth as artists in drawing their responses and helps them speak in metaphors to more richly describe their experience. Designing these tools works best when each organization can match the creative tools to the arts modalities that they offer.
All five of these tools can be found in the BYAEP Workbook (PDF)
