NYC Outward Bound Schools Crew Evaluation
By Metis Associates
August 2020
Purpose of the Study
The evaluation study was designed to assess:
- The impact of Crew on middle school students’ academic, social-emotional, and behavioral outcomes;
- The relationships between specific elements of Crew and student outcomes;
- Opportunities for improving Crew;
- Implications for schools that use or are contemplating using an advisory structure.
WHAT IS CREW?
Crew is the signature advisory structure at all NYC Outward Bound Schools, and provides essential social-emotional support for students. Crews are made up of 10-16 students led by an adult Crew Advisor, often a teacher, and meet frequently — sometimes daily — at our schools. Many Crews stay intact throughout a student’s middle or high school career.
Study Design & Implementation
This multi-year study began in 2016 and was completed in 2020. It was conducted by Metis Associates and funded by The Booth Ferris Foundation. It focused on implementation of Crew in three NYC Outward Bound middle schools (“study” schools): WHEELS, in Manhattan, MAELS in Staten Island, and MELS in Queens.
To determine possible effects on outcomes, a quasi-experimental approach wherein comparisons to outcomes from seven non-Outward Bound schools was employed. Comparison schools were selected based on their similarity to the three study schools on metrics including, school size, grades served, geography, demographics, and aggregate academic metrics. The study looked at possible impacts of Crew on average daily attendance, chronic absenteeism, scores on NY State ELA and Math tests, and social-emotional/character growth, as measured by the Resiliency Attitudes & Skills Profile (RASP), an instrument that the students took several times over the course of the study. This was combined with primary information gathering about Crew implementation in the target NYC Outward Bound schools, which included site visits, focus groups, observations, student and teacher surveys, and principal interviews.
Promising Findings
Crew had a statistically significant impact on reducing chronic absenteeism. Students in the NYC Outward Bound schools had a significantly lower proportion of chronic absentees when compared to their comparison school counterparts. There were twice as many chronic absentees among the comparison group students than among the NYC Outward Bound Schools students (11.4% vs. 5.3%). This finding was so noteworthy that Metis is recommending submitting it to the What Works Clearinghouse for publication.
In interviews and focus groups, administrators, staff and students from across the 3 study schools spoke very highly of the Crew experience. They cited many benefits, including the following:
- Relationships. School staff and students indicated that Crew plays a significant role in their schools by providing a unique opportunity for building relationships.
- Community. Crew serves as a student’s primary community within the school.
- Academics. Crew supports students’ academic growth, though stakeholders said that academic impacts may not be readily apparent at the middle school level but rather might emerge as a long-term outcome.
- Moral support. Students spoke of the importance of the moral support they received from their Crews.
- Family-school connections. Crew serves as an important link between a student’s family and the school.
Overall, perceptions of Crew were positive based on the responses from all groups. 78% of students felt that their Crew leader cared about them. In addition educators reported their satisfaction with the supports they received and their opportunities for collaboration. For example, most teachers (78%) indicated that they received just the right amount of support around Crew.