2020-2021 YEAR END REPORT
Engage San Francisco Literacy Leo T. McCarthy Center for Public Service and the Common Good Dresden Frazier-Smith, Literacy Program Coordinator Cassidy Newberry, Data Science Graduate Assistant
2
TABLE OF CONTENTS Overview _________________________________________________ 3 General Program Structure _________________________________________ 3 Program Highlights _________________________________________________ 3 Assessment and Evaluation Highlights _______________________________ 3 Key Highlights _____________________________________________ 4 Virtual Tutoring in a Pandemic ______________________________________ 4 Cultural Humility Curriculum _________________________________________ 4 Community Speaker Series _________________________________________ 4 Assessment and Evaluation ________________________________ 5 Spring 2021 Engagement with Children ______________________________ 5 Partner Programmatic Feedback ___________________________________ 5 Partner Evaluations of Tutors ________________________________________ 5 Summary and Discussion ___________________________________________ 6 Tutor Feedback ____________________________________________________ 7 Fall 2020 Program Assessment Results ________________________________ 7 Free Response Highlights ___________________________________________ 8 Spring 2021 Program Assessment Results _____________________________ 9 Free Response Highlights ___________________________________________ 9 Summary and Discussion __________________________________________ 10 Fall 2021 Goals and Structure _____________________________ 11 Returning Tutors ___________________________________________________ 11 New Site Leads and Graduate Intern _______________________________ 11 Program Structure and Recruitment ________________________________ 11 Assessment and Evaluation ________________________________________ 11
3
OVERVIEW General Program Structure Engage San Francisco Literacy partnered with three sites this year, Prince Hall Afterschool Program, Rosa Parks Elementary, and Dr. William L. Cobb Elementary to facilitate virtual tutoring. We unfortunately were not able to partner with Booker T. Washington Community Service Center or Magic Zone due to a lack of compatibility between in-person learning hubs and ESF Literacy’s online tutoring options. In Fall we had 42 tutors, and in Spring we had 34 tutors. The 2020-2021 school year was our second year utilizing the Site Lead structure. We had two undergraduate students leading each site team, with the except of Cobb which had one Site Leader. These nine students facilitated monthly team meetings and one-on-one check-ins twice a semester. They also communicated regularly with site staff, responded to low-level tutor concerns, and ensured smooth functioning for their site.
Program Highlights We successfully worked with three of our five partner sites to run fully online tutoring. For our SFUSD partners, tutors worked as teaching assistants, providing support to teachers in their online classrooms and chaperoning breakout rooms to provide more individualized attention to the children. At Prince Hall, we designed an online afterschool program that utilized zoom breakout rooms to allow our tutors to work with children in a one-on-one setting. We also piloted the Cultural Humility Curriculum in Fall, which prompted tutors to evaluate their role in the Western Addition community, and identify how they can enact the Leo T. McCarthy Center values and practice cultural humility by employing antiracism, empathy, and love in their work as tutors. Lastly, in Spring we held a community speaker series that highlighted work being done in the community by the community.
Assessment and Evaluation Highlights In total, ESF Literacy worked in community approximately 93 hours/week during the Spring Semester. The area we tutored in most was math at 46.6%. Second was literacy, with 41.5% of our time spent on reading and writing. Tutors also reported that 94% of the time the children showed medium to high engagement. The results of tutor evaluations were very positive. Our tutors were scored primarily as “Meets Expectations” or “Strength Area” in all competencies and their scores increased from Fall to Spring. Lastly, tutor feedback was also highly positive. Scores were particularly high in areas of ESF Leadership and Partner Staff support. Tutors also rated the effectiveness of the educational components well.
“Overall, such a great experience. It is a very well thought out program with incredible learning opportunities and staff support. Wish I could stay longer.” -Meisy Tunay, Site Lead
4
KEY HIGHLIGHTS Virtual Tutoring in a Pandemic Due to the COVID-19 Pandemic, the ESF Literacy Program needed to pivot to an online format in order to stay connected to the community. We worked closely with Prince Hall to ensure that our partners were proficient in Zoom and constructed an entirely online afterschool program. Tutors were placed into breakout rooms with children to work one-onone with them on homework and reading. In Spring semester, we also piloted a new feature of this online program, Activity Time. Each day, one to two tutors hosted a breakout room for children to socialize and play games. Our SFUSD sites were also very successful in incorporating our tutors into their virtual classrooms. Tutors worked as teaching assistants, proving support to teachers in their online classrooms and also chaperoned breakout rooms to provide individualized attention to the children. When SFUSD returned to in-person, tutors continued to support teachers virtually. In the physical classroom, teachers paired children with a tutor on the computer, again for individualized support. We also provided nonacademic support with one of our tutors, Chloe, hosting an art workshops that prompted the children to reflect on and process the events of 2020 through the creation of silhouettes.
Cultural Humility Curriculum In Fall, our tutors participated in six one and a half hour team meetings and spent an additional one to three hours per week completing independent work in preparation for these meetings. This curriculum prompted tutors to evaluate their role in the Western Addition community and identify how they can enact the Leo T. McCarthy Center values and practice cultural humility by employing antiracism, empathy, and love in their work as tutors. By the end of the course, tutors were able to identify how to enact the LTMC values and mission in their work as literacy tutors, evaluate the history of oppression in the Western Addition as it relates to the current state of the community, identify antiracist practices to enact in their work, create an actionable definition of empathy to use in their work with children, identify how to enter the community in a culturally humble manner, and identify how to they can employ love as an act of social justice.
Community Speaker Series In Spring, we invited three McCarthy Center staff members to talk about our mission and how we execute that vision, as well as three community members to teach us about their work in community. LTMC staff members provided the tutors with a dynamic look at our history as a center and the ways in which we approach community engagement. The community guest educators provided a wealth of community-grown knowledge and expertise. Dr. Sheryl Davis, Director of the Human Rights Commission, walked tutors through her journey to the Human Rights Commission, highlighting the ways in which advocacy has defined her professional career. Neva Walker, of Coleman Advocates, touched on the core aspects of community organizing and the power that community holds to exact transformational change at the systemic level. Lastly, Darren Kawaii, Principal of Rosa Parks Elementary, dispelled the idea that children are going to be “behind” post-pandemic, and instead offered the statement that “all learning is learning” and the kids have been and will continue learning.
5
ASSESSMENT AND EVALUATION Spring 2021 Engagement with Children At the Prince Hall Afterschool Program, our team tutored approximately 25 hours per week. Before SFUSD returned to in-person, our Rosa Parks tutors were in the virtual classroom on average 40 hours per week. At Cobb, our tutors were in the virtual classroom on average 28 hours per week. In total, Engage Literacy worked in community approximately 93 hours/week during the Spring Semester. Engagement time decreased over the 15 week period. This is due to a number of interrelated factors: 1) In April, SFUSD began their return to in-person which decreased our ability to tutor with those sites, 2) Prince Hall saw a decrease in attendance over the semester due to student burnout, and 3) Tutors’ data submission decreased over the semester as well. Interestingly, the area we tutored in most was math at 46.6%. Second was literacy with 41.5% of our time spent on reading and writing. Tutors also reported that 94% of the time the children showed medium to high engagement. Specifically, 28% reported medium levels engagement and 66% reported high levels of engagement. Under the description for high levels of engagement, the tutors described the children as attentive, focused, engaged. For medium levels of engagement, the tutors described the children as focused and engaged, but also tired and sometimes distracted. Limitations: Due to the COVID-19 pandemic, data was not collected for Fall 2020. Additionally, tutors were inconsistent with submitting data and some tutors never submitted any data. The schedule and the data set were utilized to make the above approximations.
Partner Programmatic Feedback In order to not burden our partners who were already at capacity managing COVID response, We did not request extensive feedback on the program for this year. However, the limited feedback we received was overwhelmingly positive: “I know the USF tutors were very consistent and supportive for teachers this year! The tutors showing up each day ensured that students could have a little more 1:1 time with adults and a little more care, attention, and instruction.” - Alexandra Snell, Literacy Coach, Cobb Elementary “We could not have provided quality academic help without the USF tutors.” - Miriam Desmukes, Center Director, Prince Hall Our partners were very glad we were able to work with them through a crisis and be nimble in our response to the constantly changing landscape of education.
Partner Evaluations of Tutors Our tutors are evaluated at the end of each semester by their site supervisor. Partners score tutors in five areas as either 1 Growth Area, 2 Meets Expectations, or 3 Strength Area. We aim to have tutors score above a 2 and we meet one-on-one with any tutor who receives a 1
6 in any area. In Fall, we received evaluations for 28 of our 42 tutors. In Spring, our partners evaluated 26 of our 34 tutors. Fall 2020 Attendance and Punctuality: Barring illness or emergencies, the student shows up to every shift and starts/ends each shift at the expected time. When absences occur, the student communicates to the appropriate staff person within reasonable notice. Proactivity and Initiative: The student finds ways to be supportive to children or staff, without receiving an explicit directive to do so, even if the task falls outside of their normal routine. The student confidently makes the correct judgment calls on how to maximize their skills and time while on the clock. Accepting and Applying Feedback: The student willingly takes instruction when given on-the-spot directives during program/class time. The student respectfully receives critical feedback and puts forth effort to improve areas of challenge. Seeking Help and Reporting Concerns: The student approaches staff or Site Leaders with questions and seeks help when needed. The student accurately identifies concerns related to children’s well-being and reports those concerns to staff members in a timely manner. Compassionate Care for Children: The student has formed rapport and trust with the children they interact with; addresses behavioral concerns appropriately; and effectively supports children’s academic needs.
Spring 2021
Strength Area: 71%
Strength Area: 77%
Meets Expectations: 25%
Meets Expectations: 19%
Growth Area: 3%
Growth Area: 4%
Strength Area: 71%
Strength Area: 77%
Meets Expectations: 25%
Meets Expectations: 23%
Growth Area: 3%
Growth Area: 0%
Strength Area: 61%
Strength Area: 88%
Meets Expectations: 39%
Meets Expectations: 12%
Growth Area: 0%
Growth Area: 0%
Strength Area: 64%
Strength Area: 91%
Meets Expectations: 36%
Meets Expectations: 9%
Growth Area: 0%
Growth Area: 0%
Strength Area: 92%
Strength Area: 96%
Meets Expectations: 7%
Meets Expectations: 4%
Growth Area: 0%
Growth Area: 0%
Summary and Discussion As a whole, our tutors scored well in all areas and their scores increased from Fall to Spring. We had one tutor each semester who scored low on attendance and this was addressed in a one-on-one meeting with each tutor. While we have used these competencies for two years now, I am not sure they effectively measure the tutors’ performance. I would like to revisit how we assess our tutors in Fall 2021.
7 Cultural Humility Curriculum Assessment In order to measure the efficacy of the Cultural Humility Curriculum, we asked the tutors to complete an anonymous free response survey that covered core concepts from the curriculum: cultural humility, antiracism, love, and justice. Their responses were scored by the program coordinator on the same scale as the evaluations: 1 Growth Area, 2 Meets Expectations, or 3 Strength Area. In response to the question “What are the important components of antiracism” the team’s average score was 2.53. In response to the question “What are the key factors of cultural humility?” the team’s average score was 2.42. Overall, the tutors exceeded expectations and successfully met the learning outcomes of the curriculum. Key Response Excerpts: How does your work align with the McCarthy Center mission and goals? "I particularly strive towards the core value of humility in my work, accepting my strengths and weaknesses and entering moments of self-reflection in order to make myself not only a better tutor but a better person overall." How does love relate to Social Justice? "Both love and social justice require empathy and genuine care. In a way social justice is a larger scale of love not only for yourself or those immediately around you but also the greater community around you." "Love is how we transcend past our own boundaries and experiences and put ourselves in others' shoes and gain understanding for the life and unjust experiences happening all around us."
Tutor Feedback Every semester we give tutors the opportunity to anonymously assess the program. In Fall we offered a Student Satisfaction Survey that asked tutors to rate various statements on a scale of 1-5, with 1 being strongly agree and 5 strongly disagree. The statements covered the Cultural Humility Curriculum which we often referred to as professional development, team meetings, and personal and professional support from myself the Program Coordinator, Site Leads, and Site Staff. There is also a free response section where tutors can offer open feedback on management, site environment, and the professional development. Fall 2020 Program Assessment Results 1 Strongly Agree
2
3
4
5 Strongly Disagree
I feel the professional development readings/videos are promoting my growth as a tutor and/or a human
75%
12.5%
12.5%
0%
0%
I feel the team meeting discussions are an effective way to collaborate and grow as socially conscious tutors
75%
8.3%
12.5%
4.2%
0%
8 I feel personally supported as a human and student by the program coordinator
87.5%
8.3%
4.2%
0%
0%
I feel personally supported as a human and student by my site leader
79.2%
16.7%
4.2%
0%
0%
I feel personally supported as a human and student by program staff
79.2%
12.5%
8.3%
0%
0%
I feel supported professionally as a tutor by the program coordinator
87.5%
8.3%
4.2%
0%
0%
I feel supported professionally as a tutor by my site leader
79.2%
16.7%
4.2%
0%
0%
I feel supported professionally as a tutor by site staff
75%
16.7%
8.3%
0%
0%
I am able to prioritize my personal wellbeing and schoolwork over my job
45.8%
41.7%
4.2%
8.3%
0%
I enjoy my job
70.8%
20.8%
8.3%
0%
0%
Free Response Highlights Other thoughts or comments about the program coordinator: “She is the BEST program coordinator I have had in my 4 years of being at USF. Love her energy and the positive vibes that she brings to all of our meetings. She understands what it is like to be a working student and helps us in any way that she can. The weekly emails are always super informative and appreciated.” Other thoughts or comments about your site leader “Callie and Becca are really trying their hardest and I can tell that as a vet tutor. It's hard staying engaged right now and it's even harder finding ways to work and make curriculum for the kids to keep them engaged but I really feel like they are doing their best given the circumstances!” Other thoughts or comments on the professional development readings/videos and worksheets: “I think all the content we've been learning not only relates to our work at tutors but they're extremely beneficial in learning how to be a better human in general. It applies to everyday things and everyone around us so I think it's been great learning all these things.”
9 Spring 2021 Program Assessment Results 1 Strongly Agree
2
3
4
5 Strongly Disagree
The Speaker Series was beneficial to my work and growth as a socially conscious tutor
69.2%
23%
7.7%
0%
0%
The Speaker Series was interesting and engaging
73.1%
23%
3.8%
0%
0%
Team meeting discussions are an effective way to collaborate and grow as socially conscious tutors
65.4%
19.2%
15.4%
0%
0%
I feel personally supported as a human and student by the program coordinator
96.2%
3.8%
0%
0%
0%
I feel personally supported as a human and student by my site leader
88.5%
7.6%
3.8%
0%
0%
I feel personally supported as a human and student by program staff
88.5%
3.8%
3.8%
3.8%
0%
I am able to prioritize my personal wellbeing and schoolwork over my job
68.2%
23%
7.7%
0%
0%
I enjoy my job
84.6%
15.3%
0%
0%
0%
Free Response Highlights Feedback about the learning aspects of this program: “I’ve learned so much from the speaker series and cultural humility curriculum. Looking back on myself when I first started tutoring, I feel as though the self-reflection skills I’ve learned have been crucial to better fulfilling my role as a tutor.” Thoughts or comments about the program coordinator's management of the program: “Honestly, the program wouldn't be where it is today without Dresden. She is the backbone of ESF at USF. She is EXTREMELY understanding and relatable and I could not ask for a better manager.” Thoughts or comments about your site leaders management of your team/site: “Meisy and Cassidy added to Dresden's management flawlessly, we always got down to business but they were supportive friends along the way. They really opened up the space to some important conversations since we all felt comfortable with one another.” Thoughts or comments about the environment of your site: “Although Zoom was difficult, I felt supported by the teachers I worked with. We had great communication and planning skills.”
10 Thoughts or comments related to COVID, virtual engagement, returning to in person, etc: “I am very nervous to return in person and would prefer to not next semester if possible. In-person classes are going to be enough of an anxiety and I would still like to work without worrying of getting ill or spreading an illness.” Summary and Discussion Overall, tutors seem highly satisfied with the program. Scores were particularly high in areas of ESF leadership and partner staff support. Those scores increased from Fall to Spring. Tutors also rated the educational components high with 75% of tutors selecting strongly agree for the statement, “I feel the professional development readings/videos are promoting my growth as a tutor and/or a human.” The only areas where I see a need for improvement are in the effectiveness of team meetings and tutors ability to prioritize school over their work. For team meetings, I believe this could be due to the Zoom format. We collected tutor feedback on how they’d like team meetings to be structured moving forward, and will use their input to make adjustments. Additionally, I’d like to increase the teams’ skills in and understanding of work boundaries so they can feel comfortable prioritizing their school work and wellbeing.
11
FALL 2021 GOALS AND STRUCTURE Returning Tutors There are 25 confirmed tutors returning in Fall 2021. We are looking to hire 50-75 new tutors across the 2020-2021 school year. In late August and early September, we will recruit 25+ new tutors. Then, over the Fall semester and into Spring our goal will be to reach a team of 75-100 total tutors. Lastly, we aim to hire 10-15 students from USF’s Undergraduate Teacher Education Center. These students will be placed at our SFUSD sites, Rosa Parks and Cobb Elementary. Current
Total Requested by Partners
Magic Zone
5 tutors
15 tutors
Booker T. Washington
3 tutors
15 tutors
Prince Hall
11 tutors
20 tutors
Rosa Parks Elementary
5 tutors
20 tutors
Cobb Elementary
3 tutors
10 tutors
Total 25 tutors 80 tutors * Numbers specific to each site include tutors who work at multiple sites.
New Site Leads and Graduate Intern We hired three new Site Leads to replace those who graduated. Giuliana will be joining Callie at Prince Hall, Allison will be co-Site Leading with Kristin at Magic Zone, and Aliyah with be the Site Lead for Rosa Parks co-leading a joint team with Juliana the Cobb Site Leader. We also hired a Higher Education and Student Affairs graduate intern, Carla, who will be joining the team this Fall.
Program Structure and Recruitment
Returning tutors will begin tutoring on September 7th. Carla, the Site Leaders, and I will begin recruitment August 23rd. We’ll hold group interviews September 1st - September 10th. New tutors will be onboarded over the following weeks and can begin tutoring after September 27th pending their clearance.
Assessment and Evaluation We also hope to invest more in our program assessment in order to identify our impact on the community and determine areas for improvement. Cassidy Newberry, former Site Lead and current USF Masters Student in Data Science, will hopefully be assisting with our data collection strategy and analysis.
