Page 1

Community Alliance for Progressive Positive Action: Comprehensive Program Evaluation Report June 2011

CAPPA: Creating Solutions


Community Alliance for Progressive Positive Action: Comprehensive Program Evaluation Report June 2011

Dr. Monique R. Henderson, Co-Director Dr. Marina V. Gillmore, Co-Director The Institute for Educational and Social Justice 594 Sawdust Road, #357 Spring, TX 77380 instituteforedandsocialjustice@gmail.com www.instituteforedandsocialjustice.com Submitted to: Loni Gamble, Executive Director Community Alliance for Progressive Positive Action 1919 Lincoln Drive Williamsport, PA 17701 570-326-7700 lgcappa@comcast.net www.cappaproject.org


Table of Contents

Executive Summary........................................................................2 Introduction and Evaluation Focus.................................................5 Background and Historical Context................................................6 Methodology..................................................................................10 Major Themes................................................................................16 Recommendations.........................................................................31 Conclusions....................................................................................36 References......................................................................................38 Appendices.....................................................................................39 Appendix A: Interview Protocol.................................................39 Appendix B: Observation Protocol.............................................40 Appendix C: Informed Consent Form.........................................41 Appendix D: Photo, Video, and Audio Release Form.................43 Appendix E: Release of School Records.....................................44


Executive Summary

The Community Alliance for Progressive Positive Action (CAPPA) contracted with The Institute for Educational and Social Justice (IFESJ) to assess the overall impact of the CAPPA program on both program participants and the greater Williamsport, Pennsylvania community. This report highlights evaluation results from IFESJ’s comprehensive analysis of interview data, observation notes, and document and archival data review gathered from various stakeholders associated with CAPPA. Key findings are framed in the context of the evaluation questions listed below: What are the overall perceptions of the CAPPA program? Overall, the greater Williamsport community holds the CAPPA program in high esteem. Data collected from key stakeholders confirmed that CAPPA is doing an excellent job of meeting the needs of the young people it serves. Community members outside of CAPPA generally knew of CAPPA and its director and were able to speak to multiple elements of the program and their direct impact on program participants. CAPPA’s positive reputation within the community is a clear asset that has been built over a nine-year span and should be capitalized on as the program strives to move forward in securing funding and

2

other forms of support at the local, state and national levels. What are the key benefits of the program to participants? The program is specifically targeted towards fostering strong character, high self-esteem, and academic excellence in the young people it serves. Results of data analysis indicated that young people who participate in the CAPPA program are making significant improvements in reading and math and developing greater self-concepts, while also learning the benefits of setting goals and working hard to achieve them within a structured environment surrounded by caring, supportive adults. How has the program evolved over the past nine years to meet the changing demands of the community? Overall, the researchers found that CAPPA has expanded and adapted to meet the changing needs of the greater Williamsport community. CAPPA program leaders have recognized when it was time to add additional programming and also have been strengthening the academic components of all programs over time. Additionally, CAPPA is currently well situated to expand program offerings to serve additional youth and parents/ guardians through the opening of its Unity Neighborhood Network Center. The Unity Neighborhood Network Center is located in the hub of the city and has been

CAPPA Comprehensive Program Evaluation Report June 2011


Executive Summary

““The program is specifically targeted towards fostering strong character, high self-esteem, and academic excellence in the young people it serves.”

it serves through solid, engaging, and comprehensive programming. What are strengths?

designed as a multi-use facility with the capability to also expand programmatic offerings to include a health and wellness program. To what extent has the program been successful in accomplishing its mission? The Mission of CAPPA is to “motivate, engage, and support the development of youth. CAPPA further seeks to focus, direct, and empower young idle minds toward good character, high self-esteem and academic excellence. It is [CAPPA’s] mission to enable each participant to attain a vision and to develop personal goals that will move them to become successful, happy, and productive citizens.” An analysis of the data found that, for the most part, CAPPA is successfully supporting the development of the youth

the

program’s

greatest

The premise of CAPPA is that children need inspiration to motivate themselves to have a stronger belief in their dreams and goals and to pursue them with an undying commitment. One of the greatest strengths of CAPPA is that it combines innovative, high-interest programming with a highly structured environment, where rules are enforced and young people can see the benefits of working hard to reach their goals. Overall, it is the relationships that are established between CAPPA participants and leaders that help students to excel academically, while also following program guidelines and becoming immersed in the CAPPA culture. CAPPA’s diverse staff and volunteers understand the students they serve and the backgrounds from which they come and this understanding goes a long way in fostering success. Additionally, the researchers found that CAPPA effectively serves students who are not engaged in other areas, including afterschool programming and recreational youth sports. Many CAPPA students are not involved in other community programs. Without CAPPA programs, these young people would likely be at home or gathering with friends, instead of engaging in structured activities. Some

CAPPA Comprehensive Program Evaluation Report June 2011

3


Executive Summary

young people also would likely be involved in self-destructive activities, according to some of the students and other stakeholders interviewed. CAPPA helps children identify their strengths within a structured and supportive environment, which in turn fosters engagement among CAPPA participants.

program participants, both also on the broader Williamsport community.

What are the program’s greatest opportunities for improvement and growth? CAPPA staff and volunteers are essential to the program’s success. As the program grows, there is a critical need for people who are new to CAPPA to develop a clear understanding of the program’s goals and visions, as well as why the work that they do with young people matters. Likewise, while some parents are engaged in meaningful ways with the CAPPA program, the support of more parents would be highly beneficial and would provide more young people with the support they need. If parents can strengthen their involvement in the lives of their children, CAPPA will have more effective, lifelong success, potentially transforming the lives of multiple generations of children and families. The analysis of the Institute for Educational and Social Justice indicates that the Community Alliance for Progressive Positive Action (CAPPA) has made a substantial positive impact not only on

4

CAPPA Comprehensive Program Evaluation Report June 2011

“CAPPA has made a substantial positive impact not only on the program participants, but also on the broader Williamsport community.”


Introduction and Evaluation Focus

Introduction and Evaluation Focus The Community Alliance for Progressive Positive Action (CAPPA) is nearing its tenth year of operation. CAPPA is committed to motivating, engaging, and supporting youth so that they can “attain a vision and develop personal goals that will move them to become successful, happy, and productive citizens.” CAPPA runs an afterschool tutoring program in two locations, where students receive homework assistance and individualized standardsbased curriculum instruction. Each year, in addition to its after-school program, CAPPA runs a 10-month performing arts program that teaches children music, dance and drama while providing a venue for academic enrichment. The program culminates with a performing arts showcase, where students perform in a world-class theater in front of their families, members of the community and others. Comcast Cable then broadcasts the program on local TV channels, which gives many children the positive recognition that in some case they previously lacked. Additionally, CAPPA facilitates a summer basketball league and juvenile delinquency prevention program. In the spring of 2011, CAPPA contracted with The Institute for Educational and Social Justice (IFESJ) to conduct a comprehensive program evaluation. The purpose of this evaluation was to present a comprehensive picture of the CAPPA program and its impact on participants,

as told through the perspectives of key stakeholders. This report highlights both the program’s strengths and areas for improvement and growth in an effort to increase the effectiveness of the CAPPA program in the coming years. The primary goal of the data collection and analysis process was to inform the following key questions: • What are the overall perceptions of the CAPPA program? • What are the key benefits of the program to participants? • How has the program evolved over the past ten years to meet the changing demands of the community? • To what extent has the program been successful in accomplishing its mission? • What are the program’s greatest strengths? • What are the program’s greatest opportunities for improvement and growth? This report includes evaluation results from IFESJ’s comprehensive analysis of interview, observation, document, and archival data review gathered from program participants, parents, program staff and administration, Board members, community members, school personnel, program volunteers, and other key partners.

CAPPA Comprehensive Program Evaluation Report June 2011

5


Background and Historical Context

The Community Alliance for Progressive Positive Action (CAPPA) is recognized in the Williamsport community as an organization that has grown slowly and steadily, responding to the community’s youth and families as needs and opportunities have surfaced. As one respondent said in an interview for this evaluation, “CAPPA is constantly seeking improvement – it’s dynamic, evolving to meet the needs of the community it serves.” CAPPA got its start in June 2002, when it began operating as a community advisory board. The program started by providing a Summer Youth Basketball League and Youth Empowerment Workshop Series. Since it began, CAPPA enrollment has fluctuated. Table 1 delineates enrollment numbers by both program and year. These numbers were pulled from a review of archival data and documents provided by CAPPA and were based on self-reported data. For a student to be considered “enrolled,” he or she needs to have attended the program at least 50% of the time and have a valid registration form on file.

6

Table 1: Yearly Program Participant Enrollment Numbers Summer Youth Basketball / Juvenile Delinquency Prevention Workshop Year Total Enrolled 2002 60 2003 92 2004 121 2005 76 2006 177 2007 168 2008 128 2009 147 2010 106 Academic Enrichment/Performing Arts Showcase/ Juvenile Delinquency Prevention Year Total Enrolled 2005 120 2006 131 2004 121 2005 76 2006 177 2007 168 2008 128 2009 147 2010 106

CAPPA Comprehensive Program Evaluation Report June 2011


Background and Historical Context

“While expanding its offerings, CAPPA leaders also have taken steps to ensure that the quality of program offerings is being strengthened.�

the local school district have said they would like to be able to afford for use by all students.

In September 2004, CAPPA became a fully incorporated 501(c)(3) non-profit organization. Then, in 2005, the program expanded to begin providing additional academic enrichment and performing arts opportunities. The CAPPA Performing Arts Showcase was first held in 2005, providing youth with the opportunity to showcase their talents in dance, singing, acting, writing and other areas. The showcase is now in its seventh year and is widely supported by and well attended by the community. While expanding its offerings, CAPPA leaders also have taken steps to ensure that the quality of program offerings is being strengthened. CAPPA uses evidence-based curriculums including Get Real About Violence and PLATO Learning Achieve Now, a program that representatives of

When the program started, the only employee was Loni Gamble, who serves as executive director. Since then, the program has added the two full-time positions of an executive assistant (added in 2008) and an administrative assistant (added in 2007). Additionally, CAPPA has a board of directors, an academic enrichment coordinator, a team of curriculum instructors, and a program coordinator. Holly Doyle, who serves as the executive assistant, and Carolyn E. Perry, who serves as the administrative assistant, both work out of the main CAPPA office with the executive director, Loni Gamble. The other staff members work primarily out of the two after-school Neighborhood Network Centers and on-site at the Saturday program, which is housed in the cafeteria/multi-purpose room of a local elementary school. Table 2 highlights key CAPPA staff positions and their responsibilities, according to self-reported data provided by CAPPA. It is expected that once the Unity Neighborhood Network Center opens, the location of offices and programs will be shifted around as needed.

CAPPA Comprehensive Program Evaluation Report June 2011

7


Background and Historical Context

Table 2: Key Positions and Job Descriptions Position Title Essential Functions of Job • Supervises staff and volunteers in the day-to-day operation of CAPPA Executive • Assists in the implementation of programs and services Director

• Prepares reports for funding partners • Seeks new financial resources by creating new community collaborations • Participates and represents CAPPA at community outreach meetings

Executive Assistant

Administrative Assistant

8

• • • • • • • • • •

Supervises staff and volunteers in the day-to-day operation of CAPPA Assists in the implementation of programs and services Prepares reports for funding partners Seeks new financial resources by creating new community collaborations Participates and represents CAPPA at community outreach meetings Supervises volunteers in the day-to-day operation of CAPPA Assists in the implementation of programs and services Prepares reports for funding partners as directed by Executive Director Schedules the Executive Director for community outreach meetings Attends community meetings and prepares meeting minutes

Curriculum Instructor

• Supervises participants in the CAPPA Neighborhood Network Center’s programs and activities • Administers pre tests and enters the data into program software • Utilizes data to develop instruction and administer the PLATO Learning Achieve Now curriculum • Prepares weekly report for Executive Director and Executive Assistant • Attends CAPPA staff meetings

Program Coordinator

• • • • •

Supervises students in the CAPPA programs and activities Assists in the implementation of programs and services Prepares monthly report for Executive Director and Executive Assistant Prepares schedule of CAPPA public service announcements Attends CAPPA staff meetings

CAPPA Comprehensive Program Evaluation Report June 2011


Background and Historical Context

Through the years, CAPPA has received extensive support at the state and federal level. This support has come from the U.S. Department of Justice’s Office of Juvenile Justice Delinquency Prevention, U.S. Senator Robert P. Casey Jr., U.S. Representative Christopher P. Carney and State Senator Gene Yaw, State Representative Rick Mirabito. Extensive support also has been received from the Lycoming County Board of Commissioners, the City of Williamsport, and Williamsport Mayor Dr. Gabriel Campana. The CAPPA program works in conjunction with many community partners. CAPPA’s Academic Enrichment, Performance Arts Showcase, and Juvenile Delinquency Prevention project are conducted in partnership with the Williamsport Area School District, Lycoming County Housing Authority, District Attorney Eric Linhardt, Lycoming County Children and Youth Services, and James V. Brown Library, among others. The Academic Enrichment, Performance Arts Showcase, and Juvenile Delinquency Project are all part of a ten-month program that uses research-based curriculums administered by certified public school teachers and dedicated volunteer tutors/mentors from local area colleges after school Monday through Thursday from 3-6 p.m. and on Saturdays from 10 a.m.-3 p.m. The tenmonth program culminates each year in June with an Annual Performing Arts Showcase featuring CAPPA participants at

the Community Arts Center. This showcase is professionally taped and televised throughout the summer months on local television to over 90,000 cable subscribers. According to a recent grant proposal completed by CAPPA, since 2005 CAPPA has “developed an aggressive marketing campaign that consists of print media, electronic media, posters, flyers, cable television commercials, and multiple cable television broadcasts of the Annual Showcase.”

“Throughout the years, CAPPA has received extensive support at the state and federal level.”

CAPPA Comprehensive Program Evaluation Report June 2011

9


Methodology

For the purpose of this evaluation, the data collection used in this evaluation and Institute for Educational and Social Justice the purpose of each type. (IFESJ) gathered and analyzed qualitative data from multiple stakeholders according to standard case study procedures and augmented this data with a review of archival quantitative data. The IFESJ chose Table 3: Data Collection Types and Purposes qualitative case study as the primary Type of Data Collection Purpose of Data Collection methodology because the purpose of Semi-structured, oneTo allow key stakeholders to this program evaluation was to present on-one interviews explore their own perceptions a comprehensive picture of the CAPPA with program staff/ and experiences as they relate program and its impact on participants, administration, to the central questions being as told through the perspectives of key participants, parents, studied. stakeholders. Thus, the IFESJ sought to and other key both address descriptive questions and stakeholders produce a firsthand understanding of the Direct program To triangulate interview data CAPPA program (Yin, 1994). observations of afterand gain greater insight into school program, the context of both the CAPPA Qualitative research was the most Saturday program, and program’s participants and its suitable methodology for this research year-end showcase stakeholders because it allowed for an in-depth and Community walkabouts To better situate the CAPPA comprehensive study of the knowledge program within the context of the and experiences, both lived and observed, broader community and to glean of the participants. As a form of inquiry, additional community members’ qualitative research is a field in its own right perceptions of CAPPA (Denzin & Lincoln, 2005) and “represents a legitimate mode of social and human science exploration, without apology or comparisons to quantitative research” (Creswell, 2007, p. 11). The IFESJ conducted one-on-one semi-structured interviews and then triangulated this data with direct observations (of the after-school program, Saturday showcase program, and end-ofyear showcase), document and archival data review, and community walkabouts. Table 3 summarizes the different types of

10

CAPPA Comprehensive Program Evaluation Report June 2011


Methodology

Data Collection Methods Interviews were conducted over a oneweek period in the spring of 2011 and were conducted by the IFESJ without the presence of CAPPA administrative staff. Comprehensive interview notes were taken using an interview protocol (see Appendix A) that was created by the IFESJ for the purpose of this evaluation. The IFESJ interviewed a wide variety of key stakeholders, including program staff and administration, program participants (in both the after-school program and CAPPA showcase), parents of program participants, board members, community members, program volunteers, school personnel and other key people who have interacted with CAPPA in a variety of capacities. The IFESJ also conducted observations of the after-school program, the Saturday program, and the CAPPA year-end showcase. The purpose of the observations was to triangulate the interview data and gain greater insight into the context of both the CAPPA program’s participants and its stakeholders. Observations in natural settings, which in this case were the after school centers, the Saturday showcase rehearsals, and the year-end showcase performance, were documented via observation protocols. The IFESJ used open-ended narrative notes and recorded both descriptive and reflective entries. The IFESJ was also careful to record aspects such as the physical setting, interactions among people, events and activities,

and the researchers’ reactions to the observations (Bogdan & Biklen, 1992) and used this data to help establish a context for the case. The IFESJ also reviewed archival data and documents to further triangulate the interview and observation data. The documents reviewed included recent grant applications, reports generated for accountability and funding purposes, letters of endorsements, newspaper articles, and self-reported quantitative data. Additionally, the IFESJ conducted community walkabouts to better situate the CAPPA program within the context of the broader community and to glean additional community members’ perceptions of CAPPA. The collected data helped the IFESJ develop general trends and themes related to both the program’s structure and effectiveness and provide recommendations for CAPPA going forward. Description of data sources Data sources used for the purpose of this evaluation varied in both breadth and depth in order for the IFESJ to present a comprehensive analysis of the CAPPA program. Twenty nine program participants, ranging in age from four to fifteen, were interviewed. Some participants only participated in the after-school program, some participants only participated in the Saturday showcase program, and a limited

CAPPA Comprehensive Program Evaluation Report June 2011

11


Methodology

number of participants participated in both the after-school and Saturday programs. Some had also participated in the Summer Basketball League, the Juvenile Delinquency Prevention Workshops, and the CAPPA Young Gentlemen (YGs). Nine parents were interviewed with varying levels of involvement with CAPPA. The involvement level ranged from parents who volunteered every Saturday at the showcase program to parents who sent their children to the after-school program with minimal involvement. Eight CAPPA staff and community volunteers were interviewed. The different roles of the staff and volunteers ranged from teachers who taught in the afterschool program every Monday through Thursday throughout the school year to community volunteers who participated with various levels of involvement in the Saturday showcase. Nine additional key stakeholders were interviewed as well. These ranged from local school administrators, CAPPA board members, community business people, local government officials, collaborating program staff and administrators. Interviews with program administration were used for the purpose of confirming findings and clarifying the accuracy of some programmatic elements. These interviews varied from the other interviews in that they were informal in nature and

12

“The collected data helped the IFESJ develop general trends and themes related to both the program’s structure and effectiveness.�

were used as secondary, and not primary, sources of data. Description of sampling procedures All research was conducted in the natural setting of the Williamsport community. Interviews with community stakeholders were all conducted in their respective places of business. Interviews with parents, program participants, and volunteers were conducted at the afterschool centers and the site of the Saturday showcase program. Characteristic of qualitative research, data collection in natural settings was important because it granted the IFESJ face-to-face interaction with participants and the ability to gather data by talking directly to participants and seeing them behave and act within their contexts (Creswell, 2007). Prior to the start of data collection, informed letters of consent were given to all program participants and their parents

CAPPA Comprehensive Program Evaluation Report June 2011


Methodology

(see Appendix C for Informed Consent Letter). Additionally, the IFESJ gave a verbal description of the project prior to each interview and made clear that all data gleaned was for the purpose of this program evaluation and that participants’ names and other identifying factors would not appear in the final write-up, nor would the raw data with identifying factors be made available to anyone outside of the IFESJ staff. All program participants interviewed also had two additional release forms on file (see Appendices D and E for Release Forms).

“During the observations, the researchers... interacted only casually and non-directly.”

During each interview, both IFESJ researchers took copious interview notes on a pre-designed interview protocol. Although guiding questions were used, the IFESJ asked additional questions as needed. The wording of questions was also modified to ensure interviewee understanding. For example, when interviewing younger participants (as young as four years old) IFESJ researchers asked shorter questions, provided ample

time for follow-up clarification, allowed the parents to sit in on the interview if requested, and encouraged participants to draw or act out responses in lieu of verbal responses if they so chose. Direct observations of the after-school program, the Saturday showcase rehearsal, and the end-of-year showcase were documented via field notes on a pre-designed, open-ended observation protocol (see Appendix B). The observations were used to apply the researchers’ own “knowledge and expertise in interpreting what was observed and to… record behavior as it was happening” (Merriam, 1998, p. 96). During the observations, the researchers acted as participant-observers and interacted only casually and nondirectly while conducting the observations (Gall, Gall, & Borg, 2007). Document and archival data review was conducted before, during, and after other sources of data collection. Documents were reviewed to either confirm or refute

CAPPA Comprehensive Program Evaluation Report June 2011

13


Methodology

data collected through primary sources of data collection. Additionally, self-reported quantitative data from prior self-studies was looked at to further triangulate the primary data sources of this evaluation.

for the readers” (Eisenhart, 2006, p. 570). In other words, by representing data in a variety of ways, the researchers were able to present a more comprehensive picture of the evaluation as it was conducted.

Community walkabouts were conducted informally over the course of a week. Casual conversations took place between researchers and community members in locations such as local restaurants, hotel lobbies, community gathering places (movie theatres, parks) and local businesses. Directly after each community walkabout, the researchers documented the data collection via reflective field notes. The purpose of these walkabouts was to better situate the CAPPA program within the context of the broader community and to glean additional community members’ perceptions of CAPPA.

Once the interviews were conducted, the IFESJ used inductive data analysis, analyzing themes from the bottom-up and organizing data from more concrete to increasingly abstract units of information (Creswell, 2007). The data were analyzed through the theoretical lens of critical pedagogy and through using interpretive inquiry (Creswell, 2007) to interpret what was heard, seen, and understood.

Description of data analysis procedures Creswell (2007) identified data analysis in qualitative research as the process of first organizing data for analysis, then coding the data and condensing the codes in order to produce emergent themes, and then finally displaying the data for representation through figures, tables, and/or discussion. Eisenhart (2006) maintained that data could be presented literally by reporting exact quotes, figuratively by offering an impression of what participants said, and dialogically by repeating exact conversations as they occurred. Such three-fold presentations can be used to “evoke fieldwork experiences

14

After each data collection session, the IFESJ researchers reviewed their field notes and took further notes with any additional impressions, thoughts, questions, and/or emergent thematic patterns. Data were triangulated to check the perceptions of the key stakeholders themselves (revealed through interviews) against observations, document and archival data review, and

CAPPA Comprehensive Program Evaluation Report June 2011

“The data were analyzed through the theoretical lens of critical pedagogy.”


Methodology

community walkabouts. Not all information is used in qualitative analysis, and some may be discarded (Creswell, 2007); this process is referred to as winnowing. After the researchers coded their notes, they further classified them, which involved looking specifically for categories, themes, or particular dimensions of information (Creswell, 2007). The results were analyzed through cross-case analysis (Borman, Clarke, Cotner, & Lee, 2006; Yin, 1994). The major themes that emerged can be found in detail in the next section. Data were also used to make

Table 4: Key Findings Data Collection Procedure Semi-structured, one-onone interviews Direct observations

Document and archival data review Community walkabouts

programmatic recommendations to CAPPA as the organization moves forward and expands in both breadth and depth. Table 4 highlights the key findings that emerged, delineated by type of data collection procedure.

Key Findings Based on an analysis of the stakeholders’ interviews, key findings can be represented by nine key themes related to community perceptions, program structure, effects on participants, strengths, and needs. Observations were analyzed to determine what elements of the program did – or did not – align with interview details. Observations revealed that, for the most part, program practices aligned with key stakeholders’ perceptions of the program. The observation data confirmed primary data obtained via interviews. Document and archival data review served to confirm the findings from the interview and observation analysis. Analysis of the community walkabout data also confirmed primary data. Most community members were familiar with CAPPA and could speak to its importance in the community. Likewise, widespread support for and visibility of the CAPPA showcase was seen in the form of fliers, billboards, and other promotional material throughout the community.

CAPPA Comprehensive Program Evaluation Report June 2011

15


Major Themes

The major themes identified below emerged relative to the data analysis process previously discussed. Table 5 highlights the major emergent themes and examples of these themes as evident in the interview data. A comprehensive discussion of each theme follows. Table 5: Emergent Themes and Examples Coded Theme Examples from Interviews 1: Community “What they are doing is not stale. They are always trying to figure out a need perceptions of CAPPA and going after it. They do a good job of not letting go of their mission – their backbone – but also being flexible.” 2: Strengthening of “I try to build their confidence – let them know they have a future based on Self-Esteem the decisions they are making at this time… I tell them they are people that can change the world. I want them to know that. They have to think that and say that and believe that – even if everyone around them doesn’t believe that.” 3: Involvement of “The dedication of the staff is phenomenal. Everyone – the staff, the volunteers – they always go above and beyond for the students. How many volunteers do they Caring Adults have who get up early every single Saturday morning to be there for the kids? Seeing that kind of dedication is encouraging.” 4: Emphasis on Structure 5: Motivation for Academics

6: High Expectations for Participants 7: Perceptions of the Need for Programs 8: Inclusion as a Strength

9: Need for Stronger Parent Connection

16

“I know they had good programs here and I knew they were not just babysitting here. They have good programs for their homework and learning and they have a good bit of discipline.” “(The students) have an opportunity to do schoolwork on Saturdays, but for most of them the real value is in performing and being involved in the showcase. They know they are going to focus part of their time on academics. But they know it will not be drudgery. It will not be overwhelming for them, but it will help them to fill some of those academic gaps.” “CAPPA does try to help the kids to get along. But they also draw the line and say, ‘These are the rules and you have to follow them.’ You don’t get a whole lot of chances to disrupt the program. They try to understand but also know they have to draw the line.” “A lot of times kids go home and wander the neighborhood if they are not involved in something like this. This is something positive – constructive… Our community needs this.” “When you go to the showcase, it’s pretty obvious that some of the kids do have handicaps. Some of them are in special education. But they are given the opportunity to perform. They might not be in sync and they might not have all the steps down but they are up there and they are truly participating – they are part of something. They have achieved what it takes to get on that stage in a world-class theatre. It’s impressive and they should be proud of themselves. I think, watching them, that they are.” “You can do all you want with the children but if you are not changing the child’s environment, the child goes back into that environment and it is very difficult to change; it seems like that’s the next step – they have figured out how to engage the kids; but how do you engage the parents?”

CAPPA Comprehensive Program Evaluation Report June 2011


Major Themes

Theme 1: Community Perceptions of job of not letting go of their mission – their CAPPA backbone – but also being flexible.” The Greater Williamsport Community seems to have a positive perception of CAPPA programs and CAPPA leadership. This positive perception was clear in extensive formal interviews with key community stakeholders, as well as in informal conversations with members of the community while conducting community walkabouts, where the IFESJ researchers spoke to people casually about whether they were familiar with CAPPA, and what they thought of the work being done.

While CAPPA leadership is viewed as being innovative and changing as youth and the needs of the community shift, CAPPA leaders also were praised for their consistency. Members of the community, including educators and other community partners, said they felt everyone from the executive director to college-aged volunteers were dedicated to the work they were doing, and to doing what they could to meet the needs of the children and teens in the program. One educator familiar with CAPPA said, “They have been consistent. They are there ever Saturday Several community members familiar and on weekdays.” with CAPPA said they appreciated the organization’s leadership for growing the Community leaders, including program program slowly and thoughtfully. One administrators and educators, repeatedly person familiar with CAPPA for a number said they view CAPPA staff as “good of years said, “I think they are going in partners.” School district officials said they the right direction. They have expanded were particularly pleased with CAPPA’s slowly based on opportunity and need. interest in aligning program standards with They go where technology and events take the state curriculum, which is also aligned them. [Executive Director] Loni [Gamble] with the state testing system. The district is a very good promoter. He creates those is very supportive of CAPPA’s use of PLATO opportunities for growth.” software, which officials said they could not afford to purchase for large numbers CAPPA leaders also were praised for of students. “Because of CAPPA and that being focused, but also responding to cross-communication between CAPPA and changing times and community needs. the district, some of our students now One community leader said he viewed have access to that software while they leaders as “proactive.” A key community are at CAPPA.” partner said, “What they are doing is not stale. They are always trying to figure out Several community leaders indicated a need and going after it. They do a good that CAPPA staff return phone calls

CAPPA Comprehensive Program Evaluation Report June 2011

17


Major Themes

promptly and also complete all requested paperwork efficiently and thoroughly. One key community partner said, “They have a reputation in this community for being good partners. If you ever need anything, they are there – they always get me what I need, when I need it. Really, they get it to me even before I need it. That’s one of the biggest reasons that we like working with them.” As the IFESJ conducted community walkabouts, informally visiting businesses, restaurants, tourist attractions and other venues within the City of Williamsport and surrounding areas, the researchers were repeatedly struck by how many people were familiar with CAPPA’s efforts to provide after school and Saturday programming to area youth. The IFESJ encountered very few local residents who were not familiar with the organization and its showcase program. Most of the businesses the IFESJ visited also had flyers, posters and other materials promoting the upcoming showcase, which was scheduled a few weeks after the IFESJ’s initial visit. Theme 2: Strengthening of Self-Esteem The idea that the work CAPPA is doing enhances the self-esteem and general confidence of the young people it serves was one of the clearest themes that emerged from the research. The theme was expressed either directly or indirectly by a number of the participants, as well as by educators, parents and program volunteers. A number of the comments

18

“The work CAPPA is doing enhances the self-esteem and general confidence of the young people it serves.”

about self-esteem focused on the idea that the opportunity to perform at the CAPPA showcase provided participants with a level of confidence that they had not previously known. Several of the young people who participated in the showcase mentioned that they thought the quality of the show surprised the people in the audience, including family members and key community leaders. One 9-year-old girl who has performed in the showcase, when talking about her experience performing on stage, said, “I felt like I was a star. People acted surprised because kids were going out and doing something really good – not just sitting in the house all day. They saw that kids were

CAPPA Comprehensive Program Evaluation Report June 2011


Major Themes

doing things like learning to sing and dance and be up on stage.” A 12-year-old male student who has participated in showcase said he has become more confident in his social interactions since joining CAPPA. He noted, “I am more open to speaking to people. I used to keep to myself mainly.” He said that getting on stage at the showcase made him nervous, but that fighting through his jitters was worth it: “My family was there, screaming my name. And after it felt good – really good.” Leaders who work directly with the children also made it clear that they believe building self-esteem is a top priority of theirs – and of the general CAPPA program. One employee, who works with students in the after-school program, said, “I try to build their confidence – let them know they have a future based on the decision they are making at this time… I tell them they are people that can change the world. I want them to know that. They have to think that and say that and believe that – even if everyone around them doesn’t believe that.” Efforts to boost self-esteem also can be seen in the environment of after-school programs. At one site, for example, students are greeted at their work stations by a poster that reads, “You are valuable. Don’t let anyone make you believe differently.”

self-esteem strengthened over time. The educator said, “I see some students that have been in the program religiously. And every year you just see their leadership and confidence is rising. Preparing for the showcase and stepping out on that stage gives kids confidence. You can just see them being built up, right there in front of you.” Several people familiar with CAPPA offerings, as well as extracurricular offerings available through schools, youth sports and other organizations, said they felt that CAPPA built self-esteem in young people who might not feel a strong sense of belonging anywhere else. One community partner explained, “These kids – so many of them are not doing well in school. And they are not sports oriented or they do not have the family supports to be involved in something like that. CAPPA gives them an avenue to explore – a place where they can feel good about themselves.”

The idea that CAPPA builds self-esteem by providing young people with opportunities for self-expression also was mentioned by several of the people who were interviewed. One school district leader said, “What we do here does not give kids a lot of room for self expression. We spend hours on reading and math. But there’s not a lot of room for self-expression. One educator who has worked with CAPPA reinforces the academics but also children also enrolled in CAPPA programs provides other outlets – outlets that kids said he has seen evidence that the need.” This data supported previously children who attend CAPPA have their collected self-study survey data that found

CAPPA Comprehensive Program Evaluation Report June 2011

19


Major Themes

72% of Saturday Academic Enrichment/ Performance Arts Showcase participants felt they were more sure of themselves because they attended CAPPA. Also, 74% of surveyed participants felt the leaders of CAPPA were there for them and listened to them when they talked.

that 50% of program participants exhibited an improvement in Social Competence from the first marking period to the final marking period.

Community leaders also touched on the idea that CAPPA may help young people to feel a stronger connection to the overall community. One program administrator said, “Many youth today do not feel connected to their communities. But this program gives them a way to express themselves. It helps them to feel good about themselves and it allows them to interact with the community in ways that are very positive... They are very much supported by the community when they are part of CAPPA.”

One community partner became teary eyed while talking about the level of care CAPPA volunteers and staff show to the young people who participate in Saturday, after-school and summer programs. The partner said, “The dedication of the staff is phenomenal. Everyone – the staff, the volunteers – they always go above and beyond for the students. How many volunteers do they have who get up early every single Saturday morning to be there for the kids? Seeing that kind of dedication is encouraging.”

Theme 3: Involvement of Caring Adults

Another theme that was consistently Some of the individuals who were expressed by interview participants is the interviewed suggested that CAPPA importance of caring adults to the success provides praise and support to children of the CAPPA program. Parents, students, who may not get affirmation other places. community partners and educators One parent volunteer, who expressed were among those who praised CAPPA frustration over what she sees as an overall employees and volunteers for the level of lack of parental involvement, said, “CAPPA caring they demonstrate when working gives the child so much. It builds their self- with young people. Volunteers, employees esteem. Some of these children are picked and program leaders also expressed on and talked about. But CAPPA empowers their belief that the young people who them to be more – more than what their participate in CAPPA need to know they parents tell them, which, sad to say, is that are cared about, supported, and even loved. they are nothing.”

The data collected for this evaluation One mother of a student in CAPPA’s afterconfirmed prior self-study data that school program said she has seen how indicated (for the 2010-2011 school year) much the program’s teachers care about

20

CAPPA Comprehensive Program Evaluation Report June 2011


Major Themes

“The young people who participate in CAPPA need to know they are cared about, supported, and even loved.”

the children they serve. “They are very nurturing and take out extra time with the kids to bring them into themselves. They are a child – not just a number. They are not a number in a slot. They matter.” The mother went on to talk about the deeply personal way she believes the adults interact with the children, saying, “They are full of hugs. The kids feel they can talk to them.” One of the teachers in the after-school program, who smiled almost the entire time she talked about the program and “her kids,” talked about how important she thinks it is that the children know she cares. She said, “I am most proud that I treat them like my kids. I try to make them comfortable... They need someone that cares about them no matter what. They tell me what they did wrong at school even though they don’t have to. They open up. They know I don’t judge them for what

they do.” The teacher said that in time, she learns to read the mood of the children she works with and to be proactive in helping them to solve problems. One six-year-old in the program said of the teacher, “She’s nice. She’s just like my mother. This is like my home. People call you smart here and my mom says I am smart.” Several of the children in the afterschool program described the patience and understanding of the tutors and teachers. One 11-year-old girl, for example, said, “The people are really nice... If you don’t know a word, they look it up. And they help you on the computer.” Another 11-year-old said of the tutors, “They don’t yell. They walk around and help people.” One teen said he was impressed by the environment of the after-school program, particularly the level of respect the adults show. He said, “It’s a good environment.

CAPPA Comprehensive Program Evaluation Report June 2011

21


Major Themes

You are around people who respect you… They don’t make fun of you or talk behind your back or anything.” This data confirmed previously reported self-study survey data of after-school participants that found 70% of them said CAPPA helped them learn how to make good choices and 100% of them said that CAPPA taught them how to treat people with respect. The volunteers interviewed for this report expressed a sense of love and care for the students. One mother of two, who also volunteers, said, “We really love these kids and they know that. They know that we care about them. We show them that by taking an interest in them here at CAPPA and in their lives. We believe in them – they need that.” Several interview participants expressed the idea that, while the youth involved in CAPPA are benefitting from the program, the volunteers also seem to be getting needs met. One community partner said, “The mentors must be getting a whole lot out of it. This program must pick up the mentors as much as it picks up the kids because they stick with it. They must feel great as they are building those relationships.” One program worker said she felt a strong sense of responsibility to be there for the children, especially since many of them do not have other caring adults in their lives. She said, “I believe in being here for the children. A lot of them don’t have the basics. We just give hugs to them – love

22

them. They need us to tell them we believe them and we love them. They need to hear that. They don’t always hear it from their parents.” One volunteer, who got involved in the program after meeting the program’s executive director at a community gathering place, said the program helped him to feel more connected to the community and to young people. “This gets me out of the house and interacting with people,” he explained. He said he is not sure that his skills are particularly suited to the program, but he believes the students know that he cares. “If they [the students] have a problem, I want to be there for them. There is nothing glamorous about it. But I like to try to help them in a positive way as much as I can.” Theme 4: Emphasis on Structure During site visits to the program centers, as well as interviews with program participants, volunteers and other leaders, it became clear that CAPPA places a strong emphasis on providing children with a structured, predictable environment. Schedules are displayed prominently on the walls, and students who regularly attend programs can explain, step-bystep, how the day-to-day routines work. Students clearly know that when they come in the door in the after-school program, for example, they are expected to sign in and put their coats and backpacks away. They also know that they are expected to complete specific jobs and that specific

CAPPA Comprehensive Program Evaluation Report June 2011


Major Themes

“CAPPA places a strong emphasis on providing children with a structured, predictable environment.”

times are set aside for homework time and computer use.

reasoning behind it. They need to know why we do what we do.”

One after-school teacher said she thinks having jobs for the students goes a long way in creating a sense of safety and belonging in the program. She added, “I give them responsibilities – make them responsible for their space… They know their responsibilities and what is expected. It works so smoothly when we have the structure. We try to help them get the

During our visit to the Saturday program, students were practicing for the upcoming Showcase performance. Little time was wasted, and students knew they were expected to remain seated, coloring, doing schoolwork or watching quietly when they were not involved directly in the practice. One community leader, who has seen the Saturday program several times, said, “It’s very impressive. You will have 75 to 100 youths and they don’t say a word when he [Executive Director Loni Gamble] is talking. They know the expectations that Loni and his staff have and they meet those. They know to give him respect and to follow the rules.” One school administrator said he thinks CAPPA’s highly structured environment sets it apart from some other programs. He said, “The way they are expected to behave in CAPPA is very similar to how they are expected to behave in school... It’s not every man for himself. There are expectations.” One mother said she was drawn to CAPPA’s after-school program in part because of its highly structured environment. She said, “I know they had good programs here and I knew they were not just babysitting here. They have good programs for their homework and learning and they have a good bit of discipline.” One of the teachers involved in the program said providing

CAPPA Comprehensive Program Evaluation Report June 2011

23


Major Themes

structure is one of the ways that she helps the students to feel safe and nurtured. A teenage student said he liked that the program allowed him to get his homework done “without distractions.”

do schoolwork on Saturdays, but for most of them the real value is in performing and being involved in the showcase. They know they are going to focus part of their time on academics. But they know it will not be drudgery. It will not be overwhelming for Several children said they knew that if they them but it will help them to fill some of did not follow the rules, they might be those academic gaps.” removed from the program – an idea that also was expressed by several community One mother of a student in CAPPA’s afterleaders. Still, the participants also seemed school program said she appreciates to know that if they made a mistake, they CAPPA’s use of gaming technology to help would have opportunities to return to good teach children information they will need standing in the program. One eleven-yearold girl, for example, said, “The teachers treat us good. If we do something bad “Still, the they give us another chance. They want participants us to be able to have fun and stuff.” seemed to know that if they made Theme 5: Motivation for Academics a mistake, they would have One of the key premises of CAPPA is opportunities to clearly that engaging some young people return to good in tutoring and other academicallystanding in the oriented programs may be challenging – program.” but young people are often more eager to participate in such activities if there are paired activities they find more entertaining, such as basketball, drama, dance or music. Key partners, educators and others seemed to understand this concept and to regard it as effective. One longtime educator in the Williamsport area said, “We can have a Saturday morning tutoring program, but how many students do you think are going to get up and come to that?” He said that he thinks CAPPA’s approach is an effective one: “[The students] have an opportunity to

24

CAPPA Comprehensive Program Evaluation Report June 2011


Major Themes

to be successful in school – an idea that also was mentioned by others. “To them it’s games but we know it’s more than that. We know that they are learning a lot. It’s like a full serving of vegetables but they don’t know it.” Some students did recognize the academic benefits of CAPPA as well. One nine-year-old student said, “They basically get you ready for next year ….” Posters at one of the after-school sites reinforced the idea that CAPPA programs are about both learning and having a good time, declaring the site “a place where learning is fun.” Community partners credited CAPPA leaders with being in touch with the interests and needs of the area’s young people and providing programming that grabs their interest and keeps them engaged. One partner said, “They find activities that hook the kids somehow – and the performing arts does seem to be very appealing to them… The hope is that the kids get on stage and learn and enjoy themselves. But staying in school and doing well is the bigger goal. The stage is just the mechanism.”

Theme 6: High Expectations for Participants Community leaders repeatedly said that just as the program is highly structured, there also are high expectations for the behaviors and work of the students who are involved. One community partner explained the approach of program leaders by saying, “[The program leaders] hold the students to a high standard. So often, people are inclined to say, ‘They are just kids. You can’t expect them not to use foul language or not to have their pants sagging down.’” In the observations that we did at the after-school sites, as well as the Saturday program, it was clear that students understood the behavior expectations and seemed to follow them. Students spoke respectfully to CAPPA staff and volunteers, behaved in an orderly way and did not tease or bully each other. When children do need reprimanding, students, parents and program leaders all said an effort is made to ensure that students understand why specific rules are in place, and why they must be followed. Students are given opportunities to apologize to anyone they have wronged when they do

“Community partners credited CAPPA leaders with being in touch with the interests and needs of the area’s young people.”

CAPPA Comprehensive Program Evaluation Report June 2011

25


Major Themes

find themselves reprimanded. One school district official explained, “CAPPA does try to help the kids to get along. But they also draw the line and say, ‘These are the rules and you have to follow them.’ You don’t get a whole lot of chances to disrupt the program. They try to understand but also know they have to draw the line.” One other program volunteer said, “They [CAPPA staff] acknowledge when they [the participants] do wrong but also when they do right… It’s a very positive atmosphere.” One key community partner who sees CAPPA children outside the program said she thinks the program’s high expectations help participants to behave well not just at CAPPA, but in the community at large. “You see such a change in some of the kids because that is what is expected. At the show, you see young men in tuxedos holding open doors and saying, ‘Yes ma’am,’ – using manners. They know not to misbehave because that’s not something you do at CAPPA.” One 11-year-old student agreed, saying, “They have taught me how to be respectful. It has changed me. It has changed the way I talk ….. And also how I act.”

“The program’s high expectations help participants to behave well not just at CAPPA, but in the community at large.”

come from low-income families. Some stakeholders stated that the need for CAPPA programming was so great that they would like to see the program expand its offerings, assuming adequate funding and other necessary resources could be secured.

While some students involved in CAPPA are involved in other extracurricular activities, including those provided through local churches and public schools, many also said that they were not involved in other activities. Some said Theme 7: Perceptions of the Need for that if they were not involved in CAPPA Programs during after-school hours or on Saturday, they would likely be at home watching Key stakeholders, including educators, television or playing with other children community partners and parents, all in the neighborhood. seemed to agree that the Greater Williamsport area has a tremendous One student involved in CAPPA’s Saturday need for the programming that CAPPA programming said, “I’d probably be hanging provides, particularly to students who out with my friends. And sometimes that gets me in trouble, especially if they are

26

CAPPA Comprehensive Program Evaluation Report June 2011


Major Themes

not making good choices and then I am there with them. And if I wasn’t out with my friends I’d just be at home watching TV or playing video games – not really doing anything… I like CAPPA because it gives me something to do besides be bored or get into trouble.” Several members of the community, including parents who volunteer with CAPPA, said they think it is difficult for many families to have their children involved in other activities. One school district official said that for some families, money is the only barrier to access. In other families, there is a lack of the resource of time, because parents are working multiple jobs to make ends meet and cannot take their children to events, such as baseball practices, soccer practices or dance classes. Single parents also may feel they have less time and energy to have their children involved in activities, one school “The greater Williamsport area has a tremendous need for the programming that CAPPA provides.”

district official suggested. Transportation also is an issue, especially since many of the most popular extracurricular activities in the city are offered on the outskirts of the city, a distance from the inner-city area where many low-income families reside. One school administrator said, “Parents are more focused on meeting those basic needs – on doing what it takes to just put food on the table.” The same school administrator also said that some parents simply do not have experience with afterschool activities and do not see their importance or know they are available. “Some parents don’t have the knowledge base,” he said. “People tend not to get involved in or get their kids involved in things they know nothing about.” One community leader spoke to the idea that CAPPA “fills the gaps” left when parents and other family members are unable or unwilling to engage children in positive, meaningful activities beyond school hours. “CAPPA kids are not involved in a lot of different activities,” he said. “A lot of times kids go home and wander the neighborhood if they are not involved in something like this. This is something positive – constructive... Our community needs this.” One community partner familiar with the neighborhoods where many CAPPA participants live said that before CAPPA started, there were more children roaming aimlessly in the neighborhood with little direction or supervision: “You had a

CAPPA Comprehensive Program Evaluation Report June 2011

27


Major Themes

bunch of eight-year-olds running around unattended. Parents were working or just didn’t care. There was a lot of criminal mischief – just kids being kids with no one supervising them. You saw some serious problems with the middle school kids... I don’t know what our kids would do without CAPPA. I have seen that it makes a difference.” While CAPPA is generally a structured, predictable place, volunteers and CAPPA employees who work with the children said the home lives of some of the CAPPA students can be filled with daily stressors that impact healthy social emotional development. One woman who works closely with students in a CAPPA afterschool program said: “These kids have a lot of worries and responsibilities. They have it worse than we adults do... Some worry whether they are even going to see their parent that day. They shouldn’t have these kinds of worries.” A mother who volunteers in the Saturday program agreed with that idea, saying, “If not for CAPPA, some of these kids would not get breakfast or lunch on Saturday. They wouldn’t get hugs or encouragement. They wouldn’t get anything.” Theme 8: Inclusion as a Strength During visits to the afterschool program sites and to the Saturday program, it was clear that CAPPA has made an effort to be highly inclusive, making a place for

28

“CAPPA has made an effort to be highly inclusive...”

students with a variety of academic, social, emotional and physical needs. One school administrator observed, “When you go to the showcase, it’s pretty obvious that some of the kids do have handicaps. Some of them are in special education. But they are given the opportunity to perform. They might not be in sync and they might not have all the steps down but they are up there and they are truly participating – they are part of something. They have achieved what it takes to get on that stage in a world class theatre. It’s impressive and they should be proud of themselves. I think, watching them, that they are.” Some of the students who excel in the CAPPA Showcase might not even be selected for performance in a middle school or high school play or concert. One community partner familiar with some of

CAPPA Comprehensive Program Evaluation Report June 2011


Major Themes

the children involved in the showcase said, “They [CAPPA leaders] take students that other people have not really involved or served that well. And they let them shine.” The mother of a special education student who participates in a CAPPA afterschool program said she appreciates how comfortable and welcome her son feels in the program. She said that her son never worries about being teased in the program, and that he feels the other students, including students in regular education, are his friends. She credits the teachers in the afterschool program for creating such a positive environment, “They understand him and they have helped the other kids to understand him, too.” Theme 9: Need for Stronger Parent Connection Community leaders, educators, CAPPA staff and volunteers all said they would like to see stronger involvement and support from the parents of students enrolled in CAPPA programs. A number of obstacles to parental involvement were added, and program leaders emphasized that they try to make programs as accessible as possible to students who do not have parents or other family members who are able or willing to help. For example, bus transportation is provided to Saturday programs. Children can be picked up at a variety of locations and accommodations can be made for children who spend the night with another family member

or friend on the weekend. “If they need transportation to get there on Saturdays, they know that we will be there. All they have to do is call,” one program leader said. Program leaders have even purchased and provided alarm clocks for children, so they can get themselves out of bed and to the bus stop on time on Saturday mornings. While a few parents do volunteer during the Saturday programming hours, many others do not. The parents who do volunteer indicated they were disappointed that more parents are not represented. Some said they thought the parent involvement would be higher if parents had a deeper understanding of the CAPPA program and all that it teaches. One mother who volunteers regularly said, “To a lot of the parents, it’s just something for their kid to do. It’s just Loni doing his shows. They don’t see the impact. But I see what CAPPA does. If you get involved you can see that.” Parents and community leaders mentioned a number of possible barriers to parent involvement. Some mentioned that parents are working multiple jobs, or do not feel that they have anything to offer themselves when they volunteer. Others are single parents who feel overwhelmed and want to use the hours their children are in CAPPA to run errands and do other things without having their children in tow. One mother mentioned that a number of parents with children in CAPPA programs are recovering addicts who spend a lot of time in recovery meetings and working

CAPPA Comprehensive Program Evaluation Report June 2011

29


Major Themes

to deal with their own challenges. “They [the parents] are into themselves a lot and it’s the children who are suffering,” she explained. Some community partners questioned whether CAPPA can be effective as its leaders would like, when parents are not more involved in the process. One community partner said, “You can do all you want with the children, but if you are not changing the child’s environment, the child goes back into that environment and it is very difficult to change. Some community partners suggested that CAPPA’s innovative spirit with students might be the ticket to solving the parent engagement issue as well. One leader said, “They have figured out how to engage the kids, but how do you engage the parents? They are good at being innovative. It’s time maybe to be innovative with the parents.”

“They try to make programs as accessible as possible to students who do not have parents or other family members who are able or willing to help.”

30

CAPPA Comprehensive Program Evaluation Report June 2011


Recommendations

The following recommendations emerged relative to the data collection and analysis process. These recommendations reflect input gleaned through key stakeholder interviews, as well as data collected and analyzed through observations, document and archival data review, and community walkabouts.

students are likely leaving CAPPA because they are engaged in negative activities.

One parent volunteer said that middle school students might need more encouragement and incentives to remain in the program, explaining, “We have lost a lot of our older girls this year. The thing is, when you come here you have to leave Explore the possibility of adding separate that world out there. It’s hard for some programming for middle and high school of them and they don’t make the right choices.” The parents said it is particularly students difficult to keep middle school girls whose One of the strengths of CAPPA seems to be parents do not strongly support the ideals the interactions that occur between people of CAPPA. In some cases, the parent of different ages, including interactions volunteer said, “the guys” are what drives between college students and youth, and middle school girls away, and they choose adult volunteers and youth. With that to spend time with boyfriends or other said, however, several parent volunteers males they know, instead of performing in and students who were interviewed the showcase. mentioned that many students seem to lose interest in the program when they One CAPPA educator said she thinks that enter the middle school years. Just before sometimes middle school students find the 2011 Showcase, for example, a core it difficult to spend their Saturdays in a group of middle school students opted out program that also serves children as young of the program, even though they had key as four. “There are a lot of little kids and big kids in close quarters,” the educator roles in the upcoming performance. said. “Maybe they need something else for Decreased participation among middle the high school kids.” school and high school students is likely part of broader issues, including that the Several people who were interviewed students feel increased peer pressure emphasized that older students can still to be involved in activities outside of benefit from CAPPA – if they do not opt out. CAPPA. Some of those activities may be “The high school kids still want someone positive ones, such as school basketball or to be there for them. They need affection softball teams, as was mentioned by some just like everybody else… But maybe they interview participants. In other cases,

CAPPA Comprehensive Program Evaluation Report June 2011

31


Recommendations

need to get that in a different way than the of ensuring that CAPPA continues to uphold younger students.” certain core values, and that new tutors, curriculum instructors and other staff The same CAPPA educator also suggested understand and share these core values, that the older students might enjoy as well as the research that supports them. having at least one or two numbers in the performance that are only for older The IFESJ recommends the development students. The students also might be and implementation of a CAPPA program more energized if they have a higher level guide that clearly explains the core of input into the performances, she said. principles of CAPPA, how it works, One middle school student in the program why it works, and how it should be also said that she thinks that maybe implemented. Information on hiring CAPPA needs different types of shows for of new staff, core program values, students who have been in the program CAPPA curriculum, employee several years. He said, “After you have guidelines, conflict resolution done CAPPA, it’s like there is not that much and financial practices new. I still have fun but some people say it are some of the topics starts to get boring when you do the same that should be included kind of thing year after year. They decide in the CAPPA program to go and do something else.” guide. The current CAPPA job descriptions also need Develop a Comprehensive Program Guide to be reviewed and analyzed Key community partners, community as part of this process. The leaders, educators, parents and students IFESJ believes this program all expressed strong support for CAPPA guide is an important step programs and also emphasized that there because it will formalize many is a need for additional programming in of the practices currently used informally by CAPPA the Williamsport area and beyond. leaders, making it possible to CAPPA leaders are currently in the process replicate the program and to of opening the new Unity Neighborhood allow new leaders who come Network Center, where additional young on board to more easily people are expected to be served. This new acclimate to the program. location represents an exciting opportunity for CAPPA, giving the organization its After the program guide largest facility yet and also allowing it a is developed, the IFESJ stronger presence in the community. But recommends additional with that opportunity comes the challenge training for CAPPA staff and volunteers.

32

CAPPA Comprehensive Program Evaluation Report June 2011


Recommendations

This training should focus on some of the information included in the program guide, with an emphasis placed on the core values of CAPPA, as well why and how the program works. This will allow the program to have greater consistency from site to site and to ensure that the integrity of CAPPA programming remains, while also fostering a greater sense of buy-in among staff and volunteers.

information on the organization’s finances were better understood in the broader community. And even though CAPPA has annual financial audits prepared by an independent CPA, it is recommended that CAPPA make their financial sustainability more visible to the community.

Constantly having to solicit donations for immediate needs weighs heavily on the executive director and other members of the staff, taking time and energy away from other needs, including program development and more direct work with the students and their families.

Explore the possibility of hiring more staff

Securing additional grant funding also would strengthen the image of CAPPA in the community, sending a clear message that the program is sustainable. One Aggressively seek additional funding. educational leader, for example, said he During the time that the IFESJ was knows that CAPPA struggles with funding, conducting research in Williamsport, it was “With CAPPA, funding is always an issue. clear that finances and fund-raising efforts They struggle with that. We [in the school were an ongoing challenge. The executive district] can support CAPPA with space director was making plans to spend several and with recruiting and promoting but days visiting local business leaders and we don’t have the funds to do more than other donors, asking for money to pay for that… I know we would be happy to see components of CAPPA Showcase, which them come up with a model that is more was about three weeks away. [financially] sustainable.”

Key community partners repeatedly mentioned how many hours CAPPA’s executive director and key staff work in any given week. This also was clear to the IFESJ as they spent time with CAPPA leaders while conducting research. It seemed CAPPA could benefit greatly from securing that employees made a habit of working one or more sizable public or private grants, through lunch, staying late and also which would allow the organization to working on the weekends. One community continue funding the CAPPA Showcase and partner noticed this hardworking culture, to expand its after-school programming saying, “I don’t think Loni [Gamble] is on a over time. Additionally, several community clock. He’s there at night and on Saturday members said they thought that grant and his staff is, too. He gets the job done. funding might be more readily available if His staff is there morning, noon and night.

CAPPA Comprehensive Program Evaluation Report June 2011

33


Recommendations

They really stick with it and put in the time. After years of working Saturdays, they might be tempted to say, ‘I don’t think I am going to do this anymore.’ But they are here, year after year.”

what was required of her during the early months of her employment.

Even though key staff members have been with the program over an extended time, it does seem that the organization is at risk of burning out some of its key leaders. For this reason, the IFESJ recommends that an external evaluation of the program’s organizational structure be conducted. The evaluation should identify who is responsible for which jobs, how responsibilities and other information are communicated, and where additional staff might be needed. This evaluation might then be helpful in any grant proposals and other requests for funding.

As discussed in the Key Themes section of this report, key community leaders, educators and parents volunteers all spoke to what they saw as a critical need to do more to involve the parents of CAPPA students. As one leader said, “They [CAPPA leaders] have figured out how to engage the kids. But how do you engage the parents? They are good at being innovative. It’s time maybe to be innovative with the parents.”

Strengthen Parent Programming

Outreach

and

It seems that one of the strongest assets of CAPPA is the parent volunteers who are engaged and present each week. One key element of the external These parents spoke passionately about evaluation of organizational structure the value and importance of getting should be revised job descriptions for other parents involved, even while also all employees. This might be helpful in recognizing the challenges of such efforts. clarifying the responsibilities of staff at At least one parent volunteer suggested both the administrative and site levels. that it might be effective to put together a One site level employee, for example, said group of parent volunteers who could visit that she found her job description was parents in their homes and talk to them confusing and did not clearly outline all the about CAPPA and the help that is needed. components of her job. She said a clearer This seems like it might be an effective job description at the time of her hiring approach, since parents might be more might have allowed her to more easily do receptive to other parents, who might understand the realities of their day-to-day lives better than individuals from outside the neighborhood, who might be viewed

34

CAPPA Comprehensive Program Evaluation Report June 2011


Recommendations

“One of the strongest assets of CAPPA is the parent volunteers who are engaged and present each week.�

make the event feel more approachable than having it at a CAPPA center. Several parent volunteers said that more than anything, parents seem to need help seeing that the time they spend with their children and the efforts they make really matter. For this reason, any events that celebrate the importance of parents, as well as the role they play as their children’s first and most influential teachers, might be highly valuable.

as outsiders who do not understand their challenges. Parent seminars that discuss practical parenting techniques and strategies, as well as ways to deal with challenges and to help build meaningful connections with their children also are likely to be valuable. One parent suggested that offering a free meal might be a good hook to get parents in the door. Having such gatherings in the home of a parent volunteer also might

CAPPA Comprehensive Program Evaluation Report June 2011

35


Conclusions

Overall, the researchers’ sense is that other forms of support at the local, state CAPPA is an effective program that meets and national levels. significant needs among young people in the greater Williamsport community. • While some parents currently volunteer in CAPPA and are engaged in meaningful The program is effective in part because ways, the support of more parents it combines innovative, high-interest would be highly beneficial, providing programming with a highly structured young people with the support they environment, where rules are enforced need not just in CAPPA but also at and young people can see the benefits of home. If parents can strengthen their working hard to reach their goals. CAPPA involvement in the lives of their children, staff and volunteers consistently interact CAPPA will have more effective, lifelong with the young people involved in caring success, potentially transforming the ways, supporting them in their efforts and lives of multiple generations of children giving them a sense that they can achieve and families. their goals with effort. The Institute’s analysis reveals the following insights about the program: • The adult staff and volunteers who work with the program are essential to its success. For this reason, it is critical that as the program grows, people who are new to CAPPA develop a clear understanding of the program’s goals and visions, as well as why the work that they do with young people matters.

• Program leaders have done a solid job of finding programming that meets the needs of young people in the Greater Williamsport community. Leaders have recognized when it was time to add additional programming and also have been strengthening the academic components of all programs over time.

• CAPPA seems to do a particularly good job of serving students who are not engaged in other areas, including after• CAPPA’s positive reputation within the school programming and recreational community is a clear asset that has been youth sports. Many of the students built over a nine-year span and should who are in CAPPA programs are not be capitalized on as the program strives involved in other community programs. to move forward in securing funding and Without CAPPA programs, these young people would likely be at home or gathering with friends. Some, according to student and parent interviews, would

36

CAPPA Comprehensive Program Evaluation Report June 2011


Conclusions

likely be involved in self-destructive activities. Overall, the evaluation data clearly illustrate that CAPPA is a strong, evidencebased program that is meeting the needs of both the participants it serves and the greater Williamsport community. With time and strategic planning, CAPPA can expand both the breadth and depth of its programming to meet the needs of even more youth while remaining true to its mission and key focus. “The program is effective in part because it combines innovative, high interest programming with a highly structured environment.�

CAPPA Comprehensive Program Evaluation Report June 2011

37


References

Bogdan, R. & Biklen, S. (1982). Qualitative research in education. Boston, MA: Allyn & Bacon, Incorporated. Borman, K.M., Clarke, C., Cotner, B., & Lee, R. (2006). Cross-case analysis. In J. L. Green, G. Camilli, & P.B. Elmore (Eds.), Handbook of Complementary Methods in Education Research (pp. 134-140). Danbury, CT: Lawrence Erlbaum Associates. Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches. Minneapolis: Sage Publications. Denzin, N.K., & Lincoln, Y.S. (Eds). (2005). Handbook of qualitative research (3rd ed.). Thousand Oaks, CA: Sage Publications. Eisenhart, M. (2006). Representing qualitative data. In J. L. Green, G. Camilli, & P.B. Elmore (Eds.), Handbook of Complementary Methods in Education Research (pp. 567581). Danbury, CT: Lawrence Erlbaum Associates. Gall, M.D., Gall, J.P., & Borg, W.R. (2007). Educational research: An introduction (8th ed.). San Francisco, CA: Pearson. Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Sage Publications. Yin, R. K. (1994). Case study research: Designs and methods (3rd ed.). Newbury Park, CA: SAGE Publications, Incorporated.

38

CAPPA Comprehensive Program Evaluation Report June 2011


Appendix A: Interview Protocol

Project: CAPPA Comprehensive Program Evaluation Report Time of Interview: Date: Place: Interviewers: Dr. Monique R. Henderson and Dr. Marina V. Gillmore Interviewee: Position of Interviewee: Description of Project: The purpose of this program evaluation is to present a comprehensive picture of the CAPPA program and its impact on participants, as told through the perspectives of key stakeholders. Guiding Questions: Question 1: What is your involvement with CAPPA? Question 2: What do you see as some of the benefits of CAPPA? Question 3: What do you think CAPPA might be able to do better? Question 4: What are some of CAPPA’s strengths? Question 5: How do you think other people in the community view CAPPA? Question 6: What other community opportunities exist for youth outside of CAPPA? What, if anything, sets CAPPA apart from these other programs? Question 7: How do you think children benefit as a result of their involvement in CAPPA? Question 8: Is there anything else that you’d like us to know about your experience with CAPPA? Thank you for participating in this interview. Your input will help us complete the comprehensive program evaluation for CAPPA. Please be assured that the confidentiality of all persons participating in this study is assured. If you have any questions, please do not hesitate to contact us.

CAPPA Comprehensive Program Evaluation Report June 2011

39


Appendix B: Interview Protocol

Project: CAPPA Comprehensive Program Evaluation Report Observer: Page Number: ___ of ___ Time

40

Observation

CAPPA Comprehensive Program Evaluation Report June 2011

Time of Observation: Date: Place: Reflection


Appendix C: Informed Consent Form w/ CAPPA Cover Letter May 16, 2011 ATTENTION PARENTS / GUARDIAN OF CAPPA PARTICIPANTS Dear Parent / Gaurdian: The week of May 16, 2011, CAPPA will be visited by Dr. Henderson and Dr. Gillmore of the Institute for Educational and Social Justice for purposes of program evaluation. The professors would like to interview some CAPPA participants about their experiences being a part of CAPPA. In order to do this ethically, parents and guardians of CAPPA participants must sign a release form which is attached. By signing you will be giving your child permission to participate in group discussion, individual interviews, and written survey evaluation. Interviews that are recorded both on tape and on paper will be used strictly by the Institute of Educational and Social Justice to create a final report for CAPPA. The final evaluation report created for CAPPA will be used to improve the programs in which your child participates. Thank you for your co-operation. Participation in this process is crucial to CAPPA's continued success in applying for state and federal funding that allows us to provide programming at no cost to participants; as well as continuing to provide the best programs possible for your children and the community in which you live. On the attached form please Print your child's name on the line above "Research Participant Name". Then please place your signature on the line above "Signature of Parent or Guardian". Please have your child return the signed form to their respective after school program tomorrow, Tuesday May 17, 2011. Thank you, Holly Doyle Executive Assistant

CAPPA Comprehensive Program Evaluation Report June 2011

41


Informed Consent Letter I voluntarily agree to participate in the evaluation of the CAPPA program. I understand that this evaluation is being conducted by Dr. Monique R. Henderson and Dr. Marina V. Gillmore of The Institute for Educational and Social Justice on behalf of CAPPA, Inc. in an effort to more fully understand the CAPPA program and its outcomes. I also understand that the program evaluation may be used to secure future funding for CAPPA Inc., as well as to advertise and benefit CAPPA, Inc. in ways deemed appropriate by CAPPA, Inc. and its Program Director, Loni Gamble. I understand that the evaluation methods in which I might participate include: 1. recorded observations of CAPPA activities and programs in which I participate; 2. my completion of evaluation questionnaire(s); 3. my participation in a 30-60 minute interview; 4. my participation in a focus group discussion. I grant permission for the interview and/or focus group to be tape recorded and transcribed, and to be used only by The Institute for Educational and Social Justice for analysis of interview data. I grant permission for the evaluation data generated from the above methods to be published in an evaluation report and in future publications, as deemed appropriate by CAPPA, Inc. I understand that any identifiable information in regard to my name will remain confidential in the above-mentioned evaluation report to funding organizations and in all subsequent publications associated with this program evaluation. I understand that there are no know personal risks or discomforts associated with this study and that I may withdraw from the evaluation study at any time without penalty. I also understand that there are no personal benefits or compensation associated with my participation in this study. Dr. Gillmore and Dr. Henderson are available to answer any questions you may have before, during, or after your participation in this study. They can be reached at instituteforedandsocialjustice@gmail.com and (760) 902-2377. Research Participant Name (Please print)

Research Participant Signature

Signature of Parent or Guardian (if participant is under 18 yrs)

42

CAPPA Comprehensive Program Evaluation Report June 2011

Date

Date


Appendix D: CAPPA Photo, Video, and Audio Release Form This letter confirms the agreement between you and CAPPA regarding your participation in approved CAPPA activities in which you may be photographed, videotaped, or audio recorded (the Property) from time to time. For valuable consideration received, you hereby irrevocably grant to CAPPA perpetually, exclusively, and for all media throughout the world (including print, non theatrical, home video, CD-ROM, internet and any other electronic medium presently in existence or invented in the future), the right to use and incorporate (alone or together with other materials), in whole or in part, photographs, video footage, or audio recordings taken of you/your child as a result of your/your child's participation in approved activities of CAPPA. You do hereby agree that you will not bring or consent to others bringing claim or action against CAPPA on the grounds that anything contained in the Property, or in the advertising and publicity used in connection herewith, is defamatory, reflects adversely on you, violates any other right whatsoever, including, without limitation, rights of privacy and publicity. You do hereby release CAPPA, its directors, officers, successors and assigns from and against any and all claims, demands, actions, causes of actions, suits, costs, expenses, liabilities, and damages whatsoever that you may hereafter have against CAPPA in connection with the Property. This agreement shall not obligate CAPPA to use the Property or to use any of the rights granted hereunder, or to prepare, produce, exhibit, distribute or exploit the Property. CAPPA shall have the right to assign its rights hereunder, without your consent, in whole or in part, to any person, firm or corporation. DATE SIGNED: Month

Date

Year

Signature of Parent or Guardian Print name of Participant: Telephone Number:

Cell Phone Number:

CAPPA Comprehensive Program Evaluation Report June 2011

43


Appendix E: Academic Support Program Release of School Records ACADEMIC SUPPORT PROGRAM Release of School Records RE:

Participant Name __________________________ DOB ____________________________ Address __________________________ SS# ________________________________

I hereby authorize The Community Alliance for Progressive Positive Action to: [X] Release to or [X] Secure from Williamsport Area School District _____________________________________________ (School District, Agency, or Individual) Name of School Currently Attending The Following Information is for School Year 2010 - 2011. [X] Progress Notes _______ [X] Report Cards _______ [X] Attendance Data ______ [X] Teacher Observations ______ [X] Other (Specify) _PSSA_________ [X] Other (Specify) _4 Sight Assessment_________ Note: My child’s records may be released as indicated by my initials next to the information released. This information is needed for the following purpose (s): [X] For participation in the Youth Intervention and Development Project Please send the requested information to the following address: Carolyn E. Perry, Administrative Assistant CAPPA Inc. 1919 Lincoln Drive Williamsport, PA 17701 All information presented will be held in strict confidence. This release will be considered valid for the period of: 1 Year From 9/2010 thru 8/2011 This authorization may be revoked by written request at any time. Revocation will not affect action taken by the above prior to the date of revocation. The above named Agencies and individuals may release only information they have generated and not information created by other agencies or institutions. Information from other agencies or institutions will require a separate release. ____________________________________ ________________________ Signature of Student/Parent/Guardian/Other Date If other, please specify relationship: _________________________________________________

44

CAPPA Comprehensive Program Evaluation Report June 2011


CAPPA: Creating Solutions CAPPA Comprehensive Program Evaluation Report June 2011

45

CAPPA Program Evaluation 2011  

Community Alliance for Progressive Positive Action: Comprehensive Program Evaluation ReportJune 2011

Read more
Read more
Similar to
Popular now
Just for you