Beginning Teacher Induction and Mentoring (BTIM) Program: Evaluation Report
After receiving the data files from TEA, the evaluation team screened the files for consistency. Following the screening, data was merged and cleaned (e.g., looking for missing data, outliers, or invalid responses). Descriptive frequency data was shared with TEA to promote validity and resolve obvious errors.
2.3.5 Data Collection Activities Data was collected from a variety of sources, including BTIM grant applications, focus groups and interviews with key stakeholders, and surveys. Existing data sources also were collected and used to supplement these sources. The data collection activities utilized in the evaluation process are discussed below.
2.3.5.1
Review of Grantee Applications
Grantee applications were reviewed and data were extracted to comprehensively address some of the research questions. In addition, information from the grantee applications was used to provide context for the six individual case studies. Data on budgeted costs by various categories (e.g., payroll, contracted services, supplies and materials) were extracted and put into a database for all grantees. This included data on the budgeted amount for mentor teacher stipends. In addition, these databases were designed to pull out data on the number of mentor teachers and beginning teachers targeted by grade-level to participate in the BTIM program. Grantee progress reports were used as a source for determining the actual funds spent to date (based on reported draw-downs from grantee accounts with TEA). These reports were also compared to aggregated data on grantee performance to create a complete picture of the district.
2.3.5.2
Case Studies
Case studies provide the means by which the evaluation can explore the complex interactions between beginning teachers and mentors to better understand how induction might influence student achievement and teacher retention. ICF evaluators used a multiple-case design to collect data that enhances quantitative analyses of survey and extant data, which is based on the full sample. While this methodology is limited because intensive study of one or a few sites may not generalize to broader contexts, it is commonly accepted as an exploratory tool or as a way to provide context to quantitative data. Through the case studies, the evaluation team was able to capture descriptive data on BTIM processes of induction, mentor support and training, impacts on student learning, and other key study objectives. Qualitative data for the case studies was collected through district and campus administrator interviews, beginning teacher focus groups, mentor focus groups, and interviews with mentor and beginning teacher dyads. The extant data and document review (e.g., grantee application review) also informed the case studies. Selection Process for Case Study Sites The evaluation team selected six participating districts that represented a range of scenarios of interest to TEA. The team first looked at the number of districts participating in the BTIM grant program within each ESC region, and the number of campuses within each participating district. The districts were selected from ESC Regions 1, 4, 11, 13, 19, and 20 because they had the most campuses participating in BTIM Cycle 1. Due to the interest in variations in program implementation by community type, the next step was to group the participating districts by community type and organize them into three groups:
33