Shairose jessani mphil final

RELATIONSHIP BETWEEN COLLEGE ENTRANCE TEST SCORES AND GRADE ELEVENTH SCIENCE STUDENTS’ PERFORMANCE IN ANNUAL BOARD EXAMINATIONS

BY SHAIROSE IRFAN JESSANI

Supervised By Prof. Dr. Muhammad Yousuf Sharjeel

DEPARTMENT OF EDUCATION COLLEGE OF ECONOMICS AND SOCIAL DEVELOPMENT INSTITUTE OF BUSINESS MANAGEMENT KARACHI-PAKISTAN 2016

RELATIONSHIP BETWEEN COLLEGE ENTRANCE TEST SCORES AND GRADE ELEVENTH SCIENCE STUDENTS’ PERFORMANCE IN ANNUAL BOARD EXAMINATIONS

BY SHAIROSE IRFAN JESSANI DEPARTMENT OF EDUCATION INSTITUTE OF BUSINESS MANAGEMENT KARACHI-PAKISTAN 2016

SUPERVISOR PROF. DR. MUHAMMAD YOUSUF SHARJEEL

A THESIS SUBMITTED IN PARTIAL FULFILLMNT OF THE REQUIREMNTS FOR THE DEGREE OF MASTERS OF PHILOSOPHY IN EDUCATION DEPARTMENT OF EDUCATION COLLEGE OF ECONOMICS AND SOCIAL DEVELOPMENT INSTITUTE OF BUSINESS MANAGEMENT KARACHI – PAKISTAN 2016

THIS THESIS IS SUBMITTED BY SHAIROSE IRFAN JESSANI

IN PARTIAL FULFILLMENT OF THE DEGREE FOR MASTERS OF PHILOSOPHY IN EDUCATION

2016

SUPERVISED BY PROF. DR. MUHAMMAD YOUSUF SHARJEEL

DATED: AUGUST 25, 2016

CERTIFICATE OF APPROVAL

It is certified that Shairose Irfan Jessani has successfully completed her research study entitled “RELATIONSHIP BETWEEN COLLEGE ENTRANCE TEST SCORES AND GRADE ELEVENTH SCIENCE STUDENTS’ PERFORMANCE IN ANNUAL BOARD EXAMINATIONS” under my supervision, in partial fulfillment of requirement for degree of Masters of Philosophy (M. Phil) in Education. The dissertation has been reviewed and meets the standards set by BASAR, IoBM, College of Economics and Social Development (CESD), Institute of Business Management.

Dr. Muhammad Yousuf Sharjeel Supervisor

TITLE: RELATIONSHIP BETWEEN COLLEGE ENTRANCE TEST SCORES AND GRADE ELEVENTH SCIENCE STUDENTS’ PERFORMANCE IN ANNUAL BOARD EXAMINATIONS

BY SHAIROSE IRFAN JESSANI DEPARTMENT OF EDUCATION INSTITUTE OF BUSINESS MANAGEMENT KARACHI-PAKISTAN 2016

_____________________ EXAMINER – 1

_____________________ EXAMINER – 2

To My Three Little Stars: Misha, Manal & Amish I have given them BIRTH, but they have given me LIFE

TABLE OF CONTENTS ACKNOWLEDGEMENT ................................................................................................................. i ABBREVIATIONS ........................................................................................................................ iii ABSTRACT................................................................................................................................... v CHAPTER I .................................................................................................................................. 1 INTRODUCTION .......................................................................................................................... 1 1.1 Introduction .................................................................................................................... 1 1.2 Background of the Study ............................................................................................... 2 1.3 Problem Statement......................................................................................................... 4 1.4 Research Question(s) ..................................................................................................... 6 1.5 Purpose and Method of the Study ................................................................................ 7 1.6 Significance of the Study ............................................................................................... 9 1.7 Scope of the Study ........................................................................................................ 10 1.8 Delimitations of the Study ........................................................................................... 10 1.9 Basic Assumptions ....................................................................................................... 11 1.10 Operational Definitions ............................................................................................. 11 1.11 Structure of the Thesis............................................................................................... 12 CHAPTER II ............................................................................................................................... 15 LITERATURE REVIEW ................................................................................................................ 15 2.1 Introduction .................................................................................................................. 15 2.2 High-Stakes Testing (HST) ......................................................................................... 17 2.2.1 Historical background .......................................................................................... 17 2.2.2 Philosophical Issues .............................................................................................. 20 2.2.3 Forms of high-stakes testing in Pakistan ............................................................ 25 2.3 College Entrance Test .................................................................................................. 26 2.3.1 A brief history ....................................................................................................... 26 2.3.2 Functions of admission testing ............................................................................. 29 2.3.3 Systems and criteria of college admissions ......................................................... 31 2.3.4 Limitations and challenges ................................................................................... 32 2.4 Factors Influencing Students’ Performance .............................................................. 37 2.4.1 Academic factors ................................................................................................... 37 2.4.2 Non-academic factors ........................................................................................... 41 2.5 Conclusion .................................................................................................................... 43 CHAPTER III .............................................................................................................................. 45 RESEARCH METHODOLOGY ..................................................................................................... 45

3.1 Introduction .................................................................................................................. 45 3.2 Research Design ........................................................................................................... 45 3.3 Context of the Study .................................................................................................... 47 3.4 Population ..................................................................................................................... 49 3.5 Sample and Sampling Procedure................................................................................ 49 3.6 Data Collection Methods and Tools ........................................................................... 50 3.7 Data Analysis ................................................................................................................ 52 3.8 Ethical Consideration .................................................................................................. 53 3.9 Conclusion .................................................................................................................... 53 CHAPTER IV .............................................................................................................................. 54 DATA ANALYSIS AND RESULTS ................................................................................................. 54 4.1 Introduction .................................................................................................................. 54 4.2 Quantitative Data Analysis ......................................................................................... 54 Hypothesis 1.................................................................................................................... 54 Hypothesis 2.................................................................................................................... 63 Hypothesis 3.................................................................................................................... 65 Hypothesis 4.................................................................................................................... 67 4.3 Qualitative Data Analysis ............................................................................................ 70 4.3.1 Students’ perceptions and the role of CET ......................................................... 70 4.3.2 Students’ perceptions and the CET model ......................................................... 73 4.3.3 Students’ perceptions and factors influencing their performance ................... 78 4.4 Conclusion .................................................................................................................. 100 CHAPTER V ............................................................................................................................. 102 DISCUSSION............................................................................................................................ 102 5.1 Introduction ................................................................................................................ 102 5.2 Findings....................................................................................................................... 103 5.3 Implications and Recommendations ........................................................................ 118 5.4 Contribution of the Present Findings ....................................................................... 120 5.5 Future Research Directions ...................................................................................... 121 5.6 Conclusion .................................................................................................................. 121 REFERENCES ........................................................................................................................... 123 APPENDICES ........................................................................................................................... 131 Appendix A ....................................................................................................................... 132 Ethical Consent Letter to Students for Participation in the Research Study ............. 132 Appendix B ....................................................................................................................... 133 Ethical Consent Letter to Teachers for Participation in the Research Study ............ 133

Appendix C ....................................................................................................................... 134 Semi-Structured Tool for Focus Group Discussion (FGD) .......................................... 134 Appendix D ....................................................................................................................... 135 Interview Guide for Teachers ......................................................................................... 135 Appendix E ....................................................................................................................... 136 Guide for Individual Student’s Interview ...................................................................... 136 Appendix F ....................................................................................................................... 137 Comparative Performance of Students from two Examination Systems ................... 137 Appendix G ....................................................................................................................... 138 Correlation of Students’ Scores (%) in Different Performance Domains .................. 138 Appendix H ....................................................................................................................... 139 Correlation of Students’ Performance (%) in SSC II and HSSC I Annual Exams ... 139 Appendix I ........................................................................................................................ 140 Correlation of Students’ Performance (%) in SSC II and HSSC I Annual Exams with Respect to Shift in Examination Boards ........................................................................ 140 Appendix J ........................................................................................................................ 141 Students’ Performance (%) in HSSC I Annual Examinations .................................... 141

LIST OF FIGURE

Figure 2.1. Conceptual Framework on Correlation between College Entrance Test and Science Students’ Performance.................................................................................... 16

LIST OF TABLES

Table 4.1: Students’ Performance in HSSC I & CBT - Biology …………………….55 Table 4.2: Correlation of Students’ Performance in HSSC I & CBT- Biology…….. 56 Table 4.3: Students’ Performance in HSSC I & CBT – Chemistry……………….....57 Table 4.4: Correlation of Students’ Performance in HSSC I & CBT- Chemistry...…58 Table 4.5: Students’ Performance in HSSC I & CBT –Physics ………………….… 59 Table 4.6: Correlation of Students’ Performance in HSSC I & CBT- Physics …...…60 Table 4.7: Students’ Performance in HSSC I & CBT – Science ………………....... 61 Table 4.8: Correlation of Students’ Performance in HSSC I & CBT- Science …….. 62 Table 4.9: BIEK Students’ Performance in HSSC I-Science & CET ………..……… 63 Table 4.10: Correlation in HSSC I-Science & CET for BIEK Students …………….64 Table 4.11: AKUEB Students’ Performance in HSSC I-Science & CET………....…65 Table 4.12: Correlation in HSSC I-Science & CET for AKUEB Students …………66 Table 4.13: Multiple Linear Regression Model Summary and Overall Fit Statistics .67 Table 4.14: Analysis of Variance and Model Significance ………………………….68 Table 4.15: Model Coefficients Table ……………………………………………….69

i

ACKNOWLEDGEMENT

After thanking Al-Mighty Allah, who has given me the skills and strength to accomplish this task, I wish to acknowledge the contributions of many other people who have made this research, both possible and successful.

First of all, I would like to acknowledge my supervisor Dr. Muhammad Yousuf Sharjeel, for his scholarly advice, help and critical feedback which has contributed significantly to the completion of this study. It has been an enriching learning experience. Dr. Sharjeel’s questions, appreciation and above all his belief in my capabilities helped me in becoming an independent learner.

I would extend my thanks to the internal and external examiners who gave their valuable time and input to improve the quality of this thesis. Also, my gratitude is due to Rector Mr. Talib Karim, HoD and all faculty members for providing learning platforms during these two years of my M Phil studies. I am also grateful to Dr. Munira Amir Ali for her valuable guidance which she offered at the time of any perplexity or confusion.

I am extremely thankful to the school where I conducted my study. My gratitude is to the school management, teaching, non-teaching staff and students for their sincere help, wishes and suggestions throughout the study. Specifically, I want to extend my sincerest thanks to Mr. Fareed Rahim who assisted me with the students’ records. I am indebted to the students of my focus group for their valuable time, views and reflections which helped me in successfully completing this study. I am also thankful to my colleagues who include Ms. Yusra Dana, Ms. Raiha Arshad and Ms. Prisca Sabir, who willingly assisted in proofreading and editing my paper. My special

ii

appreciation is to Ms. Anna Rizwan and Ms. Kaneez Zainab, for their unlimited support during these two years of study at IoBM. A heart-felt commiseration is to my mother-in-law. I would not have been able to accomplish this task of writing my thesis if she had not immensely supported me. I feel proud of my daughter Misha who took care of her younger siblings, Manal and Amish in my absence. I also extend my gratitude to my five-year old daughter and three-year old son for displaying patience for all the time I was not able to be around them. An indispensable part of my success is my beloved husband, Irfan. His encouragement and support helped me to strive for excellence. I cannot thank him enough for the role he played during these two years.

iii

ABBREVIATIONS

ACT

American College Testing

AKUEB

Aga Khan University Examination Board

AT

Achievement Test

BIEK

Board of Intermediate Education Karachi

BSEK

Board of Secondary Education Karachi

CAP

Centralized Admission Policy

CBT

Computer Based Test

CEEB

College Entrance Examination Board

CET

College Entrance Test

CIE

Cambridge International Examination

DLA

Disbursement Learning Indicator

ESEA

Elementary and Secondary Education Act

FGD

Focus Group Discussion

GoP

Government of Pakistan

HA

High Achievers / High Ability

HSSC

Higher Secondary School Certificate

HST

High Stakes Testing

LA

Low Achievers / Low Ability

MA

Mid-level Achievers / Mid-level Ability

MCQ

Multiple Choice Question

NAEP

National Assessment of Educational Reform

NCLBA

No Child Left Behind Act

NDEA

National Defense Education Act

iv NEP

National Education Policy

RSU

Reform Support Unit

SAT

Scholastic Achievement Test

SES

Socio-economic Status

SSC

Secondary School Certificate

USA

United States of America

USAID

United States Agency for International Development

v

ABSTRACT

An increasing trend of testing candidates before giving admissions in an educational institute is commonly observed in Pakistan. This trend can be particularly seen in the private educational institutions in the name of quality induction. The purpose of this study is threefold. First, to find out the appropriateness of College Entrance Test (CET) model used in the private higher secondary school in Karachi, Pakistan for the predictability of students’ performance in Higher Secondary School Certificate (HSSC) I annual board examinations. Second, to examine the correlation between students’ performance in college entrance test and in annual examinations with special focus on science subjects. Third, to investigate the factors which influence the performance of science students in annual board examinations. Following hypotheses were formulated to test the correlation. Four hypotheses guided the study significantly in this direction. The mixed method study was conducted in two phases. In phase I, data of n= 225 students who got registered in the sampled college in 2014 were gathered. Students’ scores in the college entrance test 2014 and annual board’s result 2015 were correlated through regression analysis at α = 0.05 level of significance by using Statistical Package of Social Sciences (SPSS) version 22. In phase II, self-prepared semistructured interview tools (see Appendices C, D & E) were used during focus group discussions (FGDs) and individual interviews of students identified in the data based to find out students’ experiences leading to their learning outcomes in science. The finding of first hypothesis revealed highly significant correlation between students’ performance in CBT and HSSC I in

vi

Biology (r = 0.35, p = 0.00), Chemistry (r = 0.37, p =0.00), Physics (r = 0.50, p =0.00), and overall science subjects (r = 0.57, p = 0.00). Similarly, significant correlation was found between CET scores and cumulative science performance of students affiliated with BIEK (r = 0.70, p = 0.00) and AKUEB (r = 0.51, p = 0.00). Findings also suggested CET as a good model for the predictability of students’ performance in annual board examinations for Grade 11 pre-medical group (R Square= 0.43, F= 50.0, p = 0.00). Moreover, the discussions with students revealed the central role of parents’ approach towards education, teachers’ attitude and practices, examination systems and standards, school culture and policies and psycho-socio factors in determining students’ performance in science subjects. In the light of these findings, several recommendations are made for various stakeholders on the policies and practices related to school admissions and science education at higher secondary level.

Key words: College entrance test, computer based test, science subjects, students’ performance, annual board exams

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

1

CHAPTER I INTRODUCTION 1.1 Introduction Testing is an old phenomenon, but recently it has been given much importance throughout the world particularly in the field of education. It provides information that can be used for diagnosis; it also provides motivation to the learners for reaching their goals and ensures efficiencies in teaching and learning when standards are aligned (Phelps, 2006). In addition, testing provides an evidence of one’s capacity as well as state of learning. This summative data is useful for the predictability of students’ performance in the future. Keeping these benefits of testing in mind, many educational institutions in Pakistan have adopted their admission and entrance test policy for effective intake of students. These entrance tests aim to provide reliable information for valid decision about the selection of students for various programs particularly in the context of Grade 11 board examinations conducted by the respective authorities in Karachi.

Many developed countries have started conducting standardized tests for admission in higher education institutions. Ali and Ali (2013) claimed that “the scores on such tests demonstrate applicants’ intellectual ability and knowledge” (p.77), hence predict the applicants’ success or failure in a particular field of study in the future. This increasing trend of testing for the purpose of admission is equally evident in developing countries like Pakistan, particularly in the private educational institutions at secondary and tertiary levels for quality induction. This call for competency test for merit-based admission is the response to reliability, validity, quality, and alignment issues of public examinations (Pakistan, Ministry of Education, 2009). On the other hand, these tests are of prime importance for parents and students

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

2

as these are considered as the gate keeper for students getting admission in reputable educational institutions, and ultimately to their bright future and good jobs.

1.2 Background of the Study Entrance test in educational institutions is seen as a tool that filters students on the basis of merit, increases the nation’s intellectual productivity and restores global competitiveness. The concept of entrance test originated in early 20th century with the desirability of classifying students at the time of entry in technical courses to avoid wastage. This wastage remedy was proven effective to reduce the dropout rates in the courses, hence found beneficial for sustaining intellectual assets of higher education institutions (Schaefler, 1962).

In Pakistan, most of the annual public examination systems at the school level test memorization and recall, and thereby promote rote learning which is detrimental for understanding and for application of knowledge at all levels, as well as poor preparation for higher education (Government of Pakistan, 2002; Pakistan Ministry of Education, 2009). Consequently, admission to higher education institutions, on the basis of examination scores, would not be a valid tool to measure students’ knowledge and ability for future studies. It is therefore proposed in the National Education Policy (NEP) -2009 that entrance tests be conducted by colleges and universities to select their students from a pool of applicants affiliated with different examination systems.

Considering the need of institutions’ control on the intake, many private schools have started administering self-prepared admission tests to ensure students’ academic ability, tendency to cope with the stress level and hidden talent. Further, to cater to a vast number of applicants for the limited seats, these entrance tests are made as a filter for quality intake into prestigious institutions. There are some underlying assumptions about these entrance tests. First, students who excel in achievement tests (i.e. public

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

3

examinations), but have attended schools with fewer resources or less rigorous learning process would score low in the aptitude test. The second is that, regardless of the type of resources and education system, some very talented students who could not perform in the achievement test for any reason will outshine in the aptitude test.

This is supported by the findings that many high achievers in the achievement exams got failed in the aptitude test of a well-reputed science and engineering university (Askari, 2015). This situation triggers the critique on several aspects of science education in Pakistan, for example the standard of examination system, quality of testing in science, alignment of secondary and tertiary science curriculum, and the ethical dimensions of entrance tests of higher education institutions.

Science education enjoys the centrality of educational disciplines as this is considered to be the key to the development and progress of any nation. Further, the futuristic and instrumental value of science-oriented learning (Osborne & Collins, 2001) is reflected in the entrance tests of higher education institutions of medical and engineering education in Pakistan. This is revealed in the higher weightage that they give to science as compared to other subjects in their admission tests. In addition, the scores of science are taken into special consideration when making decisions about admission since it is regarded as the key indicator of students’ competency. Hence, it is clear that the students’ learning outcomes in science is determinant of their success in the admission tests of all reputable institutions. Many studies show several personal and school related factors influencing students’ learning and performance in science.

Though it is difficult to carry out predictive analysis of students’ future performance in science, admission tests are considered as potential indicators to determine the chance of success in the field of science studies. Therefore, assessing the predictive

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

4

validity becomes more significant to see if the tests are valid in making admission decisions and to ensure that the information gathered from such tests is accurate.

The other aspects associated with entrance tests are psychosocial and economic dimensions which have severe implications on both the society and its individuals. These entrance tests place a considerable pressure on the students and parents leading to psychological problems and even suicide among school children (Davey & Higgins, 2005; Dai, Chen, & Davey, 2007). There is also stress upon schools and teachers whose reputations are at stake based on the number of their students’ getting admissions in institutions of prestige (Lewin & Xu, 1989; Dai et al., 2007, Wang & Ross, 2010). In addition, each test charges a fee and the cumulative total of these fees and the cost of test preparation programs may be daunting to many families. It is reported that students and parents denounced these tests and considered it dual educational systems and source of stress for them (Reporter Dunya News TV, August 30, 2015).

1.3 Problem Statement Academic retention and success of students, especially during first year, are among the major concern for colleges and universities. This is because first year is the time when students lay the foundation on which their subsequent academic success and persistence rest (Schaefler, 1962). However, since there are many variables that influence students’ academic success, deciding which criteria are the most accurate in predicting student academic success in higher education is a difficult task.

Reseon, Terenzini, and Domingo (2006) indicated that, before selecting academically competent students, it is essential to identify variables that are used as indicators of success to reduce the failure rate of students in training. This is used to evaluate the effectiveness of the training program and decrease wastage of the

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

5

educational institution’s time and resources on students who have little chance to succeed. Many studies have confirmed that cognitive measures such as high school GPA and college entrance exam test scores, and psychological variables such as selfefficacy and achievement motivation predict first-year university students’ grades better than cognitive measures alone (Pascarella, Sedlacek, & Terenzni, 1991; Ting, 2001).

In Pakistan, research on predictive validity is a new field for researchers and therefore, studies conducted so far, on the predictive validity of Entry Tests are scarce. Though no such study is carried out at higher secondary level, but few of such kind have been done at professional medical colleges. For example, one of the studies

by Baig, Ghouri, Fehmida, and Nargis (2001) revealed that there was a negative correlation between the entry test scores and the academic achievements of students.

Also, the study conducted by Huda, Dosa, Alam, and Agha (2001) concluded that none of the components of admission criteria that is Secondary School Certificate (SSC), Higher Secondary School Certificate (HSSC), Medical University admission test and interview scores, predicts the academic achievements of medical students in professional examinations. The findings from these studies do not confirm the findings in this area at international level. Hence these pose a question mark on the applicability of entry test in Pakistani context. Further, these studies do not explore the factors which impact the correlation of entrance test and annual performance AGAZ Higher Secondary School (pseudonym) has faced a substantial increase in student demand with the passage of time. It seems particularly important that it reassures that its admission criteria are as educationally sound as possible. Starting from its establishment, the admission criteria of AGAZ Higher Secondary School was mainly based on cognitive factors. These cognitive predictors include the previous

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

6

academic achievement and college entrance test which carry 80% weightage in deciding students’ enrollment. Few aspects of non-cognitive predictors are taken care of through interview processes which carry less weightage in the overall admission scores. So, students start their studies on the bases of cognitive predictors alone, without much consideration of non-cognitive (psychological variables) predictors, which are very important to identify the right students for the curriculum. The need arises to check if these tests would have the potential to identify students who have capacity, but have not demonstrated it so far.

1.4 Research Question(s) To guide the course of investigation in this study, answers will be sought to the following main question: Is there any significant relationship between College Entrance Test (CET) scores and Grade 11 students’ performance in science subjects in the annual Board Examinations?

Subsidiary Research Questions and Hypotheses In the light of the above main question, the following subsidiary questions will be explored in the study.

1. To what extent are the Computer-based Test (CBT) scores significantly related with students’ performance in annual Board Examinations in biology, chemistry, and physics?

2. To what extent are CET scores significantly related with the students’ performance in annual Board Examination with reference to science subjects in the Board of Intermediate Karachi (BIEK) examination boards?

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

3. To what extent are CET scores significantly related with the students’ performance in annual Board Examination with reference to science subjects in the Aga Khan University (AKU) examination boards?

4. Which components of CET model predict Grade 11 science students’ performance in annual board examinations?

5. How do Grade 11 science students perceive computer-based test, interviews, and Secondary School Certificate (SSC) - I results as CET model?

6. What factors influence the relationship between CET scores and students’ performance in the annual board examinations with reference to science subjects?

The hypotheses that emerge are: H1:

There is a significant relationship between CBT scores and students’ performance in annual Board Examinations with reference to physics, chemistry, and biology.

H2:

There is a significant relationship between CET scores and students’ performance in BIEK examinations with reference to science subjects.

H3:

There is a significant relationship between CET score and students’ performance in AKUEB examinations with reference to science subjects.

H4:

CBT scores, SSC I scores, and Interview scores predict Grade 11 science students’ overall performance in annual board examinations.

1.5 Purpose and Method of the Study AGAZ (pseudonym), a private higher secondary school in Pakistan selects its promising students based on the secondary school certificate (SSC I) results, interview score, and computer based tests (CBT). The prediction based on these indicators ought to be examined to ensure precise and reliable admission decision.

7

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

8

Making inaccurate prediction about students’ ability for future performance would add a burden to the school.

Further, these tests are used to make important decisions about students’ future career so it is necessary to investigate its appropriateness to ensure effective admission practices. Also, much of the existing literature focuses on the correlation of CET and students’ future performance. This study has analyzed multilevel factors that influenced the relationship between the study variables. The insight on the interplay of these factors with students’ learning in science would help in designing comprehensive plans and policies to promote science education at higher secondary level. The study aims to 

understand the students’ perceptions about the purpose, model and impact of admissions tests;



examine carefully the statistical justifications for the use of admissions tests, interviews, and SSC I results, including their usefulness in predicting annual performance of science students;



evaluate the degree to which existing admission criteria meet the needs of the faculty and students; and



draw conclusions and make recommendations regarding the future use of admission tests for the purposes of both eligibility and selection and lay the groundwork for a broader faculty dialog on these issues.

An ex-post facto study has been conducted using archival data that is available to the researcher in her role as Vice-Principal of a private college in Karachi. Data that were collected included students’ scores on admission test 2014 and their performance in annual board examinations 2015 and the correlation was determined.

Further, students’ experiences were collected through focus group discussions (FGDs)

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

9

and analyzed to see how these intermediate students perceive the role and effectiveness of college entrance test and different academic and non-academic factors in determining their annual board performance in science.

1.6 Significance of the Study There is hardly any published research study available in the context of Pakistan on the predictability of college entrance test scores for students’ performance in annual board examination at higher secondary level. This mixed research study therefore, may provide a basic understanding about the same in the Pakistani context specifically in the area of science education. Further, the work association of the researcher with AGAZ Higher Secondary School would provide an opportunity to relate the study findings to its students’ academic support programs, and for reviewing its admission policy. Also, this study can be used as a baseline to investigate more academic and non-academic variables in determining students’ learning achievement in science. Thus, the study will give important information to educators, policy makers, counselors, students and other concerned bodies in following respects: •

Researchers can use these findings for further research in this area and for suggesting a better prediction model in the form of admission policy.

•

Educators may improve the academic performance of their students through designing educational practices that could positively affect students’ learning in science classroom.

•

Students can evaluate their social and psychological variables and analyze the effect on their academic performance.

•

School management, counselors and parents can work with students for better academic and personal planning.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

•

10

Other concerned bodies such as policy makers, curriculum designers, and public examination departments can use students’ experiences to bring reforms in their respective domain.

1.7 Scope of the Study In the present study, all three components of students’ college entrance test (interview, CBT, and SSC I) have been considered either wholly or partially based on science test scores. In a future study, students’ scores in each component need to be divided into several areas (e.g., subject areas, interview indicators) to compare the predictive validities for each area in detail. Also, the empirical part of this study explores different cognitive and non-cognitive factors influencing students’ performance in annual board examinations. However, validity of these predictors, such as students’ socio-economic status, parental qualification, examination systems, and teachers’ experience need to be examined further in detail in the follow up studies.

1.8 Delimitations of the Study This study mainly focuses to explore the correlation of college entrance test of a private higher secondary school with students’ achievement in science subject in eleventh grade board examinations. As this study involved only pre-medical students of a private college, there is a limitation of generality in this study and hence findings cannot be generalized to the students of other disciplines and context.

Another limitation is on the scope of the study. Though the empirical part of this study explored the test takers’ perspective and experiences which helped in identifying the key issues and benefits of admission process, test taker’s perspective was missing and could not lead to a holistic picture of the phenomena. Further, it

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

11

delimits to provide an in-depth explanation of broader educational issues like CET linkages with national educational policies and curriculum.

Another limitation of the study is the unavailability of data for entire applicants (i.e. data for both the selected and unselected applicants). This information could be used for correcting observed validity coefficients for restriction of range.

1.9 Basic Assumptions The data are inclusive of all students affiliated from different examination systems at secondary level. Thus, overall data analysis overlooked the process and policy differences of these different systems. Scores recorded for students’ performance in entrance test and board exams are considered as an accurate representation of ability. Further, it is assumed that students involved in the focus group discussions have reflected Grade 11 students’ lived and past experiences in the context of research area.

1.10 Operational Definitions Annual board examinations. Public examinations conducted by the Board of Intermediate Education, Karachi (BIEK) and Aga Khan University Examination Board (AKUEB). BIEK was established by the Government of Sindh and organizes the higher secondary education examinations in Karachi whereas AKUEB is a Federal Board of Intermediate and Secondary Education that offers qualifications at secondary and higher secondary levels across Pakistan.

Higher secondary school. College which takes students after ten years of formal schooling for two-year period and prepares them for Higher Secondary School Certificate (HSSC), which is the pre-requisite for getting admission in professional colleges and universities in Pakistan.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

12

College entrance test. This refers to the entry test in a private higher secondary school in Karachi for the students’ admission in HSSC program. It comprises: a) computer-based tests (CBT), b) interview, c) Secondary School Certificate (SSC) I results. They all have different weightage in overall admission score.

Computer based test: MCQ-based admission test which students take by using a software and it comprises 15 questions each from chemistry, mathematics, and physics, 25 questions from biology, and 30 questions from English for pre-medical group.

Students’ performance. In this study, students’ performance or achievement is taken as the learning outcomes in the form of obtained marks in annual board examinations.

1.11 Structure of the Thesis This thesis explores the relationship between college entrance test scores and performance in science subjects for Pre-Medical students at a private higher secondary school. Further, students’ perception related to the role, effect, and model of CET and their experiences during the first year of studies are investigated to see how the above correlation may influence due to different academic and non-academic factors in students’ life. This thesis consists of five chapters. Chapter one is the introduction in which the background of the study, statement of problem, research questions and hypothesis, purpose, significance, scope, delimitation of the study, basic assumptions, operational definitions of key terms, and an overview of dissertation have been discussed.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

13

Chapter two is about literature review that helped in developing conceptual framework for this study. In this chapter, historical background, philosophical issues and forms of high stakes testing, brief history, functions and criteria, limitations, and challenges of college admission tests are discussed. Also, the academic and nonacademic factors are highlighted that influence students’ science performance in annual board examinations. This provided a lens to the researcher to see these phenomena in a broader perspective during the study.

Chapter three focuses on the methodology for this study along with research design. The rationale for selecting mixed research paradigm is discussed along with the detailed description of context of study, sample, and sampling process. The last part of this chapter discusses data collection methods and tools, data analysis procedures, and the ethical consideration of this study.

Chapter four presents the findings obtained from the quantitative and qualitative methods of the study. In the first part of this chapter, descriptive and analytical data are presented in tabular form related to all hypotheses. Further, the regression analysis is presented to explore the best fit model of college entrance test.

The second part is based on the qualitative data where stories of two outlier students, each from low and high ability groups are narrated. Also, based on these stories and focus group discussions of the students, six major themes are discussed related to the factors influencing eleventh grade science students’ performance in annual examination.

Chapter five is based on the discussion of study findings in connection to the relevant literature and purpose of the present study. Each hypothesis is discussed in the light of quantitative and qualitative data with several possible explanations.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

Further, implications and recommendations are shared about the role of policy makers, school management, teachers, and parents in improving the testing phenomena for improved performance of science students. Also, last section of chapter five presents the contribution of this study and the emerging areas for future research in the context of college entrance tests.

14

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

15

CHAPTER II LITERATURE REVIEW 2.1 Introduction Entrance test for students who seek admission to higher secondary school after completion of secondary school education is important for parents, teachers, and students alike. This is because the results of CET have an impact on students’ career and future. Also, it has a dominant influence on future professional objectives, social dynamics, and learning strategies of the students. Though McClelland (1973) claims that no research evidence is available that shows the correlation between test scores and the students’ actual accomplishment in job and life, it still plays a key role in getting access to quality education and environment in the college of repute (Kellaghan, 2004; Phelps, 2006). Walberg (1984) highlights different factors related to aptitude, environment, and instructions that hinder or facilitate students’ academic progress. It is therefore, important to explore how these factors mediate the school’s input for eleventh Grade students’ performance in board exams.

This section presents a review of relevant literature related to the definition, historical background and philosophical perspectives of high-stakes testing (HST), brief history, practices, limitations, and challenges of college entrance tests. Further, different factors including academic, personal, and psychosocial aspects influencing students’ achievements in science are also discussed. The literature review of research is based on the conceptual framework shown below.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

16

Secondary School Certificate I Scores

(Walberg, 1984)

Interview

Academic Factors

Computer Based Test

College Entrance Test Scores

Non-Academic Factors (Walberg, 1984)

Self-Concept Socio-economic Status

Learning Environment Resources Teachers’ Competency

Annual Board Performance in Higher Secondary School Certificate Exams

School Policies

Parental Involvement Un-precedent situations Peer Influence

Study Habits

Technology and media

Influence on Psycho-social dynamics of Testing

Figure 2.1. Conceptual Framework on Correlation between College Entrance Test and Science Students’ Performance

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

17

2.2 High-Stakes Testing (HST) Blake (2012) claims that the notion of high-stakes testing emerged in the name of democracy, meritocracy and individual freedom, however, its philosophy and purpose depict the utilitarian approach. Blake (2012) referred to the Mill (1999) who justifies that any action on ethical grounds based on the consequences; proponents of high stakes testing consider it necessary for educational standardization and development (Phelps, 2006; Ruoling, 2010). Madaus as cited in Blake (2012), presents a comprehensive definition for high-stakes testing which is referred by most of the educational researchers High-stakes test include those used for the certification or recertification of teachers, promotion of students from one grade to the next, award of a high school diploma, assignment of a student to a remedial class, allocation of funds to a school or school district, award of merit pay to teachers on the basis of their students’ test performance, certification or recertification of a school district, and placement of a school system in education receivership. (p. 30) The above definition reflects that these test scores can burnish or tarnish the reputation of an individual in public for bigger good that is improving the educational system. This is in contrast to low-stakes testing which does not have any significance or consequence for an individual in the public. The best example of later is classroom assessment where teachers’ remarks would help student to identify the strengths and shortcoming for future efforts. Thus it is the function and not the form that makes high-stakes test distinct from the low-stakes test (Blake, 2012).

2.2.1 Historical background. According to Madaus and Russell (2010, 2011), the origin of high-stakes testing is based on political motive. However, to understand this, it is important to see the development of Western schooling concept in 18th century.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

18

Though learning together in the form of group is a primitive human civilization, Plato laid the importance of schools for the ideal state. Plato mentioned in The Republic that the state should take responsibility for training children from the age of three and that each citizen could be guided by the system towards an ideal conception of justice and into the social class and occupation best suited for him. In the 18th century, La Chalotais (a proponent of national schooling in France), proposed a secular system of education to replace the religious hegemony on education. After the France’s successful model of national education as state’s function, this concept of compulsory schooling slowly penetrated to Prussia and other parts of the world in order to give their citizens cohesion, a commitment to the nation and willingness to sacrifice for its good. In order to devise a model of schooling built around a centrally controlled curriculum, several structures were introduced; for example, compulsory attendance and examinations so that system can be regulated and normalized. The accountability notion through examination is well reflected in the speech of Eamon De Valera, the Prime Minister of Ireland, who proposed before Parliament in 1941 a system of certification examinations at the end of primary school: …if we want to see that a certain standard is reached and we are paying the money, we have the right to see that something is secured for that money. The ordinary way to test it and try to help the whole educational system is by arranging our tests in such a way that they will work in a direction we want.

(Ireland. Dail Eireann cited in Madaus & Russell, 2010-2011, p. 21) Hence, in 1940s, most of the schools throughout the United States started using some form of standardized testing which was further extended as tracking and selection tool in 1950s. On October 4, 1957 the Soviet Union successfully launched the first Earth orbiting satellite, Sputnik, which shattered the Americans technological

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

19

superiority. Concern arose that American educational system was not preparing enough scientists and engineers; hence, National Defense Education Act (NDEA) was passed to provide funding to improve American schools and to promote postsecondary education to meet national security needs. Furthermore, in 1965, the Elementary and Secondary Education Act (ESEA) was passed to emphasize equal access to education and establish high standards and accountability. Hence, in the 1970s, state-mandated minimum competency testing begun. For example, National Assessment of Educational Progress (NAEP) report serves as a common metric for state’s performance on sampling basis. In order to overcome random sampling for assessment and to support standards-based education reform under No Child Left Behind Act (NCBA) of 2001, federal role in public education is expanded through annual testing, annual academic progress, report cards, teacher qualifications, and funding changes (Blake, 2012; Brosio, 2003; Kellaghan, 2004) . Recently, the highstakes testing in United States of America (USA) has taken a new form in connection to the Race to the Top movement; particularly for raising standards and aligning policies and structures to the goal of college and career readiness (Blake, 2012; Thomas, 2013).

This brief historical account depicts that the high-stake tests are made to measure the outcomes of education which reflect a larger belief in the use of metrics to determine the success of any policy and system. The same concept prevails throughout the world. As Pakistan Ministry of Education (2009) mentions in NEP: National standards shall be developed to reduce the differences in quality across regions. Assessment processes shall be standardized to become uniform across the Boards over time, so that students appearing in examinations under different Boards are assessed against standardized benchmarks. (p. 39)

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

20

2.2.2 Philosophical Issues. In the late 19th and early 20th centuries, modernism was shaped up by the modern industrial societies followed by the World War I. Modernism rejected the religious beliefs and focused on the centrality of human being for ethical decisions. All existed notions were re-examined to see what is holding back the human progress and economic stability. The concept of highstakes testing linked to educational reform emerged at the same time carrying the same utilitarian phlosophy of Mill, however, the politicization of assessment made it as a “multi-faceted and misconstrued construct within the educational setting” (Blake, 2012, p. 7). It is, therefore, criticized that “It is nothing short of disastrous that more than ever before, one antidemocratic system of ideas-market ideology-almost exclusively defines the terms of educational politics and charts the path of education reform.” (Engel as cited in Thomas, 2013, p.

87) In this section, different ideological and philosophical underpinnings of highstakes testing are explored while reflecting on its purpose, processes and impact.

2.2.2.1 Purpose. Testing in modern times is seen as a tool to developing a world-class educational system, motivating the un-motivated, filtering students on the basis of merit, increasing the nation’s productivity and restoring global competitiveness (Madaus & Russell, 2010-2011). As a result, it increases the inequity gap in society based on income, wealth, privilege, access and merit. Foucault as cited in Thomas (2013) argued that “the exanimation combines the techniques of an observing hierarchy and those of normalizing judgment. It is a normalizing gaze, a surveillance that makes possible to qualify, to classify, and to punish” (p. 89). This shows that the raise in its power within the accountability paradigm as a control system leads to the testing capitalism, where test scores as well as humans function as

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

21

commodities which are regulated by authority. This is well reflected in NEP – 2009 of Pakistan where Government assured to establish National Education Testing Service to design and administer standardized test for the admission of professional institutions and thus, control the system. Hence, the hidden purpose of high-stakes testing is to benefit those in the power having control over the system and wanting to justify their supremacy on the score based scientific objectivity. Foucault, the great modern philosopher strongly criticized this grand reality as it is using human as an object / mean to obtain the desired outcome.

2.2.2.2 Process. Though Phelps (2006) claims that high-stakes tests are more valid, reliable, and fair testing system as they are made by experts keeping all standards in mind, literature shows several issues as mentioned below: Fairness. It is known that the achievement of test scores depends upon the familiarity of context and format. Academicians raise a question on many high-stakes test as how fair it is to ask about the concept/context which child is never exposed to.

For example, McClelland (1973) highlighted the cultural clash in American Indians who are more towards non-verbal communication for learning at home and are assessed through verbal testing tool. Similarly, students with dyslexia or impaired vision and non-English speaking learners usually get marginalized due to contextual, physical, and linguistic barriers.

Validity. McClelland (1973) refers several research studies in his article which shows no direct relation of intelligence with job or life performance of students. On the other hand, it is the socio-economic status that gives access and confidence leading to success in the future. Also, high-stakes tests cover only a limited range of skills that a child might be using in the professional life. Major critique to the validity

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

22

is that if high-stakes tests is neither holistic nor representative in nature, how valid is this in determining what it is supposed to (Madaus & Russell, 2010-2011).

Reliability. Several researchers argue that sequence, selection, format of questions in the paper along with the time, space, and vigilance factors influence on students’ test scores (Kellaghan, 2004; Kirkpatrick & Zand; 2011; Madaus & Russell, 20102011; McClelland, 1973). This raises a question on the credibility and reliability of high-stakes test scores. One time activity in an artificial setting through paper-pencil test cannot provide reliable information of students’ performance.

To counter the above critique, Phelps (2006) proposed an effective student-testing system which should be multi-layered, multi-targeted and performance based.

Furthermore, Sackett, Schmitt, Ellingson, and Kabin (2001) suggested strategies to cater diversified groups of examinee and remove the culturally- biased test items.

High-stakes tests in the “age of infinite examination and of compulsory objectification” (Foucault, as cited in Thomas, 2013, p. 91), however, considered as important by many proponents based on its potential to provide information and to differentiate between self-motivated kids to those who quit trying years before (Phelps, 2006). Brosio (2003), however, critiqued the whole process of high-stakes testing with two philosophical perspectives. First, it reflects a class war where teachers are workers and belong to Proletariat class. Policy makers impose their decision on teachers in the name of competition and standardization. These imposed policies ignore teachers’ professional discretion and judgment and see them as “movable things upon the capitalist theoretical board game where everyone has equal opportunity to success or fail” (Brosio, 2003, p. 4). The victory of this capitalist imperative can be seen by “profit more pay less” philosophy in terms of teachers’

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

23

working hours, stress level, and salary. The other dimension highlighted was the educational essentialism which is reflected in the philosophy of high-stakes testing process. Essentialists believe that they know what the essential in curriculum for everyone is and what objective / standard aspects should be measured. They force schools to adopt those pedagogical forms which they consider objective and essential to achieve the goals. For example, phonics method for language development. Dewey and Freire refused this idea of “filling station pedagogy” (Brosio, 2003, p.2) as they believe that learning is very subjective and based on personal experiences.

2.2.2.3 Impact. Thomas (2013) agreed that test-based ranking of countries has no clear positive or negative correlation with the countries’ economic power. The best example is Finland which has top ranking given by Program for International Student Assessment (PISA), but is higher on the poverty scale as compared to United States of America (USA). However, as the discourse and policy is mainly driven by test data, it has economic pressure on the countries specifically developing countries where financial aids and grants are given based on students’ performance in such tests.

Standardized Achievement test in Pakistan is one example where it is made as a disbursement learning indicator (DLI) by USAID for grant release purpose. Further, at individual level, this economic pressure can also be felt by parents and schools due to high cost and funding implications. Along with the economic influence, high-stakes testing has severe impact on the psychological and physical aspects of children.

Madaus and Russell (2010-2011) refer how in 1980s and 1990s many schools renamed their intelligence test by readiness test for the admission of children to kindergarten or in first Grade due to the parents’ argument. Tagging students as poor performers not only excludes low performing students socially, but also over stresses and marginalizes them. As a result, students get demotivated; leave their studies or

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

24

become hopeless for future (Kirkpatrick & Zang, 2011; Liu, 2013). Likewise, few suicidal cases can be found every year even in Pakistan as students are overburdened with this high-stakes testing. Blake (2012) reports that severe pressure is found on the low performing schools in USA due to serious implications of these tests result which include closing of school, firing of principal and staff. Paper-pencil tests are usually based on theoretical aspects leading students to selective study and rote learning without focusing on its application in daily life. Noddings cited in Blake (2012) endorsed the criticism on HST by saying that “these have sacrificed richness, depth and creativity to a dull struggle for higher test scores and material that is quickly forgetting when the test is over� (p. 15). This teaching- to-the test approach ultimately becomes a matter of investment and return rather than focusing on the holistic aspects of children. Educational system becomes a tool to attain the hierarchy of power by making head-hand divide. There is no free market choice rather a control system and process. Labour power is commoditized and scores are considered as the scarce capital to be maintained by the law of supply and demand. Using unfair, illegitimate means and copying and plagiarizing becomes common and grades are on sale due to its significance for students’ future. Despite all these criticism by post-modernists who do not believe on objectivity and one reality, there are number of reasons why a comprehensive replacement for high-stakes testing has not been emerged such as alternatives are prohibitively expensive, excessively subjective, comically lenient, or too unrelated to the college or job screening process (Phelps, 2006). This shows that framing the issue on Either-or terms is not advisable and therefore, system improvement strategies like small class size, creative curriculum reform and collaborative professional development, should be focused which somehow got ignored due to over-reliance on testing.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

25

2.2.3 Forms of high-stakes testing in Pakistan. Considering the significance of high-stakes testing in terms of information for diagnosis, alignment for efficiency, and motivation to study (Phelps, 2006), NEP Pakistan (2009) highlights to introduce a system of evaluation which emphasizes learning of concepts and discourages rote memorization. For example, examination boards are made autonomous in order to ensure the uniform quality and standards. In addition, many initiatives have been taken to improve the quality of education through testing systems at national and international levels as a back wash effect. Following three cases indicate three different types of standardized high-stakes testing in Pakistan; namely public, national and international assessment (Kellaghan, 2004).

2.2.3.1 Public Examination. Public examination is a large scale high-stakes test to ensure the quality of educational systems in any country. In Pakistan, the first public examinations are held at Grade 9 and 10 levels which are called Secondary School Certificate (SSC) Examinations. Students of Public, private, Non-government Organizations (NGOs) and community-based schools take these exams. There also exists a different system of public examination which is run by Cambridge International Examinations (CIE). However, the affiliation to this system greatly depends upon the affordability of parents for this expensive system. There are many differences in these two systems of syllabi, conduct of exams, paper format, reliability and credibility of scores.

2.2.3.2 Standardized Achievement Test. This is an example of standardized national testing which caters a particular population to identify gaps in education delivery at provincial level which would further lead to major policy, programmatic and structural changes needed in the system. In 2012, Reform Support Unit (RSU) Education and Literacy Department, Government of Sindh initiated Standardized

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

26

Achievement Testing project to have a contextualized testing system for the Public schools in the Province of Sindh (Institute of Business Administration, 2013).

2.2.3.3 International English Language Testing System (IELTs). In order to avoid issues of credibility and norm reference assessment system, certain international standardized tests are introduced in Pakistan which tell an individual about his standing in the global market. IELTs is an example, which caters less number of people as compared to those accommodated by the above two types of standardized testing and has international recognition for university admission process in English speaking countries.

2.3 College Entrance Test In addition to the above major forms of high stakes examinations, there is an increasing trend of using testing for the sake of admission to the higher education institutions of high repute. These tests are either centralized or institution-based and have great impact of students’ academic and professional life. Following section describes its origin, functions, and trends at national and international levels, systems and criteria, benefits and limitations for the purpose of admissions.

2.3.1 A brief history. Broom (1962) as cited in Zhu (2014) claimed that the emergence of college entrance requirement started back to 17th century when Harvard being the only college at that time gave admissions based on the verbal competency of students in Latin language. This was based on the rationale that Latin was the language of scholars and hence, considered as the criterion for intelligence. With the modified concept of intelligence in the 18th century, entrance examination was introduced at Harvard in the year 1800 covering different subjects including Math, Algebra, Geometry, History, Chemistry, and Physics. The testing method at that time was oral which later became difficult to execute with the scaling of educational trend

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

27

and higher institutions in the society. This resulted in the diploma system for admission in US which means students who get diploma from preparatory schools are eligible to take admission without further exams. For a better reciprocal relationship between colleges and preparatory schools in the diploma system, both need proper coordination and timely inspection which was felt as a difficult and cumbersome activity. This intensified the need of a separate, centralized and autonomous system controlling the enrollment in higher education institutions. Thus, in the early 20th century college entrance examination board (CEEB) was established and for the first time students in the US appeared in a single test for the entrance of several universities. After two decades of conducting exams called Achievement test (AT), a new system called Scholastic Achievement test (SAT) was introduced in 1926 based on the intelligence tests developed in the late 1800’s and early 1900’s and first administered on a large scale to army recruits during World War I. In 1959, a parallel

system called American College Testing (ACT) started which was used as an alternate to SAT and was based on the concept taught at the school (Zhu, 2014). This raised a question as either intelligence or subject matter should be considered as the criteria for selection in the higher institutions of learning. Min (2013), on the other hand, argued the ineffective reliance on the academic factors only for the admission purpose as there are social and personal factors that influence on students’ performance greatly. Zhu (2014) reported that many universities considered the limitations of achievement and aptitude tests and have added personal experiences, interests, special skills and social activities into consideration for the admission. This points out that one policy fit for all is neither possible nor required and hence many institutions devised their own policies based on what they want to see in their students. Helms (2008), however, suggested a careful examination of associated issues while selecting

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

28

any specific admission policy. These issues include overall control of the tertiary education system and its impact on society and the economy; reliability and validity, equity and fairness, as well as the particular considerations of developing countries, such as resource scarcity, admission systems inherited from former colonizers, and lack of adequate data collection and analysis systems.

Countries in the sub-continent like Pakistan and India since the time of establishment relied on students’ performance in statewide national examinations.

However, these public examinations are mainly based on rote memorization and therefore, could not give true picture of students’ knowledge and aptitude (GoP, 2002). Thus, strong need of modernizing higher education specifically in Pakistan was realized in the late 1990s and early 2000s and equally emphasized by the task force on the improvement of the higher education in Pakistan (GoP, 2002). According to the task force report, it would be better to provide “a reliable alternative examination system at the SSC and HSC levels that can significantly improve education in general rather than to establish a national testing service for the purpose of selecting candidates for higher education” (GoP, 2002, p.6). The dilemma is that in all educational initiatives and planning, the higher secondary colleges have been ignored. Relying on outdated textbooks and learning by rote, they produce students who are ill-prepared for university (Promotion of Education in Pakistan Foundation, 2006). Moreover, Task Force suggested giving autonomy and freedom to every college and university in Pakistan for the selection of students of desired competency from the pool of applicants. This decentralization is a true essence of standardization of admission process which otherwise was difficult for the administrative effectively and quality assurance. However, this could not be achieved without proper training of

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

29

admissions counselors in the area of academic assessment and score interpretation (Zwick, 2007).

In Pakistan, the increased expectations for getting admission in quality tertiary education institutions had ripple effect on the higher secondary level. Most of the private institutions started using admission tests for screening competent students who then successfully complete their two years of study and outshine in the admission process of professional colleges. This however, is not the case for the admission in government colleges which still follow the Centralized admission policy (CAP) which is purely based on the students’ results in public examination at secondary level. This dual standard policy creates a social disparity among the students, educational institutions, and in general public and raises many ethical questions on the citizens’ rights to get better education from institutions of repute.

2.3.2 Functions of admission testing. Admission testing could serve many educational purposes including prediction, development, evaluation, standardization, and reporting. Though Ruoling (2010) identified social and political functions of admission tests in China, educational function supersedes all. These tests usually provide diagnostic information to the colleges about students which could be used for providing academic support once student enters in the college. Also, for evaluative purpose, it is used as a device for initial screening, tie-breaking, allocating merit scholarship, and placement.

Despite various roles, Zwick (2007) highlighted that neither all function are known to test providers nor it is fully used in any system. Mostly, it is considered as the statistical and valid measure of students’ preparation that would add value in school’s ability to predict students’ success which previous school grades alone cannot provide. Zwick (2007) however, emphasized that test scores should be

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

30

interpreted with lots of diligence for the screening purpose due to its limitations as a tool.

According to Schaefler (1962), the predictability of students’ capacity to learn the technical skill was first emerged in 1948 when most of the students in technical courses either dropped out or showed less attendance. The wastage was very high with regards to those who failed to get admission and those who though admitted either quit or had to repeat the courses due to poor academic performance. It was felt necessary to have some easy-to-administer filters through which only those applicants should be selected for the desired courses who are capable to continue their studies. Hence, admission test has been introduced in order to predict students’ fitness and likelihood to complete the enrolled course. Another aspect of predictability relates to the college grades as it is important to see how these tests correlates with the students’ grades after a year. A small improvement in the predictability is considered worthwhile from institutional point of view.

Zwick (2007) emphasized that ongoing research should be conducted at college and university level to verify the predictive validity of the admission criteria and tests.

Zwick (2007) suggested that comprehensive evaluation of tests’ validity must involve a consideration of the test’s design, development, content, administration, and use, however, this study has a limited scope that is exploring the correlation between students’ test scores and their performance in the college.

Zwick (2007) also raised the concern on small predictive value for a single schoolbased study as there are many non-academic factors which greatly influence on students’ persistence in the college. To overcome this limitation, this study is based on mixed method so that a complete picture of this phenomenon can be captured in the context of a private higher secondary school. The first year in science studies in

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

31

Pakistan is a critical time where students either develop the interest or get de-motivated in their academic endeavors (Schaefler, 1962; Zwick, 2007), hence this study focuses on the first year’s performance in the college and experiences of students at higher secondary level.

2.3.3 Systems and criteria of college admissions. Helms (2008) presented five typologies of admission systems currently existing across the globe. This includes the admissions based on secondary leaving examinations, university-based entrance tests, standardized aptitude tests, multiple examinations, and no-examination. Different representative systems of admissions are practiced in different countries with variety of combinations. For example, in France, Austria, Ireland, and Egypt, admissions are given on the basis of scores of secondary leaving examinations due to the overall strength and good reputation of the their secondary institutions. This however, does not suit in context of Pakistan where different standards at the level of curriculum, teaching learning practices and examination systems are found at SSC and HSSC levels.

This shows that there is no one “right” admission system. The effectiveness of a particular system depends highly on the context in which it is implemented, including the government structure, economic factors, the labor market, culture, and national strategic (Helms, 2008). The same was reflected in the task force report (2002) that the need for admission tests at the universities and colleges has been highlighted in almost every Five-Year Plan; however, the public universities have not adopted this policy despite the availability of funding due to lack of appropriate support from the leadership at higher levels.

Another variation at international level is observed in the focus and elements of admission testing; both in content and methodology. For example, ACT focuses on

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

32

the concepts taught at high school whereas SAT focuses on examining the intelligence of students. There has been a long debate as to what to focus on and how to extract information related to students’ ability. Zhu (2014) reported several studies showing that achievement as well as ability tests both are biased towards students of certain ethnic background. On the other hand, Ashenafi (2014) proposed a wide range of subjects, skills, and competence and personal qualities to be tested specifically for those who are opting medical courses for future study. He justified the inclusion of non-cognitive factors like basic abilities to recognize and solve problems, motivation and self-efficacy to performance in stressful environment as these have major influence on students’ performance. This cognitive notion is supported by the study of Meijer (2001) who found that lack of confidence is the main factor of test anxiety which has a negative influence on students’ performance.

Though many colleges across the globe now realize the equal importance of both cognitive and non-cognitive factors for determining the predictability of students’ capacity, this would make testing a non-scientific, cumbersome and lengthy process. Furthermore, it needs variety of testing methods to capture different dimensions of students’ academic and non-academic ability. For example, interviews/verbal test, written/paper based assessment, scores of previous exams need to be added in the overall college admission policy to increase the validity of inference (Zwick, 2007). From the learners’ point of view, Choi and Park (2013) however, argue that a wide range and focus of different colleges make it difficult for students to manage the diverse standards hence leading to overreliance of private coaching despite its high cost.

2.3.4 Limitations and challenges. Literature supports that testing approach of admission is the best way to ensure fairness and equity as this selects students on

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

33

merit (Phelps, 2006; Ruoling, 2010). It provides equal chances to students irrespective of their social, economic, and intellectual background to compete for the scarce educational resources available. Wang and Ross (2010) however, argue that in spite of the freedom of participation, this method is benefitting the superior education system and favours those students who get a chance to attend best preparatory schools. Zwick (2007) highlighted other issues like lack of validity and reliability as the limitations of admission testing. For example, testing a few subjects would be invalid measure of students’ ability and competence.

Wang (2011) suggested that wide range of subjects should be included in the admission tests to increase the test validity; however, Choi and Park (2013) negate the idea as this will increase the pressure on students. Also, Zwick (2007) claimed that test scores are not a true representation of the academic achievement. They are rather temporal learning due to short-lived usage and application in the professional context.

Though there is no such study available in Pakistan that shows the impact of coaching on students’ attainment in admission tests, general public perception favours the coaching before appearing in the test. Further, many universities rely on Multiple Choice Questions (MCQs) based test items to increase the convenience and objectivity in marking though Zwick (2007) raised validity issue due to the unskilled people preparing the questions. On the other hand, the inclusion of essay based test raises the question of subjectivity and inter rater reliability issues in marking.

This shows the real challenge for admission committees to devise a fair, valid and reliable admission policy. Hence, to avoid the fairness issues of standardized test, its biasness towards minorities and testing mania tendency, Rooney and Schaeffer (1998) endorsed the test optional approach of admission which favors both equity and excellence. They stated that more than 275 four-year colleges and universities across

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

34

the United States (U.S.) using this approach have reported an increased diversity in the applicant pool and enrolled classes without any loss in academic quality.

However, test optional admission policy required a standardized, transparent and reliable examination system throughout the country which is missing in most of the developing countries. The following section looks into the limitation of admission tests at the level of content and methodology, policy and impact.

Many researchers found that content, vocabulary, physical environment, and mode of testing have varied influences on the students of different age, race, and gender. For example, vocabularies used in most of the standardized tests are unfamiliar to the students of rural areas and hence create rural urban divide. Similarly, in a survey conducted by Wang (2011), 70 percent participants favoured not to include province specific question in entrance test which increases the probability of guessing and anticipation specifically by those studying in the urban context due to the access to resources and examination developers. He called this as “a selfprotection mechanism� (p. 23) of the provinces with superior higher education opportunities. Furthermore, Meijer (2001) concluded in his study that conventional tests are strongly biased by anxious tendencies. The sense of competition and social pressure to get admissions in good colleges create anxiety among students specifically those who have low endurance and interest for the subjects (Liu, 2013). Rooney and Schaeffer (1998) claimed that test scores do not equal merit and most of the time underrepresents certain groups. It is therefore, not recommended as a screening method for the female and minority students (Micceri, 2010). To overcome the bias issue, Zwick (2007) suggested to go for differential prediction of a criterion and item content and to remove the biased test items.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

35

The second aspect is admission policies which implicitly serve and give credit to certain class of society. Zwick (2007) claimed that fairness in the admission process cannot be attained unless the policies and processes are made un-politicized.

Wang (2011) mentioned in his study that due to provincial fixed quota admission, policy in China though initiated in the name of regional equality, provoked the emergence of numerous “profit-seeking behaviors,” like “college entrance examination migration” (p. 21). He also observed that the elite institutions did not follow the policy hence, cause the provincial discrepancies to continue to grow. GoP (2002) suggested a decentralized policy of admission to give ownership to individual institutions to exercise their freedom and to set priorities. However, the finding of the same study showed that independent enrollment scheme was found unfair as presented with high eligibility criteria expectations which were not possible to meet by vast majority of candidates. Similarly, for the exemption on entrance test for fast track admissions, admission recommendation policy award for talented and outstanding students had been introduced in China. However; children of state and social administrators and skills personnel got benefited 73 times and 25 times higher than those of farmers. This shows that policies can easily be manipulated by human influence.

Third aspect underpinning the limitation of admission testing is the impact of admission test. Deil-Amen and Tevis (2010) showed that both low as well as high performing minority students had negative influence of admission test. 20 percent low performers reported in the study that these test scores had a strong and negative impact on their postsecondary plans and life aspirations. On the other hand, for high performing minority students, these test scores though boosted their confidence with regard to their ability, provided a sense of false hope which they barely met once they

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

36

entered into their first year of college due to non-familiarization and nonpreparedness. Another aspect highlighted by Wang and Ross (2010) study is that the entrance test has significant influence on the teaching methodology of teachers as qualifying the entrance test is an accountability criterion for them. As a result of this admission seeking approach of teaching, the “modern students are heavy on knowledge, but light on ability� (Liu, 2013, p. 42). Another major concern raised by Choi and Park (2013) is its financial impact. In addition to the expenses of admission tests, parents send their children to private course prior to their entrance test which is quite costly and very difficult to bear for those who are from low socio-economic background.

Despite all issues, universities consider standardized admission testing essential due to the need of marketing, profit and political pressure (Rooney & Schaeffer, 1998). Helms (2008) shared few reforms in various countries for varied goals including increased fairness and equity to reduce the influence of corruption. He suggested going for an incremental and evolutionary model of admission reforms.

Zwick (2007), however, argued that any change in the content and mode of administration will not reduce the degree of controversy as these tests are a central component of a mechanism for allocating scarce resources. Thus college admissions tests will always be a focus of public debate. Phelps (2006) suggested that in the absence of a better selection process, it is not wise to abandon the existing one which at least have some scientific, rational and fair system in place. In lieu of this, existing literature has missed out to explore the merit and demerit of entrance examinations in a larger social context and to find the impact of few practical reform strategies.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

37

2.4 Factors Influencing Students’ Performance Educational outcomes are largely associated with the well-being of an individual and the country, hence, it is given significant importance throughout the globe. Though there are some limitations in measuring the educational inputs and outputs leading to immediate and long term goals, many studies have analyzed the educational productivity along with the associated factors. Walberg (1984) identified nine factors that “require optimization to increase affective, behavioral and cognitive learning” (p.20) and categorized these into three groups namely aptitude, instruction, environment. Moreover, Farooq, Chaudhry, Shafiq, and Berhanu (2011) reported that the economic, social and cultural aspects of individual are the powerful factors influencing performance in science. Besides this, several studies affirm the psychological influence of appreciation, role modeling, positive remarks, social expectations, and perceived image leading to self-fulfilling prophecy and improved performance (Tierney & Farmer, 2011; Singh & Baksh, 1977). It is, therefore, very important to have a clear understanding of all the factors benefitting and hindering the academic progress of an individual.

Following section presents the review of relevant studies on two broad themes: academic and non-academic factors influencing on students’ performance in science.

2.4.1 Academic factors. Students’ performance in the test is highly reflective of their cognitive ability, career aspirations, and study skills. Lau and Roeser (2002) argued that inclusion of cognitive and motivational factors with science achievement, engagement, and choice, among high school students increases the prediction of students’ engagement during testing, in the classroom, and outside of school, and also to the prediction of their science test scores, classroom grades, and anticipated

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

38

choices, of science-related activities in the future. They found that these psychological factors are more potent predictors as compared to the demographic characteristics as these are linked with the “individuals’ perceived competence and efficacy for succeeding on a task, as well as their valuing of those tasks, predispose them to invest their time and cognitive and emotional resources in those activities” (Lau & Roeser, 2002, p. 158 ). Students who use study timetable, organize their work, monitor their own progress and set their performance targets have shown improved learning and test performance in science (Mukhwana, 2013; Onabamiro, Omoruyi, Soyingbe, & Rosiji, 2013). This reflects that cultivating good study habits, self-image, and selfmonitoring processes in students can improve achievement within their own microsystems. However, over emphasis on students’ characteristics and negligence on the school culture, climate, and policies can undermine the holistic nature of academic aspects leading to students’ unexpected performance.

Oseguera (2013) presents a school condition framework depicting the role of school in supporting students’ college aspirations and learning. First aspect that framework emphasizes on is the role of rigorous academic curriculum as Oseguera (2013) claimed that “advanced courses not only foster academic achievement, they are a precondition to college admission” (p. 3). The same is observed when students affiliated with one examination board outshined in the aptitude tests of professional engineering college in Karachi as compared to those who were associated with the board following limited content and syllabus at higher secondary level (Daily Jung Newspaper, September 3, 2015). However, the large gaps between the written and taught curriculum in science education deepens the need to explore the actual implementation of these advanced curriculum in the classroom.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

39

Second aspect of an effective school is quality teaching. Teachers’ attitude, perceptions, and practices play foremost role in science learning as these influence greatly on students’ level of anxiety (Planchard, Daniel, Maroo, Mishra, & McLean, 2015; Sami, Erbilgin, & Arikan, 2016), conceptual understanding (Tai, Sadler, & Mintzes, 2006), performance and interest in science (Odom & Bell, 2015). There is, however, no clear consensus on the role of teacher certification (Oseguera, 2013) on students’ achievement. This shows that certification does not guarantee the pedagogical changes at classroom level. This is reflected in countries like Pakistan where despite the availability of experienced and so called certified teachers, the lack of teacher’ competency and willingness are the major hindrances in developing learner-centered classroom. One reason could be the mismatch of teacher certification courses and the practical issues in the classroom. For example, despite the over emphasis of interactive pedagogies in science classroom, many teachers do not use these due to large class size. This might be due to the scarcity of required resources, difficult administration of hands-on activities and classroom management concerns in large class settings. Similarly, the strong correlation between the instruction time and students’ learning (Huang, 2015) shows its importance however, it raises major concern on its cost effectiveness and practicality. This raises a major question on the applicability of teacher education programs in the developing context like Pakistan which faces major educational challenges like inefficient use of resources, poor administrations and governance, inadequate funding, and politicization (GoP, 2002, PEPF, 2006).

Third aspect Oseguera (2013) emphasizes in school condition framework is the safe, secure, and quality atmosphere which includes learning, physical, and the social environment in any school context. Learning environment engages students in

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

40

effective use of time and resources and thus enhances their learning. Oseguera (2013) found that students referring to the reading materials, employing study discussion groups and attending symposiums, field trips and exhibitions have shown better results in their examinations. Areepattamannil and Kaur (2012) on the other hand, emphasized the availability of quality teachers as an important element in developing learning environment. They found that students who attended schools with teacher shortage performed low on the science assessment than their counterparts attending schools without teacher shortages. This proves the dire need of attention on the teachers’ retention and professional development policies of schools. Similarly, disciplinary policy and practice in school also contributes in the learning culture of school, however, would damage the sense of belonging if the rules are perceived as unfair (Sami et al., 2016). For example, students with low motivation and ability usually do not complete their assignments as if it is found too challenging or less appealing to them (Planchard et al., 2015).

Students therefore, should be facilitated with extra support and counseling rather being punished. Another aspect is the safe and adequate school facilities which are proved to have a positive influence on students’ performance. Sami et al. discussed that a well-maintained school building, physical environment, technology based facilities influence faculty and students’ morale and boost their confidence and motivation towards learning. On the contrary, majority of public schools in Pakistan lack basic facilities, hence, results in students’ poor attendance and poor learning (Pakistan Ministry of Education, 2009). The significance of physical environment, however, does not undervalue the social environment within the school. The culture of social and academic support and relevant services including test preparation

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

41

program, counseling services enable students to seek timely help from their peers and adults resulted in increased sense of belonging to school.

The above literature supports the idea that producing and maintaining positive environment in the school for learning is key responsibility of school management and can be attained by organizing educational and extracurricular activities, formulating appropriate policies and creating supportive culture in the school.

Sacrificing the normal student life and increasing the workload in the name of accountability would decline students’ interest and motivation for studying (Nordmo & Samara, 2009) and performance (Gandara & Randall, 2015). Furthermore, Tai et al. (2006) claim that high school science experiences has been manifested in the college performance of students and hence, should be given high importance at all educational levels due to its carry forward effect.

2.4.2 Non-academic factors. Family characteristics like socio economic status (SES), parental qualification, and home environment are significant predictors for students’ performance at school. There are numerous studies showing that lower SES levels lead to lower performance of students in science studies (Areepattamannil & Kaur, 2012; Bellibas, 2016; Farooq et al., 2011; Huang, 2015; Robbins, Lauver, Le, Davis, Langley, & Carlstrom, 2004). The reasons could be low access to learning resources, low self-concept, and poor physical and mental health conditions. To meet the family’s basic needs, many at times students start earning money at a very young age. Distraction created as a result of financial instability ultimately influence on their poor performance and persistence in the school. To overcome the SES constraints, Stewart, Lim & Kim (2015) suggested the role of school by increasing the learning time through extra classes and encouraging them to establish more positive perceptions of their persistence. Bellibas (2016) on the other hand highlighted the

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

42

importance of home education resources and technology-based facilities to benefit students from low SES. The finding is equally supported by other researchers who argued on the importance of proper diet, financial support and basic facilities at home for better learning (Saleem & Qureshi, 2011, Robbins et al., 2004). This shows that a holistic approach is needed to overcome the socio economic barrier of students’ learning and hence, a proper coordination is required between school and home.

Other non-academic factors include family’s support in providing exposure, parents’ education and professional status as these could develop child’s confidence, skills and attitude towards learning. Walberg (1984) reported that the indicators like supervised homework and school parent programs to improve academic conditions in the home are twice as predictive of academic learning as is family Socio-economic status (SES). However, without consistency, sustenance, and alignment in these efforts, correlation fades out. Several studies show the positive correlation between parents’ education on students’ performance (Jabor et al. 2011; Min, 2013; Sami et al., 2016) though the relation with parents’ occupation is still ambiguous as different studies showed varied correlation (Jabor et al., Farooq et al., 2011).

This could be because educated parents serve as a model for learning, determine the educational resources available in the home and hold particular attitudes and values towards education. The direct role and support of parents however is difficult to justify for the students at higher educational level who are relatively independent in their studies and more influenced by peers and schools.

Considering parents’ education as universal factor for determining students’ performance, Jabor et al. suggested the nationwide policies and programs for citizens’ education. This also reflects the important role of significant others in developing self-efficacy in students for the improved performance by influencing feelings,

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

43

thinking, motivation, and behaviors towards learning. Further, this self-efficacy leads to positive career aspirations (Onabamiro et al., 2013).

Several empirical studies support the idea that appreciation, role modeling, positive learning environment and support are key factors in developing positive selfconcept which is then reflected in students’ performance (Areepattamannil & Kaur, 2012; Ashenafi, 2014; Jabor et al. 2011, Reitzes & Mutran, 1980; Singh & Baksh, 1977). This shows that the self-concepts are formed as reflections of the responses and evaluations of others about oneself which can be metaphorically explained as looking-glass self. Gecas and Schwalbe (1983) however, critiqued this notion as this gives “overly passive and over-socialized view of human beings” (p. 77). They believe that beyond the looking-glass of self is a self that develops out of the autonomous and efficacious actions of the individual. Self-esteem, thus, evolves by both efficacious action and reflected appraisals though former leads to earned selfesteem whereas later results into the one presented by “significant others” (Gecas & Schwalbe, 1983, p. 714).

To achieve the objective of improved students’ performance in science, it is necessary for the educators to understand how different factors may contribute in the academic success of Grade 11 science students. To determine all the influencing factors in a single attempt is a complex and difficult task, however, this study has used phenomenological approach in exploring these elements with a small, but representative sample having scattered nature of student factors, family factors, and peer factors.

2.5 Conclusion This chapter presented the review of relevant literature based on the conceptual framework of this study. It highlighted the historical and philosophical

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

44

perspective and issues of high stakes testing along with its major forms which currently exist in Pakistan. Furthermore, this chapter also elaborates a brief history, functions, systems, and criteria of college entrance test along with its limitation and challenges. In the last section of this chapter, academic and non-academic factors affecting science students’ performance in the context of high stakes examinations are discussed.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

45

CHAPTER III RESEARCH METHODOLOGY 3.1 Introduction The study is designed to address several gaps in the literature; namely, students’ views about the role and model of college entrance test, correlation between CET scores and annual board performance of Grade 11 science students and their perception on different factors influencing their learning in science. To explore the above areas, a mixed method study has been designed. This chapter describes the overall research design and the rationale of its selection. Later part covers the context of the study, sample and sampling process, data collection methods and tools, data analysis procedures, and ethical considerations during the study.

3.2 Research Design Mixed methodology though a complex process, best suits for those research problems whose complexity cannot be addressed through either descriptive or statistical analysis. Its central premise is that the use of both qualitative and quantitative approaches in combination provides a holistic picture of the area of study.

Quantitative data provides a closed-ended view to the stated problem and uses statistics or scores to answer the research question or hypothesis. On the other hand, qualitative data consists of an open-ended information which provides a rich and multidimensional analysis of a research problem. Hence, mixed method offsets the weaknesses of both quantitative and qualitative research approaches, and gives comprehensive evidence on the area of study. Though there are many approaches to mixed method (Cameron, 2009; Creswell & Plano Clark, 2011; Ponce & Maldonado, 2015), the choice of design to be used is based on the aim of the study and further suggests the data collection and analysis process.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

46

The research problem of this study is based on the need to explore the relationship of CET with Grade 11 students’ performance in annual board exams along with the factors influencing the relationship. Hence the study suits and suggest mixed research design which as per Cohen, Manion, and Morrison (2007) uses “numerical and verbal data, in order to gather rounded, relatable data” (p. 96). The nature and purpose of this study is two-fold: 

First aspect was to find the relationship between CET score and students’ performance in annual board examination, which was done through ex post facto correlational design of quantitative analysis. According to Cohen et al. (2007), the weakness of ex post facto study lies in the fact that neither control nor randomization of the independent variable is possible and therefore things have to be taken as they are and efforts cannot be made to try to disentangle them. However, they considered it a valuable exploratory tool which is mainly suitable for the educational context. In this study, sample selected were kept as homogeneous as possible to introduce measure of control as recommended by Cohen et al. (2007).



Second aspect tried to find an in-depth qualitative input to explore the personal experiences of students to gain an overall picture of the problem.

For this, phenomenological approach is used to examine how students are making sense of their personal and social world (Moustakas, 1994). The interpretation of students would provide rich experiential descriptions about different factors influencing their performance in science subjects and hence would validate the quantitative endeavor.

The objectives of the study led to a parallel phases mixed model that helped to understand the research problem in an integrated way and to find out answers with

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

47

“greater certainty in inferences, conclusion or statements which formulate its findings” (Ponce & Maldonado, 2015, p. 114). Moreover, convergence design was used to see the research problem in entirety which means quantitative findings to understand the objective aspect whereas qualitative data to facilitate in making sense of the subjective aspect of the study. The convergence of qualitative and quantitative data has allowed the researcher to explain how CET predicts students’ future performance in science and what factors influence it, according to the experiences and perceptions of the students. The identified factors would become a basis for future causal study in order to develop a comprehensive predictability model including both academic and non-academic factors.

3.3 Context of the Study The context of this study is AGAZ (pseudonym), one of the reputed private higher secondary schools in Karachi, Pakistan. This school was established in 1995 with a total 700 students and initially operated in a single shift. In the beginning, the school was affiliated with BIEK which conducted annual examinations for students enrolled for a two-year academic session in pre-medical, pre-engineering and commerce groups. Considering the validity, alignment, and fairness issues associated with BIEK, the school management decided to get affiliated with a new board simultaneously. Students at the time of admission decide which examination board they want to opt for and in case of successful admission they study the syllabi offered by the respective board. Currently, 1940 students are on board in two shifts under the disciplines of pre-engineering, pre-medical, commerce, and computer science.

A Master degree in the subject of teaching is the requirement for teachers whereas their performance during the hiring process is the main criterion for selection and not prior experience. The hiring process includes steps like a professional

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

48

teaching demonstration of a subject-specific content area, content-based test, language proficiency test, and interview. The school is considered as one of the best colleges in Karachi both for academic performance and co-curricular activities. Students participate and succeed both at local, national and international levels in various domains like literary activities, research, sports, and other activities.

There were some reasons for the selection of this context for the study. Firstly, this school is considered as one of the best higher secondary schools in Pakistan, specifically, for science (pre-medical) group. Every year, students outshine others in their annual examination performance and hence get admission in professional medical colleges and universities of high repute. As a result, there is an increasing pressure from students and parents for getting admission in pre-medical group at AGAZ Higher Secondary School every year. Secondly, since the beginning, this school has been conducting the college admission test in order to ensure merit-based admissions from the large number of applicants every year. The mode of test however, was changed from paper-pencil test to computer-based test (CBT) in 2007 for the purpose of ease and technological advancement. Hence, the context was appropriate to explore the predictability of CET scores for students’ annual performance at higher secondary level. As per the eligibility criteria for admission mentioned in the college prospectus 2014, students who expect to clear the secondary school certificate (SSC) (or its equivalent) annual examination 2014 from any recognized Board/University with a minimum aggregate of 65% marks in the science group can be included in the admission process. The college admission is based on three criteria: (a) meeting the required cut off score (as decided by the AGAZ Higher Secondary School’s selection committee every year) in the Computer-based Test, (b) SSC I results, and (c) the interview. These three components have different weightage

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

49

that is 40%, 40% and 20% respectively in the overall selection process. Computerbased test for pre-medical group comprises 15 questions each from chemistry, mathematics, and physics, 25 questions from biology and 30 questions from English.

Thirdly, the professional association of the researcher with this school helped her to appreciate the social context of the school and to relate the same with the study findings in order to understand its implications. Likewise, the study would have a reciprocal impact on the school in the form of future reforms in policies and practices.

3.4 Population The population for the study included a batch of pre-medical students who appeared in the aptitude tests of AGAZ higher secondary school for the year 20142016. In the year 2014, a total of 1238 applicants applied for admission in pre-medical group (Student Affairs Department of AGAZ higher secondary school), from which 271 were offered admission to HSSC I in the first shift. Out of these 271 students, 236 enrolled themselves for the study in Grade 11 pre-medical. These students are from mostly upper middle and upper class families with different language, socioeconomic, religious and cultural backgrounds 3.5 Sample and Sampling Procedure The study was conducted with a total sample (n=225: n1=116-AKUEB, n2=109-BIEK) who were enrolled in the college in summer 2014 and continued to study in the school for the whole academic session. These students have completed 10 years of schooling and have opted to study in the college for a two-year higher secondary school certificate program which is the requirement for their future admission to professional colleges and universities. For the qualitative part of the study, three focus groups were formed from each examination system (AKUEB and BIEK) on the basis of students’ ability in terms of their HSSC I board’s results. These

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

50

three groups include those who secured less than 60%=low achievers (LA), 60%75%=mid-level achievers (MA) and above 75%=high achievers (HA). There were 10 students in each of the six focus groups that were selected randomly. Though the overall ratio between boys and girls in pre-medical group is very low, it was tried to keep the gender balance in each group. However, this was not possible in one group of low achievers due to the presence of girls only in the group. Eight students out of total 60 refused to participate in focus group discussion and therefore their names have been replaced randomly by other eight students from the respective lists.

Though the teachers were not the primary research participants, for the purpose of triangulation of findings, one-to- one informal talk to six teachers (each from biology, chemistry, and physics subjects from AKUEB and BIEK) teaching the students of pre-medical group was carried out.

3.6 Data Collection Methods and Tools As this study mainly examined the correlation of students’ academic performances in the college’s entrance test and annual board examinations with reference to science subjects, the data were based on the scores of CET and HSSC I. Archival data used in this study was obtained from pre-existing records that were available to the researcher through the student affairs department of AGAZ Higher Secondary School. However, to ensure accuracy, some random check from the primary sources (SSC II and HSSC I mark sheet) was done. Detailed personal and academic profile of each student was obtained that included college registration number, name, address, school and scores of SSC, examination boards at SSC and HSSC levels, scores of CET and HSSCI. This detail was helpful as it showed the heterogeneity in the sample and hence ensured the validity of findings. Further, the

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

51

information facilitated the researcher in exploring relationship between various factors to get a better understanding of the area of study.

For qualitative part of the study, in-depth semi-structured interviews with students were taken to capture experiences, feelings, and perceptions by probing and expanding participants’ responses. Six focus group discussions were carried out for about 60-90 minutes each. Out of these, three focus groups from each examination system (AKUEB and BIEK) based on ability in terms of board’s results were formed (which comprised less than 60%=low achievers, 60%-75%=mid-level achievers and above 75%=high achievers). In each focus group, 10 participants were invited (five males and five females), except the low achievers’ group where the number and gender criteria were not possible due to limited number of students. In one group, four students did not turn up on the day of interview due to their other academic and personal commitments.

Self-developed semi-structured interview tool was used though the interview process was kept evolutionary and flexible to get thick and rich data on the focused factors affecting students’ performance. The need was felt during focus group discussion to collect life stories of two outliers, one from each, low ability and high ability group, hence one to one discussion was carried out for about an hour each. To triangulate the findings, one to one informal talk was also carried out with all six teachers teaching different subjects (biology, physics, and chemistry) to pre-medical group at AGAZ Higher Secondary School. These conversations were recorded after taking participants’ consent on consent letter (see Appendices A & B). Further, interview form used by AGAZ Higher Secondary School Management for admission purpose was analyzed to validate students’ perceptions related to the interview component of CET.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

52

3.7 Data Analysis Initially, a comparison of annual scores in science subjects with cumulative scores of all components was done to find the overall correlation. However, during the analysis process mean, standard deviation and correlation between other factors like SSC and HSSC with respect to examination systems and ability groups were also studied as suggested by researchers’ supervisor to lessen the researcher’s overlook on important aspect in the process of students’ performance in science. However, these analyses are shared in the qualitative section of Chapter 4 for data triangulation purpose. In addition, the scores in each of three components of CET were compared with annual board’s result of science subjects through SPSS for regression analysis.

In the qualitative part of the study, data analysis process began with the data collection. According to Glesne (1999), it is not a mechanical process, but one that begins when one starts thinking and reflecting, makes connections between data, gains new insights, and prepares for write-up. During this part of the study, data analysis was done at two stages that are: ongoing analysis during focus group discussion and collective analysis at the end of discussion. Participants were bilingual in the interviews, as they were more comfortable with this.

For participants’ validity, at the end of each discussion, the researcher verbally shared with them the important points they had made during the conversation. Also, recordings were listened to several times and then interviews were translated directly while listening to the recorder. Coding was done based on the emerging ideas from the data and some pre-determined categories based on the research question. These codes were analyzed to see the emerging categories which then helped the researcher in identifying major themes. Linkages between different themes were developed to

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

53

build the connectivity and to check the validity and triangulation of data (Cohen et al., 2007).

3.8 Ethical Consideration The most fundamental step for the ethical dimension in research study is informed consent, where the participants are informed about the nature and purpose of the research, its risks and benefits (British Educational Research Association, 2011).

Consent from AGAZ school management was taken for the use of statistical data on students’ marks in CET and board’s exams; however, consent from only those students was taken who were engaged in focus group discussions. Similarly, written consent was taken from teachers before engaging them in one to one talk. In both cases, the consent form was shared and explained to participants (students and teachers) for clarifying to them the purpose of the study, their rights to withdraw at any stage of the study, or to refuse to reply to any question.

Also a copy of their signed consent form was provided to those who requested for their reference. Data was kept at a safe place and was never shared with anyone with the exception of supervisor for accuracy of data analysis and prevention of errors while conducting hypothesis testing. Confidentiality of data was maintained and anonymity was ensured during report writing.

3.9 Conclusion Considering the problem and purpose of the study, a mixed methodology research design was used, which was found appropriate to seek answers of questions from various perspectives. This chapter described the qualitative and quantitative methods used for the data collection and for the triangulation and validating the findings. Further, the ethical dimensions which were taken care of during the study were also discussed in the chapter.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

54

CHAPTER IV DATA ANALYSIS AND RESULTS 4.1 Introduction The study is intended to examine the predictive validity of college entrance test scores, which comprises CBT, SSC I, and Interview scores as criteria in the admission process to AGAZ Higher Secondary School in Pakistan. The study also examines the factors influencing eleventh grade students’ annual board performance based on their real world experiences during the first year of college life.

This chapter presents the results of the data analysis of quantitative as well as qualitative parts in two different sections. First section that is quantitative part presents data analysis according to the order of the research hypotheses. It discusses the correlation of CBT scores with respect to each science subject and then cumulative science scores of HSSC I. Further, correlation of CET scores with Grade 11 students’ science performance in two different examination systems is also explored. Lastly, finding from the regression analysis of the predictor variables and first-year college scores is also shared. Second section that is qualitative part highlights students’ perceptions related to the role of testing, CET model being used, and factors influencing their performance in annual exams. In this section, stories of two outliers each from low and high ability groups are narrated to understand the phenomena.

4.2 Quantitative Data Analysis Hypothesis 1: There is a significant relationship between CBT scores and students’ performance in HSSC-I Annual Examinations with reference to physics, chemistry, and biology.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

55

A. Correlation between HSSC I & CBT- Biology Table 4.1 Students’ Performance in HSSC I & CBT- Biology

Domains

N

M

SD

HSSC I Biology (%)

225

78.71

12.80

CBT Biology (%)

225

57.38

13.58

Note: HSSC I = Higher secondary school certificate- part I, CBT=computer-based test, M=mean scores, SD= standard deviation, N=sample size.

Table 4.1 illustrates that with n=225 in CBT Biology, the mean score (M1) is 57.38 and standard deviation (SD1) is 13.58 whereas n=225 in HSSC I Biology, mean score (M2) is 78.71 with standard deviation (SD2) 12.80.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

56

Table 4.2 Correlation of Students’ Performance in HSSC I & CBT- Biology Domains

HSSC I Biology (%)

CBT Biology (%)

HSSC I Biology (%)

CBT Biology (%)

1

0.35** 0.00 225

Pearson Correlation Sig. (2-tailed) N

225

Pearson Correlation Sig. (2-tailed) N

0.35** 0.00 225

1 225

**. Correlation is significant at the 0.01 level (2-tailed)

Note: HSSC I = Higher secondary school certificate- part I, CBT= Computer based test, N=Sample size.

A Pearson correlation coefficient was computed to assess the relationship between the scores of CBT biology and scores of HSSC I biology. The study found that the relationship is highly significant and helps to predict the scores of HSSC I biology examinations. The findings imply that CBT strategy of testing biology content is reliable in predicting students’ performance of biology in HSSC I annual examinations. There was a positive correlation between the two variables, r=0.35 (two tailed), n=225, p=0.00 at 0.05 level of significance.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

57

B. Correlation between HSSC I & CBT- Chemistry Table 4.3 Students’ Performance in HSSC I & CBT- Chemistry Domains

N

M

SD

HSSC I Chemistry (%)

225

75.24

13.76

CBT Chemistry (%)

225

50.93

15.92

Note: HSSC I = Higher secondary school certificate- part I, CBT=computer-based test, M=mean scores, SD= standard deviation, N=sample size.

Table 4.3 illustrates that with n=225 in CBT Chemistry, the mean score (M1) is 50.93 and standard deviation (SD1) is 15.92 whereas n=225 in HSSC I biology, mean score (M2) is 75.24 with standard deviation (SD2) 13.76.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

58

Table 4.4 Correlation of Students’ Performance in HSSC I & CBT-Chemistry Domains

HSSC I Chemistry (%)

CBT Chemistry (%)

HSSC I Chemistry (%)

CBT Chemistry (%)

1

0.37** 0.00 225

Pearson Correlation Sig. (2-tailed) N

225

Pearson Correlation Sig. (2-tailed) N

0.37** 0.00 225

1 225

**. Correlation is significant at the 0.01 level (2-tailed)

Note: HSSC I = Higher secondary school certificate- part I, CBT= Computer based test, N=Sample size.

A Pearson correlation coefficient was computed to assess the relationship between the scores of CBT chemistry and scores of HSSC I chemistry. The study found that the relationship is highly significant and helps to predict the scores of HSSC I chemistry examinations. The findings imply that CBT strategy of testing chemistry content is reliable in predicting students’ performance of chemistry in HSSC I annual examinations. There was a positive correlation between the two variables, r=0.37 (two tailed), n=225, p=0.00 at 0.05 level of significance.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

59

C. Correlation between HSSC I & CBT- Physics Table 4.5 Students’ Performance in HSSC I & CBT-Physics Domains

N

M

SD

HSSC I Physics (%)

225

76.63

14.93

CBT Physics (%)

225

56.26

16.36

Note: HSSC I = Higher secondary school certificate- part I, CBT=computer-based test, M=mean scores, SD= standard deviation, N=sample size

Table 4.5 illustrates that with n=225 in CBT physics, the mean score (M1) is 56.26 and standard deviation (SD1) is 16.36 whereas n=225 in HSSC I physics, mean score (M2) is 76.63 with standard deviation (SD2) 14.93.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

60

Table 4.6 Correlation of Students’ Performance in HSSC I & CBT-Physics Domains

HSSC I Physics (%)

CBT Physics (%)

HSSC I Physics (%)

CBT Physics (%)

1

0.50** 0.00 225

Pearson Correlation Sig. (2-tailed) N

225

Pearson Correlation Sig. (2-tailed) N

0.50** 0.00 225

1 225

**. Correlation is significant at the 0.01 level (2-tailed)

Note: HSSC I = Higher secondary school certificate- part I, CBT= Computer based test, N=Sample size.

A Pearson correlation coefficient was computed to assess the relationship between the scores of CBT physics and scores of HSSC I physics. The study found that the relationship is highly significant and helps to predict the scores of HSSC I physics examinations. The findings imply that CBT strategy of testing physics content is reliable in predicting students’ performance of physics in HSSC I annual examinations. There was a positive correlation between the two variables, r=0.50 (two tailed), n=225, p=0.00 at 0.05 level of significance.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

61

D. Correlation between HSSC I & CBT Science Table 4.7 Students’ Performance in HSSC I & CBT- Science Domains

N

M

SD

HSSC I Science (%)

225

76.85

12.65

CBT Science (%)

225

55.32

11.81

Note: HSSC I = Higher secondary school certificate- part I, CBT=computer-based test, M=mean scores, SD= standard deviation, N=sample size

Table 4.7 illustrates that with n=225 in CBT science subjects, the mean score (M1) is 55.32 and standard deviation (SD1) is 11.81 whereas n=225 in HSSC I science mean score (M2) is 76.85 with standard deviation (SD2) 12.65.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

62

Table 4.8 Correlation of Students’ Performance in HSSC I & CBT Science Domains

HSSC I Science (%)

CBT Science (%)

HSSC I Science (%)

CBT Science (%)

1

0.57** 0.00 225

Pearson Correlation Sig. (2-tailed) N

225

Pearson Correlation Sig. (2-tailed) N

0.57** 0.00 225

1 225

**. Correlation is significant at the 0.01 level (2-tailed) Note: HSSC I = Higher secondary school certificate- part I, CBT= Computer based test, N=Sample size.

A Pearson correlation coefficient was computed to assess the relationship between the scores of CBT science and scores of HSSC I science. The study found that the relationship is highly significant and helps to predict the scores of HSSC I science examinations. The findings imply that CBT strategy of testing science content is reliable in predicting students’ performance of science in HSSC I annual examinations. There was a positive correlation between the two variables, r=0.57 (two tailed), n=225, p=0.00 at 0.05 level of significance.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

63

Hypothesis 2: There is a significant relationship between CET scores and students’ performance in BIEK examinations with reference to science subjects.

Table 4.9 BIEK Students’ Performance in HSSC I-Science & CET Domains

N

M

SD

HSSC I Science (%)

95

80.06

12.39

CET (%)

94

58.71

5.97

Note: HSSC I = Higher secondary school certificate- part I, CET=College entrance test, M=mean scores, SD= standard deviation, N=sample size, BIEK=Board of Intermediate Education Karachi

Table 4.9 illustrates that with n=94 in CET, the mean score (M1) is 58.71 and standard deviation (SD1) is 5.97 whereas n=95 in HSSC I science mean score (M2) is 80.06 with standard deviation (SD2) 12.39. Sample size for testing hypothesis two is reduced to 94 due to the missing data for the CET of those students who have come from the Cambridge system. Cambridge International Examination (CIE) system provides subject wise grades in the student’s statements of result instead of marks.

Hence, the missing data were deleted from the main sample.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

64

Table 4.10 Correlation in HSSC I-Science & CET for BIEK Students Domains

HSSC I Science (%)

CET (%)

HSSC I Science (%)

CET (%)

1

0.70** 0.00 94

Pearson Correlation Sig. (2-tailed) N

95

Pearson Correlation Sig. (2-tailed) N

0.70** 0.00 95

1 94

**. Correlation is significant at the 0.01 level (2-tailed)

Note: HSSC I = Higher secondary school certificate- part I, CET= College entrance test, N=Sample size, BIEK=Board of Intermediate Education Karachi.

A Pearson correlation coefficient was computed to assess the relationship between the scores of CET and scores of HSSC I science for BIEK group. The study found that the relationship is highly significant and helps to predict the scores of HSSC I science examinations. The findings imply that CET overall is reliable in predicting students’ performance of science in HSSC I for BIEK annual examinations.

There was a positive correlation between the two variables, r=0.70 (two tailed), n=94, p=0.00 at 0.05 level of significance.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

65

Hypothesis 3: There is a significant relationship between CET scores and students’ performance in AKUEB examinations in science subjects.

Table 4.11 AKUEB Students’ Performance in HSSC I-Science & CET Domains

N

M

SD

HSSC I Science (%)

105

73.34

12.21

CET (%)

103

55.67

5.03

Note: HSSC I = Higher secondary school certificate- part I, CET=College entrance test, M=mean scores, SD= standard deviation, N=sample size, AKUEB=Aga Khan University Examination Board

Table 4.11 illustrates that with n=103 in CET, the mean score (M1) is 55.67 and standard deviation (SD1) is 5.03 whereas n=105 in HSSC I science mean score (M2) is 73.34 with standard deviation (SD2) 12.21. Sample size for testing hypothesis three is reduced to 103 due to the missing data for the CET of those students who have come from the Cambridge system. Cambridge International Examination (CIE) system provides subject wise grades in the student’s statements of result instead of marks. Hence, the missing data were deleted from the main sample.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

66

Table 4.12 Correlation in HSSC I-Science & CET for AKUEB Students Domains

HSSC I Science (%)

CET (%)

HSSC I Science (%)

CET (%)

1

0.51** 0.00 103

Pearson Correlation Sig. (2-tailed) N

105

Pearson Correlation Sig. (2-tailed) N

0.51** 0.00 105

1 103

**. Correlation is significant at the 0.01 level (2-tailed)

Note: HSSC I = Higher secondary school certificate- part I, CET= College entrance test, N=Sample size, AKUEB=Aga Khan University Examination Board

A Pearson correlation coefficient was computed to assess the relationship between the scores of CET and scores of HSSC I science for AKUEB group. The study found that the relationship is highly significant and infers the scores of HSSC I science examinations. The findings imply that CET overall is reliable in predicting students’ performance of science in HSSC I for AKUEB annual examinations. There was a positive correlation between the two variables, r=0.51 (two tailed), n=103, p=0.00 at 0.05 level of significance.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

67

Hypothesis 4: CBT scores, SSC I scores, and Interview scores predict Grade 11 science students’ overall performance in annual board examinations.

Table 4.13 Multiple Linear Regression Model Summary and Overall Fit Statistics

Adjusted R Model 1

R .662a

R Square .43

Square

Std. Error of the Estimate

.42

7.15

a. Predictors: (Constant), Interview Scores (%), CBT Scores (%), SSC I scores (%)

Table 4.13 shows the multiple linear regression model summary and overall fit statistics. It is found that the coefficient of determination R² = 0.43 that means that the linear regression explains 43% of the variance in the data.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

68

Table 4.14 Analysis of Variance and Model Significance

Model 1

Sum of Squares df

Mean Square

F

Sig.

Regression

7680.47

3

2560.15

50.07

.000b

Residual

9868.43

193

51.13

Total

17548.91

196

a. Dependent Variable: HSSCI Sores (%) b. Predictors: (Constant), Interview Scores (%), CBT Scores (%), SSC I Scores (%)

The linear regression's F-test has the null hypothesis that there is no linear relationship between the variables (in other words R²=0). The dependent variable is HSSC I scores whereas Predictor variables are interview scores, CBT scores, and SSCI scores. The F-test is found highly significant. From the analyzed data, it is inferred that there is highly significant linear relationship between CBT scores, SSC I scores, and interview scores with HSSC I scores. This signifies the fact that interview scores, CBT scores, and SSCI scores are highly significant to predict HSSC I annual examination scores.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

69

Table 4.15 Model Coefficients Table

95.0% Unstandardized Standardized Coefficients

Coefficients

Std. Model

B

Error

Confidence

Collinearity

Interval for B

Statistics

Lower Upper Beta

t

Sig. Bound Bound Tolerance VIF

1 (Constant) 13.66

7.10

1.92

.05

-.34

27.66

CBT Scores (%)

.45

.05

.50

8.53

.00

.35

.56

.84

1.17

SSC I Scores (%)

.40

.08

.26

4.48

.00

.22

.57

.84

1.18

Interview Scores (%)

.05

.04

.06

1.16

.24

-.03

.14

.96

1.04

a. Dependent Variable: HSSCI Sores (%) As all the variables are included in the multiple linear regressions, beta weights and collinearity in Table 4.15 are interesting. Beta expresses the relative importance of each independent variable in standardized terms. Firstly, it is found that CBT scores and SSC I scores are the most significant predictors, secondly it is established that CBT score has a higher impact than SSC I scores (beta = .50 and beta = .26). Interview scores are found relatively less significant predictor (beta=0.06).

Also, Table 4.15 checks for multi collinearity in multiple linear regression model.

Tolerance should be > 0.1 (or VIF < 10) for all variables, which they are.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

70

4.3 Qualitative Data Analysis Perceptions play a vital role in the development of attitude and identity which ultimately influence the actions and performance of an individual. Though this is primarily a correlational study, it is important to explore students’ perception regarding the testing and learning experiences at higher secondary level in order to gauge the effectiveness of testing paradigm. Further, quantitative or numeric data do not give a complete picture of any phenomena unless it is enriched with the real life experiences of the people involved in the process. This section presents students’ views shared during focus group discussions about the role and model of CET and the influencing factors on students’ performance in annual board exams. Moreover, teachers’ opinions are also shared for the purpose of data triangulation.

4.3.1 Students’ perceptions and the role of CET. Students considered college entrance test as a tool to judge one’s ability and prior learning. Majority of students called it a “fair mechanism to differentiate between capable and incapable and to identify most deserving students for quality education” (BIEK_HA: FGD, dated, 22-22016).

A student reflected, (In a context like Pakistan) it (CET) is important because of infra-structure limitations. If institutions are in small numbers then how will they manage the large influx of students coming to those universities and schools? If we take into consideration the data; in Karachi all medical universities select total 600 students and applicants are 23000. (AKUEB_HA: FGD, dated, 22-2-2016) This fact shows that scarcity of quality educational resources signify the role of CET in the name of talent hunting for merit based admission. Students shared the value of CET at individual as well as organizational level. Firstly, they appreciated CET as a

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

71

second chance for those who are intelligent, but could not score good results in their secondary board exams for any reason. This would give them an opportunity to exhibit their potential which might not be catered by public examinations. Further, at a higher secondary level, they found the whole process of admission a learning experience which will be beneficial once they apply for professional colleges or job.

For example, they learned how to cope with the anxiety level while attempting the aptitude test paper and how to present themselves in the interview. Nevertheless, they emphasized the sharing of CET results with students, as this would help them to identify their weakness and to work upon them. Secondly, students highlighted the unfair examination systems and different standards of education existing in the country.

In such a situation, they considered it important to have a fair and standardized system of quality induction.

This is equally important for the educational institutions of a high repute who want to offer their best services to those who have actual potential and talent. This merit based admission policy “contributes to the reputation of institutions” (AKUEB_HA: FGD, dated, 22-2-2016) and hence become a necessity for their survival. Students at the same time questioned the role of highly reputed schools for the development of an individual and society as if they give admissions to good students to produce good results.

They also shared the discomfort in the discrimination caused by this phenomenon as it deprives many students of their rights of quality education. They also, realized the limitations of CET in determining one’s capacity. A student said, Intelligence cannot be judged by entrance test as luck majorly works…. May be the same questions come in examinations which I’ve practiced.… Intelligence has no bound. Intelligence is not only related to science or math.… I do not

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

72

think test can justify how much capability a person has. (AKUEB_MA: FGD, dated, 1-3-2016) This raises a question on the validity of entrance test due to its limited scope and time constraints. Students said that CET has a high stake on their future and hence create a social pressure as well as fear. Though few students considered this fear a source of motivation and push for learning, its various social and psychological impact on the students of low ability are to be realized. A student stated; Fear is the greatest motivation. Fear of not getting admission in good universities … in the society if you want to gain a respectable position then you need to get a good percentage. In other countries even a bus driver is a respectable person.… But here if you’re not a doctor, no matter how good looking you are, you will not even get a good wife. (AKUEB_HA: FGD, dated, 22-2-2016) This shows that CET decides students’ admission in the quality institution which ultimately guarantees their decent future and social reputation. This leads to a competitive culture in the society where everyone is working hard for their survival. In this race, however, low ability students suffer the most. Inability to get admission leads to demotivation, poor self-concept and hopelessness among students which sometimes results in school drop outs or shift in the career. Interestingly, a student who successfully entered the AGAZ Higher Secondary School realized that by getting admission, she developed confidence, but afterwards faced difficulty in surviving in a more competitive environment. These findings show that CET sometimes gives false hopes to students and become a barrier for them to think of their actual capacity and foresee the upcoming challenges. Besides all these limitations and challenges, students

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

73

appreciated the CET as this is the only parameter available for screening the best among the mass.

4.3.2 Students’ perceptions and the CET model. Students criticized traditional testing approach and called it “old and failed phenomena” (AKUEB_HA: FGD, dated, 22-2-2016) to judge one’s capacity. They argued on the limited scope and time constraints in annual examinations which is being used as selecting criteria for public colleges. One student commented, … but the thing is three hours of paper. This cannot decide my fate. In three hours if my luck is not with me.…or for any reason if I could not perform well in those three hours. On this basis, thousands of students could not get into the good colleges. (BIEK_MA: FGD, dated, 7-1-2016) Contrary, they appreciated the diverse tools used in the CET of AGAZ higher secondary school including entrance test, previous board’s result and interview to test students’ knowledge, skills and attitude. Students’ critical views related to each component of CET are mentioned below.

4.3.2.1 Annual SSC board examinations. Students perceived SSC I result as a major criteria having highest weightage in the overall admission policy, hence deciding the selection of a student in AGAZ Higher Secondary School. They realized its importance as it reflects students’ past efforts, however, suggested to minimize its weightage or to limit it only as an eligibility criteria for appearing in the college entrance process. They justified their suggestions on the basis of two reasons. Firstly, they evaluated different examination systems prevailing in the country namely Cambridge system, AKUEB, Hyderabad board, Punjab board, Karachi board, and many more. They found that the examination papers from these diverse systems have different format, difficulty level and assessment criteria. Hence it is difficult to judge

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

74

the standard value of scores obtained from these varied systems. This perception is also endorsed by independent sample t-test of SSC I scores of AKUEB and BSEK students (see Appendix F). Result showed that there is a significant difference between the two groups’ SSC I scores (AKUEB n1=101, M1=77.92, SD1=5.85; BSEK, n2=96, M2=83.49, SD2=5.24). Secondly, they shared that over reliance on past questions paper, unfair practices at board, cheating culture during exams, and absence of inter-rater reliability in some of the examination systems make it unfair to use the scores as a screening tool. Further, one student questioned the SSC I cumulative score policy for college admission. She said, You consider SSC result overall in admission policy. It might happen that a student is good in one subject and poor in other. Like, I am not good at math. If result drops then how would you decide who is good and who is not.

(AKUEB_HA: FGD, dated, 22-2-2016) This shows that overall subject scores might not represent students’ capacity in science subjects. However, this perception was rejected by the correlation value, r=0.54 (two tailed), n=191, p=0.00 at 0.05 level of significance which is found highly significant between SSC overall and SSC science scores of students affiliated with National Examination boards (see Appendix G). Teacher on the other hand supported students’ view and said, “now professional medical colleges have started focusing on the aggregate science scores for their admissions as they do not care what have they (students) scored in social sciences” (chemistry teacher, dated 16-3-2016).

4.3.2.2 Computer Based Test: In the absence of a single, reliable and standardized public examination system in Pakistan, students considered it important to have a college-based entry test in order to judge students’ conceptual understanding of science. They appreciated technology integration in admission testing in comparison

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

75

to the paper pencil test as former gives a quick and reliable marking and provides less chances of cheating due to randomly selected questions by the software. One student shared his experience of computer based test; The test was computerized so I did not put too much efforts … my writing skills are not tested rather my concepts were checked…. It was good in the sense that it negated all those factors which are not actually required for the science students. (BIEK_MA: FGD, dated, 7-3-2016) Conversely, students from BIEK realized that computer-based format is not aligned with the requirement of public examination board which actually check those characteristics which the college does not focus on in entrance test” (BIEK_MA: FGD, dated, 7-3-2016). They shared that annual exams filter students based on their recall ability, writing speed and way of presentation whereas CBT was based on students’ conceptual understanding. This gap between the CBT and HSSC I board approach raises a validity issue of test indicators in CBT for measuring competence. However, this perceived gap does not produce any impact on the students’ performance as shown by highly significant correlation value, r=0.64 (two tailed), n=109, p=0.00 at 0.05 level of significance between CBT and HSSC I science performance of BIEK students (see Appendix G).

Another concern raised by students was the bias in the content, format and mode of computer-based testing. They felt that CBT is based on the MCQs design which are aligned with AKUEB pattern and hence benefit those students. However, independent sample t test suggest no significant difference in the CBT science performance (AKUEB, n1=101, M1=55.61, SD1=10.449; BSEK, n2=96, M2=54.28, SD2=13.60) of AKUEB and BSEK students (see Appendix I). On the other hand, students’ identified few science concepts covered in the test which were not the part of CIE syllabi, for

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

76

example animal kingdoms in biology. Also, CBT marginalized those who have come from the remote areas and don not have these facilities at home and hence become pressurized and anxious due to this mode of testing. Further, students from BSEK raised their concern regarding chemistry and biology subjects which they studied in grade IX and therefore faced difficulties while attempting CBT in these subject areas.

They found AKUEB on edge as they studied these subjects in grade 11 as well as grade 10. The perception of students when tested through the independent sample t test (see Appendix F), it is found that there is no significant difference observed between AKUEB and BSEK students’ performance in CBT-chemistry (AKUEB, n1=101, M1=51, SD1=15.43; BSEK, n2=96, M2=49, SD2=16.64) and CBT-biology (AKUEB, n1=101, M1=58.57, SD1=12.38; BSEK, n2=96, M2=56.58, SD2=15.16).

These quantitative evidence shows that there is no content and construct bias that could lead to the fairness issues in the CBT as a screening tool for college admissions.

4.3.2.3 Interview. Considering the limitations of content-focused approach of SSC and CBT exams, students appreciated the inclusion of interview process in the entrance policy. They found it an excellent opportunity to meet the school leadership, showcase their confidence, awards and accomplishments and share their hidden talents which is neither represented by CBT nor SSC I scores. One student commented, “It (interview) judges your confidence that how well you can speak, how good you can present… how you interact with people, it also identifies other talents, for example sports” (AKUEB_HA: FGD, dated, 22-2-2016).

Teachers equally appreciated these criteria and considered it a real demonstration of one’s personality that influences the predictability of students’ participation and active involvement in science classrooms. This perception of teachers is supported by significant correlation co-efficient (r=0.16 two tailed, n=220,

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

77

p=0.01 at 0.05 level of significance) found between interview and HSSC I science scores (see Appendix L). Besides this, students acknowledged the friendly atmosphere during interviews. However, they criticized the formal and mechanical approach used by the interviewer which leads to the fake and fabricated answers. A student shared, How can you judge someone in five minutes? We knew what questions are being asked in the interview and we have prepared ourselves accordingly. I pretended that I am very studious ... We had to be fake in the interview as we wanted to come here. I have seen a person who took preparatory classes for interview, he was taught what question will he be asked, how to sit, what to answer. (BIEK_MA: FGD, dated, 7-3-2016) Another student said, You tell me if you ask someone’s name and hobby what will you come to know … in 5 minutes I can be a champion. And those who are taking interview, do they have sufficient knowledge of psychology to understand what the student is hiding from them. It seems a formality just to keep a standard. (BIEK_MA: FGD, dated, 7-3-2016) Students suggested having a panel of interviewers including a psychologist and a content person who can judge students’ aptitude and attitude towards studies.

When students were asked how they felt about the predictability of CET model for students’ annual science performance, they found both parameters distinct and unrelated. One student said, “It is the check as to how much knowledge we have grasped in the past years. Now next what is coming depends on the students and the school, how teachers help you in preparation. Both are different” (AKUEB_MA: FGD, dated, 1-3-2016).

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

78

Students considered that the role of CET is limited to admission only as there are different academic and non-academic factors influencing students’ performance throughout the year. On the contrary, science teachers realized the great potential of CET model for the predictability of students’ annual performance. However, they raised their concern on their access to this data for the formative purposes.

4.3.3 Students’ perceptions and factors influencing their performance.

The quantitative data has shown significant correlation between CET scores and annual performance of students, hence revealing the high level of predictive validity of the CET model. However, no significant correlation is observed between SSC II and HSSC I scores for the low ability students (r =0.28, n=17, p=0.26 at 0.05 level of significance) and mid-level ability students (r=0.060, n=83, p=0.59 at 0.05 level of significance) in contrast to the high ability students (r= 0.42, n=125, p=0.000 at 0.05 level of significance).

Further, Appendix H show that the mean scores dropped from SSC II to HSSC I in case of low ability (74% to 54%) and mid-level ability groups (75% to 68%) whereas students in high ability group sustain their scores (81%) in HSSC I. Though the sample size in case of low ability group is too small to claim anything, the sharp drop in the HSSC I scores depicts different factors in their first year of college studies influencing their board performance. To explore this area further, this study can be replicated in future specifically for low ability and high ability groups with a larger sample. Nevertheless, this finding is validated with the life stories of two outlier students, one from each ability group in order to gain deeper understanding of these factors and their impact on students’ performance.

First story is about Habiba (pseudonym) who secured 83% marks in SSC II, but dropped her percentage to 58 percentage in HSSC I in the board examinations.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

79

Second case is about Murtaza (pseudonym) who sustained his overall percentage in SSCII (82) and HSSC I (81).

Story of Habiba. I come from a middle class family. In fact, my family belongs to the northern areas of Pakistan. It was when my eldest sister decided to seek higher education; she, along with my father and mother, came to Karachi for the first time. Later, we all shifted to Karachi in the year 2004. We are four sisters and two brothers, living in a two-room apartment with my parents and eldest brother’s family.

Moreover, other relatives do come and stay with us whenever they come to Karachi.

My father is a security guard in a reputable private hospital and is the only earning member since we came to Karachi. Despite the financial constraints, he has continued to support extended family members on regular basis. Though my parents are uneducated, they have high hopes for me. Particularly my father always had a dream that I will become a doctor, wear white coat and serve in the hospital where he works.

After coming to Karachi, I got admission in grade I in a community-based school located in the same society where our family resides. The school caters to the need of students from low socio-economic background and is affiliated with Karachi board. During my stay at school, I observed that there was no library in the school or structured co-curricular activities. Teachers used chalk-and-talk method of teaching and prepared us for board exams through five years papers. I used to memorize things and hence always stood first in my school till Grade 8. In board exams, I secured the highest marks in my school. This resulted in gaining teachers’ appreciation, closeness and friendly attitude which gave me self-confidence and self-identity. I wanted to fulfill my father’s dream and hence decided to opt for the pre-medical group to become a doctor.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

80

My father wanted me to join the best college of Karachi. He used to say that I would accomplish his dreams. My friends and relatives always criticized me saying that securing marks in Karachi board is not a big deal. If I am really capable, I should go for AKUEB. I also heard that AKUEB develops studentsâ&#x20AC;&#x2122; conceptual understanding. This motivated me to go for AKUEB as I wanted to strengthen my concepts before I entered any professional college. On the other hand, neither any of my family members knew about AKUEB nor was I clear about the differences between these two examination boards i.e. AKUEB and BIEK. I always thought that as I am a good student, I can learn from any examination system. The day I was filling my form at home, everyone was busy in their work, no one sat beside me to guide me. While I continued to fill the form with confusion in mind, my brother-in-law and sister asked what I was doing. They suggested me to go for AKUEB as I can do it because I am the most intelligent among the siblings. After listening to that comment, I marked on AKUEB and finally got admitted in AGAZ higher secondary school.

On my first day in the school, I wanted to build an impression on teachers, but could not get any opportunity. I knew that first impression lasts forever. With the passage of time, I realized that I did not know basic concepts which my teachers expected everyone to have. My peers who had come from AKUEB always had an edge during class discussion due to their prior understanding of those concepts.

Slowly and gradually I lost my confidence. I used to curse my previous school and teachers on the sub-standard teaching learning process.

In the beginning, I followed Karachi board pattern, but then realized that for AKUEB, I should prepare from different reference books. It was very difficult for me to refer to so many books as I was not accustomed to it. Understanding the multiple

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

81

choice questions and relating it with the correct concept was the most difficult task for me. I could memorize detailed answers, however; I found it difficult to understand pictures and diagrams given in the questions. My biology teachers used to say that I did not understand the questions completely, but assumed it and wrote responses to questions on my own. During parent teacher meeting I got an idea that I have to put in extra efforts. I decided to solve the past papers and get feedback from the teachers.

This would give me an idea about the nature and format of questions given in the exam paper by AKUEB. I started solving past papers and gave it to concerned science teachers.

However, never got it back with their remarks or feedback hence this practice did not benefit me a lot. Also, due to the strict and biased attitude of few teachers I always felt disrespected and demotivated. Once the teacher gave us a question to solve in two minutes whereas I did not know how to use calculator as I never used those functions before. She pointed out in front of all students and said that if others can do this task in limited time why cannot I. Whenever I tried to ask questions from her she always responded by saying “you know it, think about it yourself”. I never got any clarification from her. This stopped me in questioning during the class.

I never shared these experiences with my parents as I was of the view that they will not understand my problem. My father always had this stereotypical view that teachers are always right and if there is some issue with the teacher, it must have been created by me. Therefore, I did not find it useful sharing my academic problems. They only had high expectations as they used to say, “We know you can do it”. I could not ask them to arrange a tuition for me as this was never practiced in my home and nobody could expect that I need tuition. In addition to the challenges at school, I faced certain problems at home. My parents used to share their financial problems in my

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

82

presence. My father maintains files for each of his children on the expenses he has done on each one. Such discussions sometimes frustrated me and I thought how I would justify all those expenses with my low performance in school. Also, my eldest brother is a drug addict as well as psychiatric patient. He used to shout and create problems in the home after every few days. Further, a small house with many people around with noise all the time never gave me an opportunity to concentrate on my studies.

I consider these few factors as main reasons of getting low percentage in HSSC. Now as I have got C grade, I have decided to change my field. I do not want to continue my career in science any more. Though my father wants me to join nursing, but I do not have any interest in that field. I have started a youth group and joined an NGO. I will pursue my higher studies in political sciences and will grow my career in social development sector. (Interview, dated 6-6-2016) Story of Murtaza. I am from an upper middle class family and my parents originally belong to Gilgit Baltistan. My father shifted to Karachi when he was eight years old. He was posted as Army officer and got retired in year 2004. After this, he pursued his Master degree in welfare studies from China and started his own business.

I was born in Karachi and am the youngest among my siblings. My siblings consisting of four brothers and a sister. We are well qualified and settled in our life. My eldest brother has completed his Masters from Karachi University and then did his PhD from Sweden. Currently, he is running his own software house in Sweden. My father also owns some land in the Gilgit Baltistan, however, my father had to face a lot of issues to get his share. Despite the initial financial constraints in the family, he never compromised on the quality of education of his children. I was sent to the best Army school and then finally shifted to a well-reputed private school in Karachi in Grade

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

83

two. Whenever I got distracted from studies, my father used to say, â&#x20AC;&#x2DC;You should have control over yourself as education is the only way to change your lifeâ&#x20AC;&#x2122;. How my parents perceive the value of education is reflected in the fact that my mother completed her graduation when she conceived her second baby.

My school had a good infra-structure and rich learning resources, for example, library, computer lab, multimedia. I got a blend of experienced and fresh teachers in my school. My biology teachers used interactive teaching methods that developed my interest in the subject. She is a mentor who believes in extended learning and hence never limited us to course books. She used to share information and learning resources through emails. Further, my extra-ordinary performance in sports, cocurricular activities and academics earned me a good image in school.

I always received affection and admiration from my teachers in my school life.

In SSC, I got A-1 grade, but I was not the highest achiever in my school. This resulted in self-motivation to put in extra efforts in SSC II. I took tuitions from a teacher who was very experienced and believed in me. He wanted us to have prior and post reading of the concepts taught in the tuition and this practice led me to a habit of selfstudy. With all hard work, exposure and better understanding of examination format, I secured highest marks in SSC II in my school.

I always wanted to become a medical doctor; however, it became reinforced when I completed my SSCII. I was given freedom by my parents to select my career field. I was inspired by one of our family friends who was a doctor. His stable life and exceptional personality stimulated me to go for the medicine field. I took admission in Pre-Medical AKUEB group at AGAZ Higher Secondary School as my previous experience with this board was very pleasant. This examination system has a good reputation in the society. I always got admiration from people around whenever I

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

84

shared my affiliation with this board. Furthermore, it is based on concept building so continuation with this board was very important for me to prosper in the professional colleges.

In AGAZ Higher Secondary School, I experienced a lot of pressure of testing and high expectations. I observed that there is a culture of competition for marks which creates stress and burden on students. Yet, I identified a few factors which helped me in being able to succumb the pressure and stress which most of my peers could not. First was my self-controlled nature. My father had an army background; this instilled the attitude of discipline in me. This attitude has been reflecting in my academic endeavors. I always completed my homework in the afternoon, went for sports in the evening and slept early at night. I never delayed any of my school assignments and tasks. This consistent and organized behavior helped me to excel in AKUEB annual examinations. The second factor was my family’s support. I live in a five-roomed apartment with four family members where one room is allocated to me for my studies.

When I sit for studies, I am never disturbed by anyone, in fact I am facilitated by being provided food and other stuff whenever required. Whenever I get stressed, my father makes me relax and takes me out for dinner. He says, ‘Give your best and then have believe in God for the result’. I offer prayers regularly and I believe that gives me strength to overcome the challenges. The third factor was the encouragement I received in the AGAZ Higher Secondary School. I was selected as a head body due to my leadership skills, disciplined nature, and academic performance.

My teachers gave positive remarks to my parents during parent-teacher meetings.

Once my chemistry teacher said, ‘Murtaza, I must say that students like you are a blessing indeed’. I felt honored to receive words of appreciation and praise from my

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

85

teachers. I never hesitated in asking questions in the science class. Though my peers criticize me for asking so many questions, but my teachers never discouraged me. I want to pursue my career in the medical filed.

After MBBS from Pakistan, I will go abroad for further studies as I have financial support and professional guidance from my family. After specialization in general surgery or neurology I will come back to Pakistan. (Interview: dated 10-62016) The above two stories and students’ focus group discussions highlight major factors in one’s life that influence the performance in science at higher secondary level. These emerging factors are: 4.3.3.1 Teachers’ attitude and practices. Students accepted the significant influence of teachers on their science learning. They considered that teachers’ practices and attitude played a vital role in developing their interest and in building their concepts in science. A student commented; Teacher plays a major role in our performance in science subjects. A teacher is the one who attracts those who do not want to learn. What is the use of having acquired Masters’ degree if they are not capable of delivering their knowledge?

(AKUEB_LA: FGD, dated, 23-2-2016) Students argued about the teachers’ hiring policy and also suggested to involve students in the decision making process while appointing a new teacher. Subsequently, many students appreciated those teachers who put in extra effort both during class and outside class to facilitate the students. Students from BIEK low ability group mentioned following comment to share the positive influence of extra classes taken before exams as a part of academic support program.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

86

chemistry teacher taught three of us separately in the library. We got focused attention. There was no one else so we could ask any thing that was unclear to us.

This really helped me in the exams as many concepts got cleared. (BIEK_LA: FGD, dated, 23-3-2016) High achieving students also acknowledged the time given by teachers to clarify the concepts left uncovered due to her participation in co-curricular activities. Similarly, students praised few teachers on their interactive teaching style and methodology. A student said, Our biology teacher was so good, whatever we were taught by her in class, we did not need to go through it again. We used to go home, revise it and it was like we already have everything in our mind. (BIEK_HA: FGD, dated, 22-22016) Students favored the content enrichment for concept building; however, those who were affiliated with BIEK considered it purely teachers’ responsibility. One student said Enrichment is good, but we cannot do it without teachers. Nobody goes through subject related videos at home. What is the need to come to college then? Why are we paying fee? In college, we sit together and watch these videos and a teacher is there to guide you. (BIEK_HA: FGD, dated, 22-2-2016) All students shared their concerns on the uneven distribution of syllabus coverage which resulted in a hasty syllabus completion, additional home task and not taught science concepts. A student commented, “College treats you as school student, but doesn’t teach you as a school student” (BIEK-LA: FGD, dated, 23-2-2016). This shows that majority students could not cope with this pressure. In addition, students’

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

87

criticized the marks oriented approach of science teachers which reflects in their attitude and communication with students. Students from low ability group mentioned.

Teachers teach us only to get good grades. The reason we are not inclined towards studies is because we know that they are teaching us only for grades and we cannot get good grades in any way. So what is the need to study? We do not get any motivation. (AKUEB_LA: FGD, dated, 23-2-2016) Students pointed out that teachers judged them on the basis of grades and treated them accordingly. In all this, the mid-level ability students suffer to some extent. For instance, the teachers constantly support the high achievers, whereas, the low achievers are given extra time and practice outside classroom. Students who mid-level achievers feel themselves left out due to the indifferent attitude of teachers.

Teachers don not have expectation from us. They have few selected students whom they prepare for so-called ‘position’. They also give support to low achievers because if they fail, it will create problem for their image. But those who are in the middle really need to be polished; teachers leave them as they know we would neither get position nor we fail. (BIEK_MA: FGD, dated, 7-32016) The same was highlighted by teachers that they mainly focus on the high and low achievers due to its high stakes on their professional appraisal. Further, the unmet expectations of teachers, lead to their humiliating attitude and discouraging remarks which make students feel disconnected and demotivated. A student said; There is a way of saying anything. It matters a lot as we are teenagers studying in co-ed system. If you say that you won’t get admission in professional college, you cannot do anything in life, you are failed then we get demotivated.

(AKUEB_MA: FGD, dated, 1-3-2016)

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

88

It was found that teachers’ strictness for the sake of maximizing students’ potential for best result has led to students’ frustration, hopelessness, and disinterest.

Importance of science teachers’ attitude for students’ self-concept and determination is also evident from the stories of Habiba and Murtaza. On the other hand, teachers highlighted that pressure of producing good results on them consumes all their energy in making students prepared for the exams and hence leaves no time for students’ counseling and personal development.

4.3.3.2 Parents’ role. Students realized the active role of parents in their education as a student said “First step of parents for child’s education is the life changing moment for him/her” (BIEK_LA: FGD, dated, 23-2-2016). They spoke about the parents’ influence in their selection of science disciplines, career, and learning institutions. Habiba in her story mentioned that she has selected pre-medical because her father wanted her to become a doctor. Similarly, in few cases, the decision to seek admission in AGAZ Higher Secondary School was based on the parents’ wish as they wanted their children to study in the best college of Pakistan. Students, however, considered parental influence over career selection as a hurdle in achieving their dreams.

A student shared that he wanted to choose his job of working in an NGO, but due to lack of awareness, parents never appreciated him for the same rather forced him to opt for medical college. One student from high achievers’ group shared, “Since the beginning, I was told to become a doctor…. Nobody ever asked me to study for learning, but to become a doctor” (AKUEB_HA: FGD, dated, 22-2-2016). On the other hand, one middle level achiever reflected.

In most of the cases parents force their children to study medicine. A child wants to get admission in commerce whereas parents send them to medical college.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

89

How could that child learn about anything if he does not have interest?

(AKUEB-MA: FGD, dated, 1-3-2016) Furthermore, students considered parents’ education as key factor influencing their science performance as it boosts students’ confidence, social-image as well as academic target. A student shared.

In our society, this is a perception that the cobbler’s son will become cobbler and doctor’s son will become a doctor. This is mind set. If parents are educated, then children will acquire good qualification. My father is a doctor; everyone among his friends is doctor…. I need to prove myself competent if I want to create space for myself in that circle. (AKUEB_LA: FGD, dated, 23-2-2016) Parents’ education is also responsible to play a monitoring role, as a student shared, “if parents are educated, they can monitor the child. Otherwise the child can easily ditch them” (BIEK_MA: FGD, dated, 7-3-2016). Also, students recognized that if any or both of the parents are from science background, it would become a source of inspiration and guidance for the student. One student shared, “Parents can do career counseling…. We sometimes don’t know where to go. If parents are aware of it, they guide us what to do and how to, specifically when they are in the same career field” (BIEK_HA: FGD, dated, 22-2-2016). Following comment of student reflects the psychological impact of parents’ professional life on students’ motivation to go into the science related career fields.

I take example of my mother, she is a doctor. I am also becoming a doctor. It’s like you get an inspiration when you see her in front of you. She has been working for around 30 years. I can see how successful she is. How she is managing her family life successfully. I get inspiration from her. (AKUEB_HA: FGD, dated, 22-2-2016)

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

90

However, all students agreed that parents are motivating factor for their performance irrespective of their qualification.

My parents are motivation for me.… I always think that they are putting so much efforts to manage my studies, I should do something in return. I feel guilty when I waste my time and think that I am not doing justice with their efforts.

(AKUEB_MA: FGD, dated, 1-3-2016) Habiba’s story also reflects that despite the low level of education of her parents, the emotional association with them motivated her to give best performance. This realization of parents’ support was evident among all students, however, low ability students showed discomfort with the parents’ unrealistic expectation and taunting behavior with them. A student said; Whenever there is parent-teacher meeting, I try my best to ensure they do not attend. I know my teachers will not praise me no matter how well I do in the examinations. Then my parents will repeat the same thing in front of my siblings that I do not study and waste my time in college. (BIEK_MA: FGD, dated, 7-32016) Similarly, Murtaza’s story suggests the importance of psychological and emotional support from parents for students’ academic achievement.

4.3.3.3 Socio-economic status (SES). Students pointed out that socio-economic status has significant effect on the academic excellence of a student. Habiba’s story reflects that due to the financial constraints she attended a low standard school which resulted in her poor concept building in science subjects. A student comments, “If your basic education is good then everything is easy for you.... If you have not gotten good exposure, your personality will not groom” (AKUEB_HA: FGD, dated, 22-2-2016).

This indicates that access to quality institutions at primary level provides a strong

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

91

psychological and academic basis to students. However, this is equally important for the tertiary education where socioeconomic factor influences students’ career selection.

A student shared, Financial status of family affects a lot. I wanted to become a doctor, but I cannot because my father cannot afford the expenses of medical college. Whenever I share my wish, he says what if you clear the test; you will waste someone’s seat.

I know how my father is managing my school fee. (BIEK_LA: FGD, dated, 232-2016) Students criticized the education system in Pakistan where access to quality education is only possible for those who can afford it. A student said.

This is an issue in our system. If we go to Germany, education from primary to higher level is free because they know that these children are the future of the country. If they get education, they will benefit us only. Here education has become a business, we sell education.… I have seen many people who could not get admission in DOW (public medical college in Karachi, Pakistan). And because they can’t afford private colleges so they leave their aspirations.

(AKUEB_HA: FGD, dated, 22-2-2016) In addition, students from low SES mentioned that they do not ask their parents for extra money to buy the notes recommended by science teachers as they know the circumstances and finical constraints. As a result, they do not buy teachers’ note instead, get the selected pages photocopied. This shows that socioeconomic status also influences students’ access to learning resources leading to poor performance in annual board. Secondly, students realized the importance of SES on their psycho-social wellbeing. They told about their peer who despite being capable enough showed lack of confidence and expressed his worries on what if his scholarship would be discontinued.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

92

This shows that scarce financial resources would lead to psycho-social pressure that ultimately has an influence on the studentsâ&#x20AC;&#x2122; performance in science subjects. A student reflected My father works on daily wagesâ&#x20AC;Ś though he tries not to give any pressure to us. But when we see him coming tired after driving on Karachi roads throughout the day and discussing in front of us that he has earned very little, I feel bad.

Then your class fellows go to KFC (restaurant) every time and ask you to come along where as you have to see your pocket as well. When you sit for studies these, things come into your mind and you get distracted. (BIEK_MA: FGD, dated, 7-3-2016) A student shared his struggle to overcome the barrier of SES by doing a job along with his studies in order to afford his college fee.

I am from a family where I even cannot think of getting admission in this college. I joined job immediately after SSC and continued till December to earn at least an amount sufficient to pay my fee of first year. I was doing night job and then coming to college in the morning ... six months were very challenging and I literally used to sleep in the class. (AKUEB_MA: FGD, dated, 1-3-2016) This shows that handling dual responsibilities is a challenging task for students and hence it influences their academic performance.

4.3.3.4 School culture and policies. Students highlighted the reasons of opting for the AGAZ Higher Secondary School, which includes acquiring quality education, getting multiple opportunities in the form of co-curricular activities and meeting the social expectations of family members and relatives. They acknowledged the high reputation of college in the society due to its physical infra-structure and discipline.

Also, they appreciated the academic rigour and quality of preparation provided by the

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

93

school for their performance in annual board examinations. This was further proved by the correlation value (r=0.84 two tailed, n=225, p=0.00 at 0.05 level of significance) between school’s internal exams scores and the annual board examination scores (see Appendix G).

This shows that school prepares students according to the demand of respective examination system to which student is affiliated. On the other hand, students showed their anxiety on the strict policies, frequent testing practices and grade seeking approach leading to the competitive and self-centered culture in the school.

A student shared that the merit-based selection develops a culture of learning and competition that helps them to increase their efforts and standards for the survival. On the other hand, most of the students including high ability group condemned the competitive environment of the school as it is creating pressure and frustration for them.

The same is reflected in the following comments from students.

Teacher, just think, it was our last paper of mid-term, we were happy that we were free and we would enjoy. They gave us a test schedule. It was only 10 days winter holidays; we had covered no further chapter. And when we came in January, they gave us our test again on the same topics that we did in midterm. The entire month of January was wasted in this. (AKUEB_HA: FGD, dated, 22-2-2016) Everybody expects the best of you. If someone does not have the capacity to get an A-one grade, why does anyone pressurize him to get A-one grade. … If they would give up, then they won’t be able to get an A grade even, which they are capable of.

(BIEK_LA: FGD, dated, 23-2-2016) Students found test-retest approach a real burden on them and a time waste activity.

They think that most of the students who cannot cope with the burden, either give up or show non serious attitude towards it. Students appreciated the internal examinations

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

94

as it gives them an idea of their own progress, however they suggested reducing the frequency of extra tests to save important teaching time which otherwise influence on the syllabus coverage pace of a teacher. Also, they called school a “pressure cooker” (AKUEB_HA: FGD, dated, 22-2-2016) where they have been asked to compromise all other activities for the sake of getting good marks.

A student said, “Teachers took our sports, information technology and library periods for extra teaching and testing. We were asked to study only. Our brains literally had shut down” (AKUEB_MA: FGD, dated, 1-3-2016). They critiqued the markoriented approach of school and said, The basic purpose of coming to this college was to experience college life, to enjoy this phase. If study were the only objective, I would have enrolled in some government college and would have taken private tuitions. (AKUEB_HA: FGD, dated, 22-2-2016) Further, students perceive that the scores are used to categorize students and penalize specially for those who cannot perform well. A mid-level ability student shared his experience of first day in the HSSC part II where teachers asked the A-one and A graders to stand up. He felt embarrassment while sitting and comparing himself with his friends. Such disparity and biased treatment lead to low self-concept, fear of failure, depression and element of cheating in the students of low ability group. Quotes mentioned below show the same.

When I scored less in my biology test, the teacher called my parents and complained… Now my parents do not trust me. My friends said why I had given my parent’s cell number. But I have learnt something for university that I will not give my father’s number. I know I have learnt a bad thing, but they taught us so.

(AKUEB_LA: FGD, dated, 23-2-2016)

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

95

College pressurizes so much that you are stressed out. Nothing is being managed.

In the first year, I felt a lot of competition that created inferiority complex in me.…

I was confused whether I would pass my exams or not. (AKUEB_MA: FGD, dated, 1-3-2016) While agreeing to the above finding, a teacher regarded it as an expectation from them by the school management. She said, We want to do a lot for the students like counseling, to raise their capacity and build their personality. But there is no such weightage in the appraisal that judges you only on students’ grades. (Biology Teacher: dated 16-3-2016) Student’s comment, “It is a race in the college.… Here we have all kinds of stress.

Only those can survive who are ignorant” (AKUEB_MA: FGD, dated, 1-3-2016). This reflects the students’ perception about the impact of school’s academic policies and practices on the overall school culture.

4.3.3.5 Examination systems and standards. Students pointed out the difference in the standards of two examination systems, AKU EB and BIEK which leads to different attitude, study pattern and expectations of students affiliated to these two boards.

Following comments expressed the same.

Coming to this school was entirely different. For me, it was setback because I thought here teachers will have their own notes. But here, they teach us the concepts only. They do not provide notes. They neither teach us how to write an answer nor how to make headings. We have to refer to notes from coaching centers which many a times are unreliable due to which marks get deducted in exams. (BIEK_MA: FGD, dated, 7-3-2016) The system of AKUEB requires you to know each and every thing. Teachers cannot explain every point. Therefore we need to go for self-study. I, instead of focusing

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

96

on one book, refer to several resources available on net or in the library” (AKUEB_HA: FGD, dated, 22-2-2016) AKUEB students have shown their confidence on the fair testing, reliable marking, paper standard and examination administration, however, expressed their concern on the lack of clarity in the scope of learning outcomes, unavailability of a single course book and unpredictability of questions in the exams. They shared that these factors put lots of pressure on them throughout their academic years. They admired the board’s focus on conceptual learning in comparison to BIEK which is considered to foster memorization in a fixed and familiar format. These strengths and challenges encouraged few students to shift their board either from BIEK to AKUEB in HSSC or vice versa. A student shared, One of my friends has studied her HSSC from AKUEB board and her percentage decreased a lot. She had to switch her college. Your future depends on these two years. Nobody wants to take any risk. (BIEK_HA: FGD, dated, 22-2-2016) This perception regarding AKUEB as less scoring examination system is also proved by the independent sample t test on HSSC I overall scores where significant difference (AKUEB, n1=116, M1=72.95, SD1=9.25; BIEK, n2=109, M2=77.26, SD2=9.04) was observed in the scores of AKUEB and BIEK students (see Appendix J).

On the other hand, Habiba’s story depicts that social image and conceptual learning of AKUEB motivated her to shift to AKUEB. Besides, it was found that shift to any board was perplexing for students due to different standard and paper pattern. A student who shifted to BIEK shared, In first test, I didn’t get good marks. Later, I realized that both patterns are completely different. You need to memorize and write in an elaborated form. You

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

97

cannot go for concepts unless you are able to complete paper in three hours.

(BIEK_HA: FGD, dated, 22-2-2016) However, the intensity of challenge is much greater for those who have shifted to AKUEB. Values of Coefficient Correlation reflect the significant correlation between SSC II and HSSC I scores of students who continued with the same board (AKUEB, r=0.78, n=86, p=0.00; BIEK, r=0.35, n=77, p=0.00) and those who shifted from BSEK to AKUEB (r=0.59, n=19, p=0.00) in HSSC (see Appendix I). However, no correlation (r=0.33, n=15, p=0.22 at 0.05 level of significance) exist in SSC II and HSSC I scores of those students who have shifted from AKUEB to BIEK in HSSC (see Appendix I).

An interesting finding was shown in the correlational analysis that the mean scores decreases from 79% in SSC II to 62% in HSSC I for those students who have shifted from BSEK to AKUEB in HSSC (see Appendix I) whereas mean scores increases from 79% in SSC II to 81% in HSSC I for those students who shifted from AKUEB to BIEK in HSSC (see Appendix I). This reflects the difficulty level of AKUEB examination systems which was also expressed by Habiba (pseudonym) how the unfamiliarity with the paper pattern and poor concepts created confusion, low self-image and poor performance in science.

Further, students highlighted the lack of alignment in the testing subjects in these boards which creates difference in the competency level of students from these two boards. In BSEK, students take chemistry and biology subjects in SSC I only, whereas, in AKUEB, students study these subjects for two years in SSC. This one year gap in the study of chemistry and biology results in blankness in BSEK students and hence they cannot show the academic rigor required by the AKUEB system in HSSC. Students expressed the need of proper guidance at the time of admission to take the informed decision. Also, they suggested to have a school based comprehensive

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

98

orientation-cum-support program at the beginning of the academic session to bridge the learning gap and get them introduced to the new system.

4.3.3.6 Students’ academic habits. Students’ learning and stress-coping strategies have been found most influential on their science performance in the competitive environment. It is reflected in the following quotes of the students, Here we have lots of pressure. We need to prepare for preliminary, final board, and university admission tests. It all depends on how a student deals with that pressure.

Many a times, students get mental breakdown if they are unable to deal with it. It is the way you cope with the pressure. (AKUEB_MA: FGD, dated, 1-3-2016) I am very pessimistic and highly sensitive. My nature is very recessive, whatever happens I take it on my nerves. Such things distract a lot… only those can survive who do not let outer negative energy influence them. (AKUEB_MA: FGD, dated, 1-3-2016) Also, high ability students considered the habit of self-study as a key to their success. Students said, I used to refer several books recommended by teachers. I used to make notes or write reference website or book’s page number in front of each specific learning outcome so that I can refer it back at the time of exams…. Self-study gives you satisfaction otherwise you can’t reach that level. You can’t excel. When I study, I feel proud of myself for accomplishing this task on my own. (AKUEB_HA: FGD, dated, 22-2-2016) The above quotes depict that self-study leads to self- appraisal, positive self-image and improved performance. However, a student commented, “We have lots of distractions. Half of our energy is wasted in it” (AKUEB_MA: FGD, dated, 1-3-2016).

This shows the number of factors effecting students’ study practices and routine. The first factor identified by students is poor health conditions. A female student shared, “I

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

99

have been suffering from major clinical depression for four years.… It was like you can’t help yourself. You can’t score better, you want to study, but you can’t” (AKUEB_MA: FGD, dated, 1-3-2016).

Another factor highlighted was additional responsibility taken by students in order to overcome the family’s emerging needs. Following comments reflect how students’ focus and priorities have been changed due to the unpredicted situations which ultimately influence their performance.

In board exams, my mother got hospitalized, she had a sudden heart attack. At night, I used to be in the hospital, taking care of her and studying at the same time. And in the morning I took papers. I had to be in the hospital as my mother wanted someone very close to be with her. (BIEK_MA, dated, 7-3-2016) I am engaged and my in-laws are settled in USA. On my last paper in HSSC I, my in-laws were coming. In the morning, I received a call from the airport that their luggage had been stolen. They asked me to call in their bank in USA to cease any financial activity taking place from their bank accounts…. I was completely blank while attempting my physics paper. It was a hopeless paper. I got D in the paper, but later, received an A. (AKUEB_LA: FGD, dated, 23-2-2016) My father doesn’t support us and my mother is handicapped …. Now as I am in college and taking tuitions, I feel I should earn so at least I can meet my expenses and do not burden my mother with these expenses. While doing this, sometimes I compromise my earning and sometimes my studies. (BIEK_LA: FGD, dated, 23-22016) These uncertainties of life lead to make students’ life chaotic which they find difficult to manage along with their academic responsibilities. Another factor was related to environment and technology. A student shared her experience of shifting from

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

100

hostel to home after getting admission in college. She compared the freedom at home with the disciplined and controlled life of hostel. Also living a life at home where she had faced minimum interference of outside world with the sudden exposure to real life challenges like commuting in the public bus to reach to the school resulted into stress and distraction. Further, students found social media as the most distracting factor for them to which they themselves feel helpless. A student shared, This is important for us because we have been exposed to this since childhood.

We have become so addicted if we give our mobile to them (parents). We can’t survive. We crave it. Cell phone is called a cell phone because it is prison for a person. You can’t live without it. Parents don’t realize it. (AKUEB_MA: FGD, dated, 1-3-2016) This indicates that students tend to struggle in managing their social and academic lives simultaneously due to the high influence of the former.

The findings show the psycho-social influence of students’ academic, environmental, and personal factors leading to their self-efficacy and performance in science subjects. The high-sense of academic self-efficacy motivates these students to put efforts to accomplish academic tasks and persevere when faced with difficulty. On the other hand, students with low academic self-efficacy put in less efforts and give up quickly when encounter problems and difficulties, the resultant effect is poor academic achievement.

4.4 Conclusion This chapter highlighted the results obtained from the quantitative and qualitative part of this study. Quantitative data approved all the hypotheses of the study and presents a highly significant correlation between CET and students’ performance in HSSC exams. On the other hand, the qualitative part explored students’ perceptions

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

101

related to the role of CET and its model and the academic and non-academic variables influencing their performance in science subjects. The significance of these two sets of variables is unquestionable for understanding the human beings, their overt and covert behaviors, potentialities, and performances in science subjects.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

102

CHAPTER V DISCUSSION 5.1 Introduction Any criteria used for the college admissions have critical role on the admission decision which ultimately impacts the students’ future. It is therefore important to validate those criteria for justifying their use. There are many types of validity evidence; predictive validity is one of them. Predictive validity means an extent to which the specific criteria predict students’ future academic success. The study aimed to explore the predictive validity of CET used by AGAZ Higher Secondary School to induct the students for two-year HSSC program. Moreover, considering the fact that psychosocial and personal factors cannot be ignored while investigating students’ first year experience in the college, this study explored through students’ voice and the post entry factors influencing their performance in first year.

In this chapter, the results of the study are discussed and placed into the context with previous research. It presents the purpose of the study; as well as summarizes the major findings. Thereafter each finding and its implications are discussed in detail, demonstrating as how academic and non-academic factors can impact the students’ performance in annual board examinations. Further, based on the results of the present study and previous discourse in the literature, recommendations are made to policy makers, school management, parents, and teachers for the change in policy and practices for better Grade 11 students’ outcome. Lastly contribution of present study is shared along with the future research directions.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

103

5.2 Findings According to the conceptual framework presented in chapter two, the study focused on relationship between Grade 11 students’ performance in CET and annual board exams.

Four hypotheses were formulated to investigate the usefulness of three admission criteria: SSC I result, CBT scores, and interview scores in predicting academic performance of Grade 11 pre-medical students in AGAZ Higher Secondary School. Further, the influence of academic and non-academic factors mentioned in the conceptual framework of the study was explored through a phenomenological approach. These factors were derived from the Walberg’s (1984) theory on causal influences on students’ learning. In this study, students’ reflections of their experiences during their stay in the school were captured in order to see the impact on their annual board performance. However, the same can be explored on students’ performance in CET which was not captured during this study and therefore suggested for future studies. In the following section, the findings of each hypothesis are discussed with the possible explanations for them.

Hypothesis 1: There was a highly significant correlation between the subject wise and cumulative science scores of CBT with the HSSC I chemistry (r=0.37, p=0.00), physics (r=0.50, p=0.00), biology (r=0.35, p=0.00), and aggregate science scores (r=0.57, p=0.00) at 0.05 level of significance. These findings prevail that the collegebased admission test has the potential to predict students’ science performance in the first year final board exams. The findings have been found consistent in the research studies from various context, disciplines, and academic levels (Ali & Ali, 2013; Ashenafi, 2014; Stewart, Lim, & Kim, 2015; Ting, 2001).

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

104

The strong correlation between CBT and HSSC I science scores can be linked with the key considerations proposed by Helms (2008) to develop the right system of admission. The first characteristic reflected was the alignment in the content of CBT with that of SSC curriculum. Though the content and construct validity of CBT was out of the study scope, school’s admission prospectus endorsed this conclusion. This alignment with SSC curriculum shows the validity of CBT as a tool to check students’ prior science concepts learnt in the school. This also makes entrance test a fair tool to check students’ prior knowledge irrespective of which national examination board student is coming from.

The independent sample t test triangulates the above finding as no significant difference is found in the CBT science performance of both groups: AKUEB (n=101, M=55.61, SD=10.44) and BSEK (n=96, M=54.28, SD=13.60) (see Appendix F). The significant relation between CBT and HSSC I science scores (Hypothesis 1) infers that the fundamental science concepts are important to students’ learning of higher concepts in science. The gap in understanding of the basic concept of science would create a vacuum which results in lack of concept building at higher level of studies.

Also, if entrance tests at college level were prepared on the previous science concepts learnt at secondary level, they would have great potential to predict students’ science performance at HSSC level.

These implications suggest the role of primary schools in students’ future academic progress in science and hence stress the promotion of conceptual learning rather than rote memorization in school-based science classroom. GoP (2002) endorsed this recommendation that education should be viewed as continuum. Higher education systems cannot operate in isolation and quality of education in schools will greatly affect the preparation of students for higher education. Likewise, quality of

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

105

higher education pressurizes schools to improve their own quality in making students prepared for the future studies.

The second characteristic is with respect to the format and structure of CBT. The strong correlation between CBT and HSSC I science scores support the idea of using MCQs in the test for admission purpose. Zwick (2007) accepted that the MCQs format is found more useful as compared to the essay writing due to reliable and convenient marking. In addition, students appreciated during focus group discussions that MCQs based test checks students’ conceptual understanding rather than language and time management skills. This indicates the reliability and fairness aspects of college admission tests at AGAZ Higher Secondary School as it uses a format conducive irrespective of difference in the examination systems as well as language proficiency.

Contrary to the above facts, BIEK students perceived that MCQs format has little validity to judge students’ capacity to perform well in the annual exams as they both have different parameters for evaluation. Nevertheless, strong correlation between CBT and HSSC I science scores of BIEK, r=0.64 (two tailed), n=109, p=0.00 at 0.05 level of significance (see Appendix G) contradicts students’ perceptions. CBT has been found as significant predictor for BIEK students’ performance in annual exams regardless of different criteria of evaluation. Zwick (2007), however, agreed that admission test should include writing task which is more cognitively demanding, holds great potential to check ability, and has a key role in college study.

Furthermore, identifying the writing need of students at the time of admission could enable teachers to provide need-based support throughout the academic session.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

106

Whereas MCQs provide very little information to teacher as how the student arrived at the conclusion or was it a lucky guess. It might happen that student has an advanced thinking pattern despite an incorrect answer which could be supported and explained further by teacher. This implies that the restructuring in the format of CBT could lead to its formative purpose of improving studentsâ&#x20AC;&#x2122; ability in science subjects.

Also, use of technology for admission purpose is appreciated by students due to its innovation and speed. However, technology integration could enable test maker to increase the test validity through graphical analysis on responses. Above discussion shows that change in CBT format would enable AGAZ Higher Secondary School in making more constructive and formative use of its CBT instead of limiting it as a screening tool for admission.

In addition to the above discussion related to the focus, functions, and methods of CBT, students shared two major factors influencing their performance in science.

First, the role of coaching classes in their achievement in CBT as well as HSSC I science as it provided directions and assistance on the pattern and focus of these tests.

This reinforces major critique by Zwick (2007) on the validity of testing. He argued that inclusion of coaching conceals the actual capacity of students. Better results might be produced due to the learned behavior, familiarization of testing phenomena, and multiple times practice. Further, it marginalizes students who cannot afford coaching due to poor SES. These students remain deprived of good results as they do not have the funds for getting additional support from the coaching. Though the impact of coaching or preparatory classes on studentsâ&#x20AC;&#x2122; learning outcomes has not been established by this study, this new dimension can be studied through comparative correlational study to

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

107

see how the performance in these tests differ for those students who take took coaching and those who do not .

Second factor highlighted by students coming from BSEK is the one-year gap for learning chemistry and biology. Students perceived CBT for chemistry and biology a real challenge and source of anxiety for them as they have not studied these subjects in SSC II in contrast to AKUEB students who studied these subjects in both SSC I and II. Though independent sample t test on biology and chemistry CBT for both AKU EB and BIEK falsifies this perception, the psychological aspect connected with this misperception is much severe. Time lapse and discontinuity in science learning produces anxiety among learners leading to disinterest and discomfort for the subjects. This implies the need of reform in the examination systems for the purpose of uniformity and steadiness in learning science concepts.

Hypotheses 2 & 3: According to the results, there is a significant correlation between overall CET scores and students’ performance in science both for BIEK (r=0.70, n=94, p=0.00) and AKU EB (r=0.51, n=103, p=0.00) students (see Tables 4.10 & 4.12). This implies that CET model used at AGAZ Higher Secondary School as a good predictor of students’ science performance in annual board examinations irrespective of the examination system.

The findings show the alignment in the standards set for CET and that of higher secondary level of science studies is irrespective of the format and approach of examination systems to which students are affiliated. Inclusion of previous Grades in addition to the college admission test is supported by Ali & Ali (2013) as it provides a range of evidence of students’ prior knowledge as well as capacity to perform in the science subjects. On the other hand, CBT provides another chance to the students to

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

108

exhibit the ability if they could not perform in the previous exams for any reason. GoP (2002) also supported the use of admission tests due to the inadequacy of SSC results in measuring students’ competence. The rote learning aspect and dishonest practices such as using unfair means and corruption associated with SSC examination systems diluted the credibility of public examination in determining their potential for higher studies. This shows that SSC I and CBT scores together provide a good sense of students’ cognitive achievement in science.

Further, interview tool used in CET process covers multi-dimensions of individual’s personality including their academic and co-curricular achievements, interests and hobby, communication and problem solving skills, and so on. These aspects play a significant role in building students’ self-concept which indirectly influences students’ performance in Grade 11 science subject. Teachers in the study realized the importance of a balanced personality on one’s academic progress in science as it requires students to be self-disciplined, pro-active and progressive for learning. This finding is supported by Mukhwana (2013) who reported the positive impact of study habits, punctuality, and discipline on students’ academic performance. The combination of SSC I, CBT, and interview scores make CET an appropriate model in determining students’ capacity of learning of science irrespective of the examination systems to which student is affiliated.

This suitability of CET model for both examination systems was equally acknowledged by students as they called CET a comprehensive way to judge students’ actual capacity which was not clearly indicated by SSC I results due to the assessment and administrative differences in both systems. Zhu (2014) also suggested using multiple admission standards for predicting students’ future performance by viewing several pre-entry characteristics.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

109

Besides the filtering role of CET model, teachers in the study suggested the use of CET data for identifying students at risk both academically and socioeconomically. This formative use of CET model could help school in providing best situated facilitation as per individual’s need before the conditions get worst. Students on the other hand, viewed CET very important in developing enthusiasm among them as science students to compete and to get admission in the institution of repute.

This is aligned with the findings of Ruoling’s (2010) study where admission process stimulated students’ interest and is considered as the symbol of institution’s quality. Finding of this study showed that the college entrance test process of AGAZ Higher Secondary School has social significance attached with it as students are bound with social pressures and expectations to succeed in order to prove their ability for science-related career.

Above discussion signifies multiple roles CET model in facilitating science students’ interest and performance for future studies. However, students perceived no predictability of CET for their annual board performance in science subjects as they considered one year school-based experience as the real determinant of their final board’s performance. This finding shows that performance maintenance cycle through school policies and practices have major influence on the sustenance of science students’ capabilities. This was best reflected in the academic support programs provided by teachers for the low ability students, special classes for the high achievers, testing and retesting processes in order to raise their performance bar in science subjects.

The high value of correlation coefficient (r=0.84, N=225, p=0.00) between students’ performance in school’s internal exams and board exams (see Appendix G) shows the alignment in the pattern and standard of both exams. Planchard, et al.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

110

(2015) study supported the finding as they noticed significant role of school in developing study skills and learning motivation among students through aligning task with the exams marks.

Moreover, stories of two outliers and the focus group discussions with students reflect the psycho-social dynamics of testing prevailing in the school. The culture of competition for best grades in AGAZ Higher Secondary School undermined feeling of connectedness, as many science students viewed school as pressure cooker where their whole life is sucked out. Further most of the students in focus groups associated the same pressure with their future studies in medicine and hence perceived it inappropriate career field for them due to its challenging nature.

This is supported by the study of Sami et al. (2016) who found the negative interaction between the student and the institutional context can reduce studentsâ&#x20AC;&#x2122; sense of belonging, resulted in alienation, frustration, and hopelessness. Particularly, students with low self-esteem and ability found it challenging to survive in the competitive environment as they feel themselves isolated and marginalized. Students shared as how they experienced lower academic self-worth after having several retests with no improvement at the end. They felt themselves less prepared and empowered for academic tasks, heightening their levels of test anxiety. This perceived academic overload could be due to their lack of adequate academic preparation and poor secondary school education.

The study of Meijer (2001) affirms this finding that low self- confidence results in high text anxiety. Moreover, students from low, mid-level, and high ability groups equally shared their concerns on the large number of formal and informal tests in science classrooms which resulted in their elevated stress level. This was consistent to the findings of phenomenological study carried out by Nordmo and Samara (2009)

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

111

who critiqued the survival efforts of high achievers in the completive environment at the cost of their interest and joy for subjects. This experience of higher levels of stress at college may be caused by the fear of failure, study on continuous basis to prepare for test, the perception of tests being extremely difficult, self-evaluation of being illprepared, fear of teachers’ criticism and taunting behavior, or the pressure and expectations of others for example family members to do well at college.

Story of Habiba in Chapter 4 depicts that first-year students mostly underestimate the amount of time required for independent learning, course preparation and overestimate their academic abilities and skills. As a result, they may be feeling overloaded by the constant academic tasks and responsibilities and ultimately show lower performance in science subjects. The above findings imply that the burden on students due to these exams mainly stems from the culture of competition. Students felt themselves pressurized by the high expectations and aspirations for the best results which are considered as the symbol of school prestige and key for their best future. There is also a general pressure of life including family’s economic pressure, peer pressure, media and social pressure which disturb science students’ focus on the studies. This suggests the role of schools to facilitate students in a severe competitive learning environment through motivational and supportive role of teachers, balanced academic and non-academic activities, counseling services, and flexible policies and programs.

Hypothesis 4: Results show the significant linear relationship between all three components of CET model with HSSC I overall scores (R2=0.43, p=0.00, F=50.07). Findings indicate that the CET model predicts students’ overall performance in HSSC I and could explain 43% variance in the data. This implies the need of adding more

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

112

variables that could further improve the model and could describe 57% of the unexplained variance in the data.

Also, beta value of interview is found 0.06 which is far less than that of CBT (.50) and SSC I (0.26) (see Tables 4.15). This shows that interview is considered significant with a comparatively less predictable power as compared to SSC I and CBT. The analysis of interview form used in the admission process reflects that interview tool lacks scoring rubric and therefore inter-rater reliability on marking scores is a big question on the validity of tool used for the purpose. Further, brief interview interval and formal questions lead to fabricated answers and hence does not capture the true picture of students’ personal, psychological, and social characteristics. The same is indicated by students who uttered how they masked their natural ability and potential and presented a fabricated picture before the panel of interviewers due to pre-defined interview questions.

The findings imply the inclusion of tangible personal and psychological variables in the CET model to increase the predictability of students’ annual results.

Liu (2013) agreed that a combination of cognitive and non-cognitive factors, such as school admission test scores and psychological variables, better predict Grade 11 grades than cognitive factors alone. Whereas in AGAZ Higher Secondary School, CBT and SSC I results comprise 80% of the admission criteria. As both of these components rely purely on content, CET model was found more inclined towards achievement rather ability (Zwick, 2007). Thus limited scope of CET model does not provide the judgment on students’ aptitude and attitude towards learning which are highly needed once students enter in their higher education institutions.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

113

Hence, CET model predicts only students’ annual performance rather than the accomplishment in future career. This however, can be explored through a correlational study on students’ CET scores and actual life success. Further, the content focused admission policy favours students who have better resourced schools, more access to private tutoring, access to specialized test preparatory programs, and greater access to books at home. This questions the fairness of admission policy as it marginalized students from low socio-economic background who were unable to get quality education, resources, and support for learning.

Despite the R2 value (0.43) which shows the predictability potential of CET model, the large values of standard deviation in HSSC I overall scores for BIEK (SD=9.04) and AKUEB (SD=9.25) in Appendix J reflect certain academic and nonacademic factors influencing students’ performance. This is reflected in the stories of two outliers reported in chapter 4. The personal, social, economic, and academic factors in one’s life influence his or her performance maintenance cycle during the Grade 11 college studies. These factors have significant impact on students’ selfefficacy, and perceived self-concept which ultimately affect their annual performance.

This conclusion was in line with the meta-analysis carried out by Robins et al.

(2004) who reported that academic goals, self-efficacy, social support, and study skills are the strong predictors of students’ retention and performance in the college.

In the present study, student with high academic self-efficacy expressed as how they mastered the academic as well as social skills and behavior necessary to accomplish task. A positive self-concept make it easier for high ability students to participate in class discussions, to participate in co-curricular activities, to ask clarifying questions about academic matters, and to interact with peers and teachers on campus; thereby making it less challenging to adjust to the college life. Major contributing factors

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

114

came up during focus group discussions were students’ previous schooling, parents’ qualification, family’s socio-economic status, teachers’ attitude and practices, examination system and unforeseen circumstances impacting students’ perceived selfefficacy, and study habits (Onabamiro et al., 2013).

First factor was students’ previous schooling which provided an array of skills and concepts necessary for their higher studies. It is found that low profile school with fewer learning resources and traditional teaching method led to poor concept building.

This reduced the students’ self-esteem and confidence when they came across with students from better schooling. Also, the learning practices with which students were habitual in their primary education made learning shift at higher secondary level more challenging. This shows the importance of former years in one’s life in developing the learning habits and aptitude.

Finding reflects that students’ perceived competency is aligned with the image of their schools in the society. It is also a symbol of status and prestige for students when coming to the context with diverse population. The association with reputable schools helps students in creating a good place among peers and teachers as well as in getting social acceptance and presumed competence.

Second factor was socio-economic factor which does not only influence on the selection of school and career, but also played vital role in students’ focus and attention towards studies. Being mature young adults, students at higher secondary level are highly sensitive to their family profile and circumstances which ultimately influence students emotionally and academically. Focus group discussion endorsed these findings that students from low SES background found themselves distracted due to financial constraints no matter whether they contribute financially to the family income or not.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

115

Min (2013) found that social and technological capital of family has the largest influence on students’ performance as they mostly communicate with their parents and use electronic tools for the study. This study found that lack of learning resources, poor quality of food and home environment result into low self-esteem, hopelessness, and anxiety among students. Further, students who struggle for their financial stability through income generation process lose academic balance in their economic endeavors. Also, high cost of professional colleges discourages the talented students from poor SES background to opt for the science-related career. This shows that SES has major influence on students’ emotional, academic, and psychological stability.

Third factor was parents’ qualification which was not perceived as a significant factor for students’ learning motivation although highly significant with reference to inspiration, guidance, and supervision of students. Students however emphasized that parents have significant place in their lives and their attitude greatly influence their academic performance. Parents’ negative attitude, criticism, and comparison among siblings resulted into the low self-esteem and performance of students. On the other hand, parents’ encouragement and involvement did not only support students emotionally, but also play key role in their self-concept and confidence.

This finding is consistent to the Jabor et al. (2011) study who found that “Either Parent College Graduate” students had higher GPA scores on science than the scores of “Neither Parent College Graduate” students. In addition to being actively involved in their children’s education, parents also provided a home environment that can affect learning. Parents served as a model for learning, determined the educational

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

116

resources available in the home and held particular attitudes and values towards education.

Fourth aspect highlighted in the finding is teachers’ attitude which play important role in enhancing students’ self-concept of ability. Findings reflect that most of the time teachers’ judgment on student’ ability was biased and without credible evidence. The limited expectations of teachers and their assumed behavior and performance tend students to believe and display the same. Singh and Baksh (1977) suggested that teachers should have a complete faith and high expectations from each student regardless of students’ potential.

This would result in self-fulfilling prophecy among low achievers and help them in meeting teachers’ expectations. Students’ remarks in the focus group discussion endorsed that the marks-oriented approach and harsh attitude of teachers resulted in undue pressure on mid-level and low ability students as they felt themselves marginalized and victimized. On the other hand, teachers’ facilitation and monitoring mechanism in the form of academic support program and internal school examinations are admired by students. These practices for students’ sustenance help them in evaluating their own learning and preparation level for exams.

Fifth aspect is related to school’s resource, culture, and policies. Findings suggest that adequate resources, support services, and developmental programs organized by school support and guide all students with their academic tasks.

However, the strict policies and competitive culture produced negative impact on students’ motivation and teachers’ practices. Teachers were being forced to teach to test. Teachers’ rating, job security, and performance evaluation are done on the basis of students’ results in high-stakes examinations. Hence most of the teaching time was spent in testing and drilling method to ensure students’ capacity to perform well in

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

117

annual exams. Teachers focus on students’ scores rather than their holistic development which is their core function as a teacher. Also, high expectations with the high ability students by giving more attention and providing extra practice to ensure their position in the board exams excluded those who are the mid-level achievers. These biased and unfair practices developed frustration among the students and they perceived school as a preparatory agency rather than a learning platform for them.

Sixth aspect which played a major role on students’ performance is the examination systems. This study found that students’ learning attitude and practices are highly aligned with the examination systems. The system that focuses on conceptual learning motivates its students to acquire independent and extended learning skills. Students as a result take ownership of their own learning and develop self-concept. On the other hand, the system that favours forecasting exam question through guess papers makes students dependent on the school to make them prepare.

They limit their potential and avoid additional practice of studying the syllabus which is neither required nor appreciated in the annual board examinations.

This difference in the attitudes and learning approach of students from two examination systems indicates the gap in the alignment of policies at national level.

This gap has serious implications for those who shift the examination system at higher secondary level. Many students in the study found that they struggled due to the difference in the examination pattern and learning approach. This indicates that dual systems and standards of examinations create confusion and anxiety. This consequently results in decline of students’ performance particularly for students who shift to more challenging system and find it difficult to cope with the demands of new board.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

118

This implies that any sudden change at higher secondary level is neither suitable nor required for sustaining studentsâ&#x20AC;&#x2122; performance. One reason could be the social stigma associated with different examination systems. Students perceive new system as an unattainable target and struggle for the survival rather than competence.

Such students give up soon and accept their failure without much hope for the future.

This sometimes results in the career shift as their poor self-concept decline their trust in their potential to compete for better future.

5.3 Implications and Recommendations The findings of this study have important implications for policy makers, school leadership, college teachers, counselors, parents, and students. Firstly, it is advisable for school leaders and teachers to focus not only on cognitive factors, but also on the non-cognitive aspects such as self-efficacy and academic achievement motivation in educational programs. Teachers working in high school may work with students to increase their self-efficacy, develop the stress coping skills and improve the academic achievement motivation.

This shift of school culture from marks-oriented approach to learning-oriented attitude could benefit students of different ability groups equally to a greater extent.

Providing special support services focusing on enhancing realistic self-appraisal of low ability students may increase their academic success, especially during the first year of studies.

Secondly, the role of CET is perceived and applied in a very narrow terms and limited to offer admissions only. Formative use of CET data could be used to device educational plans and strategies specifically for the students at risk. Also, the admission policy standards and criteria should be made clear to major stakeholders like parents and students. Likewise, pre-entry counseling service to all potential

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

119

applicants could enable students to take informed decisions related to board and discipline while taking admissions.

Thirdly, significant predictive validity of CET model implies the importance of personal, social, and cognitive factors in forecasting the students’ performance.

However, each aspect should be given equal importance in terms of weightage in the formula-based admission as non-cognitive factors are found to have great influence on students’ persistence, perceptions, and practices. This would even provide a better prediction of a student's ability to persist and achieve beyond Grade 11.

Fourthly, policy makers in different public examination systems should critically review the policies and practices as these have high stakes on students’ performance as well as future success. It is recommended to device a uniform standard based on the best practices of each board in order to avoid any discrepancy in terms of policies. The reduced gap in the quality of two systems would enable all students to compete at fair grounds without getting influenced by their perceived quality due to affiliated examination system.

Fifthly, school should provide a post-placement support based on the care model rather than the conventional model that functions only to complete cycles of examinations year after year. There has been increased need to provide curricular and co-curricular support for students irrespective of their academic abilities. Access to learning resources and opportunities, productive involvement of parents and peers, and formulation of student-friendly policies could build a sense of attachment with the school and significant others leading to better performance of students.

Finally, a centralized body called Sindh Private Colleges Association should be formed to develop, execute and monitor the standardized policy framework for the

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

120

admissions in private colleges. Such network would facilitate the collaborative efforts for system learning leading to the validated admission criteria, tool and mechanism across the region. Further, the consolidated admission policy framework would reduce the policy gap and ensure the alignment in different private institutions. Hence, it will increase the clarity, transparency and appreciative attitude among students and parents seeking for the admissions. The other advantage of forming this centralized body will be to seek present differences in assessment strategies across colleges in the province of Sindh and to formulate a policy draft for assessing the written performance of all the students in annual board examinations. This is likely to reduce the probability of errors in assessing the students’ responses.

5.4 Contribution of the Present Findings Most of the studies present their findings related to predictive validity of admission tests for students’ final scores, however, lack on how and why social and academic factors integrate with each other in order to facilitate better outcome or vice versa. Further, to the best of the researcher’s knowledge, none of the studies in Pakistan has explored and reported on students’ personal experiences at higher secondary level influencing their board’s performance. This study therefore has bridged that gap in the literature and provided a baseline for the more research studies in this area. The present research study has established concise contributions to the current knowledge and understanding of how CET model predicts the students’ performance after one year of studies.

Also, it recognized certain academic and non-academic constructs influence students’ adjustment and academic performance competencies at higher secondary level. Therefore, these findings are not only relevant to the policy makers, as well as important for both institution and students alike for their future academic success.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

121

5.5 Future Research Directions As this study was limited to Grade 11 pre-medical students of one private higher secondary school of Pakistan, it is suggested to carry out additional research at other local institutions and among other student groups like Pre-engineering and commerce. The findings from further studies would help to establish if the results obtained in this study were comparable and generalizable to students in other contexts. Also, the academic and non-academic factors identified in the study have been proposed to integrate in the CET model of AGAZ higher secondary school to increase the predictive validity. However, more research on the suggested extended CET model is needed before making this theoretical model as a part of the admission policy. Further research studies to explore the content and construct validity can be done in the same context.

Also, this study can be stretched further to see the difference of predictive validity of CET model with respect to gender, socio-economic, and ability group. This will provide deeper insight on the appropriateness of CET as a predictable model of performance for students from diverse backgrounds. Furthermore, the same study can be stretched further to see the correlation of CET and Grade 11 science studentsâ&#x20AC;&#x2122; annual performance with their HSSC part II performance to check the academic persistence. Likewise, interventions suggested in the recommendation section can be implemented and studied through correlational research design before and after intervention to see if any significant changes occur in studentsâ&#x20AC;&#x2122; performance due to facilitation.

5.6 Conclusion The purposes of this study were to know the predictive power of secondary school exams, computer based test and interview scores in predicting academic

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

122

performance of pre-medical students in first year. The study found significant correlation between CET variables and HSSC I science and overall result. Furthermore, the study explored different factors both inside and outside school influencing this relationship. Overall, the results of this study were in the directions expected and were consistent with previous similar works.

Based on the present study, one could conclude that non-academic factors cannot be ignored in exploring students’ experiences in the college. These academic and non-academic factors contribute in students’ perceptions. In addition, pedagogic practices ultimately affect Grade 11 students’ performance. Hence, these factors could be the part of CET model to predict students’ college experience and performance particularly in the annual board examinations. The inclusion of these factors should not only be limited for the college admission, but should be considered while planning any academic and non-academic school policies. In most of the high schools, special attention is given to the academics only in order to secure good marks. This study suggested for the holistic development approach for a student in the schools to provide them with skills, self- confidence, and motivation for next phase of learning.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

123

REFERENCES

Ali, A., & Ali, Z. (2013). Admission policy of medical colleges: Evaluating validity of admission test in Khyber Pakhtunkhwa, Pakistan. Journal of Research and Reflections in Education, 7(1), 77-88.

Areepattamannil, S., & Kaur, B. (2013). Factors predicting science achievement of immigrant and non-immigrant students: A multilevel analysis. International Journal of Science and Mathematics Education, 11, 1183-1207.

Ashenafi, G. (2014). Prediction of first year studentsâ&#x20AC;&#x2122; academic performance in the School of Medicine of Addis Ababa University (Masterâ&#x20AC;&#x2122;s thesis). Retrieved from http://etd.aau.edu.et/handle/123456789/6994?mode=full

Askari, S. M. (2015, September 3). The failure of A One and A grade students in the admission test of NED (translation from Urdu). Daily Jang, p. 14.

Baig, L. A., Ghouri, G.H., Fehmida, T., & Nargis, A. (2001). Is the admission test at Institute of Business Administration a good predictor of final professional test grades. Pakistan Journal of Medical Sciences, 17(3), 160-177.

Bellibas, M. (2016). Who are the most disadvantaged? Factors associated with the achievement of students with low socio-economic backgrounds. Educational Sciences: Theory & Practice, 16, 691-710.

Blake, J. E. (2012). High-stakes testing: A (mis)construed, normalizing gaze.

International Journal of Educational Policies, 6(1), 5-23.

British Educational Research Association. (2011). Ethical guidelines for educational research. Retrieved from http://www.bbk.ac.uk/sshp/research/sshp-ethicscommittee-and-procedures/BERA-Ethical-Guidelines-2011.pdf Brosio, R. (2003, June). High-stakes tests: Reasons to strive for better Marx. Paper prepared from the presentation given at Rouge Forum Summer Institute on Education and Society, University of Louisville. Retrieved from http://www.jceps.com/wp-content/uploads/PDFs/01-2-01.pdf

Cameron, R. (2009). A sequential mixed model research design: Design, analytical and display issues. International Journal of Multiple Research Approaches, 3(2), 140-152.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

124

Choi, H. J., & Park, J. H. (2013). Historical analysis of the policy on the college entrance system in South Korea. International Education Studies, 6(11), 106121.

Cohen, L., Manion, L., & Morrison, K. (Ed.). (2007). Research methods in education.

London: Routledge.

Creswell, J. W., & Plano Clark, V. I. (2nd Ed.). (2011). Designing and conducting mixed methods research. USA: SAGE Publications.

Dai, M., Chen, Z., & Davey, G. (2007, July). Mental health implications of the Chinese University entrance exam. Proceedings of Conference on Psychology in Mental Health: Prospects and Challenges, Bangalore.

Deil-Amen, R., & Tevis, T. L. (2010). Circumscribed agency: The relevance of Standardized college entrance exams for low SES high school students. The Review of Higher Education, 33(2), 141-175.

Farooq, M. S., Chaudhry, A. H., Shafiq, M., & Berhani, G. (2011). Factors affecting studentsâ&#x20AC;&#x2122; quality of academic performance: A case of secondary school level.

Journal of Quality and Technology Management, 7(2), 1-14.

Gandara, F., & Randall, J. (2015). Investigating the relationship between school level accountability practices and science achievement. Education Policy Analysis Archives, 23(112). Retrieved from http://epaa.asu.edu/ojs/article/viewFile/2013/1691

Gecas, V., & Schwalbe, M. L. (1983). Beyond the looking-glass self: Social structure and efficacy-based self-esteem. Social Psychology Quarterly, 46(2), 77-88.

Glesne, C. (Ed.). (1999). Becoming qualitative researcher: An introduction. USA: Longman.

Government of Pakistan (GoP). (2002). Report of the taskforce on the improvement of higher education in Pakistan. Retrieved from http://www.hec.gov.pk/MediaPublication/HECPublication/Documents/

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

125

Helms, R. M. (2008). Education working paper series (Number 15). University Admission Worldwide. The World Bank, USA.

Huang, H. (2015). Can students themselves narrow the socioeconomic-status-based achievement gap through their own persistence and learning time? Education Policy Analysis Archives, 23(108), http://dx.doi.org/10.14507/epaa.v23.1977

Huda, N, Dosa,T. B., Alam, E., Agha, S. (2001). Selection procedure as predictor of performance in university examination. Journal of Pakistan Medical Association, 51, 381-84.

Institute of Business Administration. (2013). Statistical Analysis of SAT-II (2013) Results. Retrieved from http://www.satsindh.net.pk/

Jabor, M. K.,Machtmes, K., Kungu, K., Buntat, Y., & Nordin, M. S. (2011). Does parental educational status matter on students’ achievement in science?

International Proceedings of Economics Development and Research, 5, 309313.

Kellaghan, T. (2004). Public examination, national and international assessments, and educational policy. Unpublished manuscript, Educational Research Centre, St. Patrick’s College, Dublin. Retrieved from http://siteresources.worldbank.org/INTAFRREGTOPSEIA/Resources/paper_Kel laghan.pdf

Kirkpatrick, R., & Zang, Y. (2011). The negative influences of exam-oriented education on Chinese high school students: Backwash from classroom to child. Language Testing in Asia, 1(3), 36-45.

Lau, S. & Roeser, R. W. (2002). Cognitive abilities and motivation processes in high School students’ situational engagement and achievement in science.

Educational Assessment, 8(2), 139-162.

Lewin, K. & Xu, H. (1989). Rethinking revolution: reflections on China’s 1985 Educational reforms. Comparative Education, 25(1), 7–17.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

126

Liu, O. (2013). Three key issues in the reform programs for the Chinese college entrance examination. Chinese Education & Society, 46 (1), 41-50.

Madaus, G., & Russell, M. (2010/ 2011). Paradoxes of high-stakes testing. Journal of Education, 190(1/2), 21-30. Retrieved from http://www.bu.edu/journalofeducation/files/2011/06/BUJOE190.1n2MadausRussell1.pdf

McClelland, D. (1973). Testing for competence rather than for intelligence. Retrieved from http://www.servicelearning.msu.edu/upload/2.8.pdf (Reprinted from American Psychologist, 28(1), 1-14.

Meijer, J. (2001). Learning potential and anxious tendency: Test anxiety as a bias factor in educational testing, Anxiety, Stress, & Coping, 14(3), 337-362, doi: 10.1080/10615800108248361

Micceri, T. (2010). Assessing the usefulness of SAT and ACT tests in minority admission. Retrieved from ERIC database. (ED510111)

Min, Y. (2013). Association with parental socioeconomic status, social capital, technological capital and studentâ&#x20AC;&#x2122;s scores for college entrance in South Korea (Master dissertation, Graduate School of Vanderbilt University, Tennessee).

Retrieved from http://etd.library.vanderbilt.edu/available/etd-12192013152919/unrestricted/Min.pdf

Moustakas, C. (1994). Phenomenological research methods. London: Sage.

Mukhwana, W. J. (2013). The role of student-related factors in the performance of Biology subject in secondary schools in Eldoret Municipality, Kenya. Journal of Emerging Trends in Educational Research and Policy Studies (JETERAPS), 4(1), 64-73.

Nordmo, I. & Samara, A. (2009). The study experiences of the high achievers in a competitive academic environment: A cost of success? Issues in Educational Research, 19(3), 255-270.

Odom, A. L., & Bell, C. V. (2015). Associations of middle school student science achievement and attitudes about science with student-reported frequency of teacher lecture demonstrations and student-centered learning. International Journal of Environmental & Science Education, 10(1), 87-97.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

127

Onabamiro, A. A., Omoruyi, A. A., Soyingbe, A. A., & Rosiji, C. O. (2013). Psycho social factors predicting secondary school students academic self-efficacy in Lagos State, Nigeria. International Journal of Humanities and Social Science Invention, 2(4), 10-17.

Osborne, J., & Collins, S. (2001). Pupilsâ&#x20AC;&#x2122; views of the role and value of the science Curriculum: a focus-group study. International Journal of Science Education, 23(5), 441-467.

Oseguera, L. (2013). Importance of high school conditions for college access (research brief). Retrieved from https://pathways.gseis.ucla.edu/publications/201307_HSConditionsRB.pdf

Pakistan, Ministry of Education. (2009). National Education Policy. Islamabad: Government of Pakistan.

Pascarella, E. T., Sedlacek, & Terenzini, (1991). How College Affects Students. San Francisco: Jossey-Bas.

Phelps, P. (2006). Characteristics of an effective student testing system. Unpublished manuscript. Retrieved from http://nonpartisaneducation.org/Foundation/EffectiveTestingSystem.pdf

Planchard, M., Daniel, K. L., Maroo, J., Mishra, C., & McLean, T. (2015).

Homework, motivation and academic achievement in a college genetics course. Bioscene, 41(2), 11-18.

Ponce, O. A., & Maldonado, N. P. (2015). Mixed methods research in education: Capturing the complexity of the profession. International Journal of Educational Excellence, 1(1), 111-135.

Promotion of Education in Pakistan Foundation, (PEPF) Inc. (2006). Leaping forward: A report on higher education in Pakistan. Retrieved from http://pepfoundation.org/leaping_forward.pdf

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

128

Reitzes, D. C., & Mutran, E. (1980). Significant others and self-conceptions: Factors influencing educational expectations and academic performance. Sociology of Education, 53, 21-32.

Reporter (2015, August 30). Students, parents denounce entry test for taking admission in medical, dental colleges. Dunya News. Retrieved from http://dunyanews.tv/en/Pakistan/295990-Students-parents-denounce-entrytest-for-taking-a

Reseon, R.D., Terenzini, P.T., & Domingo, R. J. (2006). Developing academic competence in the first year of college. Research in Higher Education, 47 (2), 149-158.

Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004).

Do psychological and study skill factors predict college outcomes? A metaanalysis. Psychological Bulletin, 130(2), 261-288. doi: 10.1037/00332909.130.2.261

Rooney, C., & Schaeffer, B. (1998). Test scores do not equal merit: Enhancing equity &excellence in college admissions by deemphasizing SAT and ACT results (ED 426107). National Centre for Fair and Open Testing, Cambridge.

Retrieved from http://www.fairtest.org/sites/default/files/optrept.pdf

Ruoling, Z. (2010). On the rationality of the college entrance examination. Chinese Education & Society, 43(4), 11-21.

Sami, M., Erbilgin, E., & Arikan, S. (2016). Factors predicting Turkish and Korean studentsâ&#x20AC;&#x2122; Science and Mathematics achievement in TIMSS 2011. Eurasia Journal of Mathematics, Science & Technology Education, 12(7), 1711-1737 doi: 10.12973/eurasia.2016.1530a

Sackett, P. R., Schmitt, N., Ellingson, J, E., & Kabin, M. B. (2001). High-stakes testing in employment, credentialing, and higher education: Prospects in a postaffirmative-action world. American Psychologist, 56(4), 302-318. doi: 10.1037//0003.66X.56.4.302

Saleem, M. A., & Qureshi, M. I. (2011). Credentials and examination of the factors affecting the studentsâ&#x20AC;&#x2122; academic achievement in higher education. Gomal University Journal of Research, 27(2), 74-80.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

129

Schaefler, G. F. (1962). Wastage remedy: The use of an entrance test. The Vocational Aspect of Education, 14(28), 15-35. doi: 10.1080/03057876280000031

Singh. A., & Baksh, I. (Eds.). (1977). Society, culture and schooling: Issues and analysis.

Retrieved from http://www.mun.ca/educ/faculty/mwatch/Society,%20Culture%20&%20Schooli ng%20-%20Part%20I.pdf#page=75

Stewart, S., Lim, D. H., & Kim, J. (2015). Factors influencing college persistence for first-time students. Journal of Developmental Education. 38(3), 12-20.

Tai, R. H., Sadler, P. M., & Mintzes, J. J. (2006). Factors influencing college science success. Journal of College Science Teaching. Retrieved from http://svsd.schoolwires.net/cms/lib05/WA01919490/Centricity/Domain/457/re search_ans_teaching_article_fc.pdf

Thomas, P. L. (2013). Testing capitalism: Perpetuating privilege behind the masks of merit and objectivity. The International Education Journal: Comparative Perspectives, 12(2), 85-103.

Tierney, P., & Farmer, S. M. (2011). Creative self-efficacy development and creative performance over time. Journal of Applied Psychology, 96(2), 277-293.

Ting, S.R., (2001). Predicting academic success of first year engineering students from standardized test sore and psychological variables. International Journal of Engineering Education, 17, 75-80.

Walberg, H. J. (1984). Improving the productivity of Americaâ&#x20AC;&#x2122;s schools. Retrieved from http://www.ascd.org/ASCD/pdf/journals/ed_lead/el_198405_walberg.pdf

Wang, H. (2011). Research on the influence of college entrance examination policies on the fairness of higher education admissions opportunities in China. Chinese Education & Society, 43(6), 15-35.

Wang, Y., & Ross, H. (2010). Experiencing the change and continuity of the college entrance examination. Chinese Education & Society, 43(4), 75-93.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

130

Zhu, R. (2014). A comparative study of college entrance examination (CEES): SAT and ACT in the United States of Amercia and GAOKAO in People’s Republic in China (Unpublished master’s thesis, University of Akron, United States).

Retrieved from https://etd.ohiolink.edu/!etd.send_file?accession=akron1418220493&disposition =inline

Zwick, R. (2007). College admission testing. Report commissioned by National Association for College Admission Counseling. Retrieved from http://www.nacacnet.org/mediacenter/standardizedtesting/documents/standardizedtestingwhitepaper.pdf

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

131

APPENDICES Appendix A: Ethical Consent Letter to Students for Participation in the Research Study Appendix B: Ethical Consent Letter to Teachers for Participation in the Research Study Appendix C: Semi-structured tool for Focus Group Discussion Appendix D: Interview Guide for Teachers Appendix E: Guide for Individual Student’s Interview Appendix F: Comparative Performance of Students from Two Examination Systems Appendix G: Correlation of Students’ Scores in Different Performance Domains Appendix H: Correlation of Students’ Scores in SSC II and HSSC I Annual Exams Appendix I: Correlation of Students’ Performance (%) in SSC II and HSSC I Annual Exams with Respect to Shift in Examination Boards Appendix J: Students’ Performance in HSSC I Annual Examinations

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

132

Appendix A Ethical Consent Letter to Students for Participation in the Research Study

Dear Student I, Shairose Irfan Jessani, student of M Phil (Education) at Institute of Business Management (IoBM) am conducting a research study under the supervision of Dr. Muhammad Yousuf Sharjeel on the factors influencing students’ performance in annual board’s examinations. My M. Phil thesis is based on the study that will continue for about six months starting from (15/1/2016) to (15/06/2016).

The purpose of the study is to learn how different factors affect students’ performance, particularly for science subjects. This study will be beneficial for you and help you identify key areas leading to your academic strengths and challenges. In this study, focus group discussion will be conducted of about one-hour or so. All discussions will be audio-taped in order to preserve data for analysis purpose.

Your interview data will be used for analysis purpose and may be quoted in the final thesis or the future research papers published by me. However, confidentiality will be maintained throughout the study. Transcripts will be secured in the locked cabinet and all soft data will be destroyed no later than July 2018.

You can withdraw your participation from the research at any stage without any explanation. At any time you can refuse to answer a question, or participate in activity though there is no risk of participation in this study. I am requesting you for your voluntary participation. If you accept the objectives of the study, kindly sign the informed consent form below.

Signature: ______________

Name: __________________ Date ______________

I have read and understood the nature, processes, purpose, benefits and my rights during this study. Hereby, I am willing to voluntarily participate in this research.

Signature: _____________

Name: __________________

Date ______________

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

133

Appendix B Ethical Consent Letter to Teachers for Participation in the Research Study

Dear Teacher I, Shairose Irfan Jessani, student of M Phil (Education) at Institute of Business Management (IoBM) am conducting a research study to explore the relationship between College Entrance Test scores and Grade XI students’ performance in Annual Board Examinations. My M. Phil thesis is based on the study that will continue for about six months starting from (15/1/2016) to (15/06/2016).

In this study, informal discussion will be conducted with for about one-hour or so.

The discussion will be audio-taped in order to preserve data for analysis purpose. The purpose of the discussion is to learn how different factors affect Grade XI science students’ performance. This study will be beneficial for you to identify key areas leading to students’ academic strengths and challenges.

Your interview data will be used for analysis purpose and may be quoted in the final thesis or the future research papers published by me. However, confidentiality will be maintained throughout the study. Transcripts will be secured in the locked cabinet and all soft data will be destroyed no later than July 2018.

You can withdraw your participation from the research at any stage without any explanation. At any time you can refuse to answer a question, or participate in activity though there is no risk in the participation in this study. I am requesting you for your voluntary participation. If you accept the objectives of the study, kindly sign the informed consent form below.

Signature: ______________

Name: Shairose Jessani

Date: ____________

I have read and understood the nature, processes, purpose, benefits and my rights during this study. Hereby, I am willing to voluntarily participate in this research.

Signature: _____________

Name: __________________

Date ______________

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

134

Appendix C Semi-Structured Tool for Focus Group Discussion (FGD) 1. Why did you want to get admission in this college?

2. What factors have influenced your admission in this college?

3. In college admission process, how important are the following criteria for merit-based selection? (a) computer based test, (b) interview, (c) SSC results 4. How does CET influence students’ performance in HSSC board examinations?

5. Which of the following factors affect your performance in board examinations with reference to science subjects? Tick as many as applicable in your case. a. Parents’ qualification b. Teachers’ attitude c. Classroom pedagogy d. Examination system e. Family’s socio-economic status f. Teachers’ qualification g. Parents’ attitude h. Personal study traits i. Peer and Social influence j. Technology and media k. Home environment

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

135

Appendix D Interview Guide for Teachers

1. What is the role of testing in science education?

2. In your opinion, does CET predict students’ performance in annual board examinations for science subjects?

3. How are the following criteria in college admission process important for merit-based selection? (a) computer based test, (b) interview, (c) SSC results 4. What are the five most important factors that influence students’ performance in science?

5. How does testing influence students’ performance in science subjects?

6. How do you see your role in students’ learning in science?

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

136

Appendix E Guide for Individual Studentâ&#x20AC;&#x2122;s Interview

1. Share your brief family profile.

2. How do you see the role of your parents in your academic performance?

3. Which incident has triggered your interest for the field of medicine?

4. Why have you selected AKUEB in HSSC? 5. For some people AKUEB is a very challenging board, what do you think?

6. What are the school-related factors that influenced your HSSC I annual board performance in science?

7. How did your home environment influence your HSSC I annual board performance in science subjects?

8. Tell me about some of your experiences as students in science classroom during your Grade 11.

9. What did you like the most in your HSSC I science classroom?

10. What did you the dislike most in your HSSC I science classroom?

11. Did you face any challenge during your academic endeavors in HSSC I? If so, how did you cope with it?

12. Would you like to continue with science related career? Give reason(s).

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

137

Appendix F Comparative Performance of Students from two Examination Systems

Variables

BSEK

AKUEB

α

M

SD

M

SD

SSC I

83.49

5.24

77.92

5.85

0.00

[-7.13, -4.00]

CBT Science

54.28

13.60

55.61

10.44 0.44

[-2.09, 4.75]

CBT Chemistry

49.27

16.63

51.00

15.43

0.45

95% CI

[-2.77, 6.23]

CBT Biology

56.58

15.16

58.57

12.38

0.31

[-1.91, 5.89]

Note: SSC I = secondary school certificate- part I, CBT=computer-based test, M=mean scores, SD= standard deviation, CI= confidence interval, α = significance, AKUEB=Aga Khan University Examination Board (N=101), BSEK=Board of Secondary Education Karachi (N=96)

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

138

Appendix G Correlation of Studentsâ&#x20AC;&#x2122; Scores (%) in Different Performance Domains

Domains

N

M (SD)

r

P

SSC II (Overall) SSC II Science

191

78.54 (8.14) 83.86 (7.54)

0.54

0.00

CBT (Science) a HSSC I (Science) a

109

57.58 (12.48) 80.67 (12.09)

0.64

0.00

Interview HSSC I (Science)

220

80.59 (11.38) 76.90 (12.75)

0.16

0.01

School Internal Exam (Overall) HSSC I Annual Exam (Overall)

225

62.33 (14.03) 75.04 (9.38)

0.84

0.00

Note: SSC II = secondary school certificate- part II, CBT=computer-based test, M=mean scores, SD= standard deviation, N=sample size, HSSC I=Higher secondary school certificate-part I, r = Pearson correlation, p= significance. a

only for the sample sub- group which are affiliated with BIEK examination board

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

139

Appendix H Correlation of Studentsâ&#x20AC;&#x2122; Performance (%) in SSC II and HSSC I Annual Exams

Ability Group

N

M (SD)

r

P

Low ability a

SSC II HSSC I

17

74.47 6.75) 54.47 (4.69)

0.28

Mid-level ability b

SSC II HSSC I

83

75.36 (10.10) 68.93 (4.07)

0.06

0.59

High ability c

SSC II HSSC I

125

81.84 (5.00) 81.89 (3.88)

0.42

0.00

0.26

Note: SSC II = secondary school certificate- part II, HSSC I=Higher secondary school certificate-part I, M=mean scores, SD= standard deviation, N=sample size, r= Pearson correlation, p= significance. a

Students who secured less than 60% marks in HSSC I annual exams

b

Students who secured between 60% to 75% in HSSC I annual exams

c

Students who secured more than 75% in HSSC I annual exams.

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

140

Appendix I Correlation of Students’ Performance (%) in SSC II and HSSC I Annual Exams with Respect to Shift in Examination Boards

Shift Patten a

N

M (SD)

r

P

SSC II HSSC I

19

79.37 (5.70) 62.00 (7.94)

0.59

0.00

SSC II HSSC I

15

79.13 (5.02) 81.91 (5.47)

0.33

0.22

SSC II HSSC I

86

75.72 (5.14) 75.42 (7.69)

0.78

0.00

SSC II HSSC I

77

81.27 (10.72) 75.42 (9.65)

0.35

0.00

BSEK to AKUEB

AKUEB to BIEK

AKUEB to AKUEB

BSEK to BIEK

Note: SSC II = secondary school certificate- part II, HSSC I=Higher secondary school certificate-part I, BSEK=Board of Secondary Education Karachi, AKUEB=Aga Khan University Examination Board, BIEK=Board of Intermediate Education Karachi, M=mean scores, SD= standard deviation, N=sample size, r = Pearson correlation, p= significance. a

Shift in students’ affiliation with examination board from SSC to HSSC

Running Head: RELATIONSHIP BETWEEN CET SCORES AND ANNUAL PERFORMANCE

141

Appendix J Students’ Performance (%) in HSSC I Annual Examinations

Variables

HSSC I

BIEK

AKUEB

M

SD

M

SD

α

95% CI

77.26

9.04

72.95

9.25

0.00

[-6.72,1.90]

Note: HSSC I = Higher secondary school certificate- part I, M=mean scores, SD= standard deviation, CI= confidence interval, α = significance, AKUEB=Aga Khan University Examination Board (N=116), BIEK=Board of Intermediate Education Karachi (N=109)