TW Report

Warner and Spencer

St John's College Access and Outreach Impact Report

Understanding Proxy Metrics Measuring disadvantage There are many ways of understanding and quantifying disadvantage. The simple splits between state and independent, selective and non-selective have never been sufficient to encompass the spectrum of multiple disadvantages that pupils face.

Indeed, even on a more granular level, there is not a single metric that encapsulates this spectrum - hence the emergence of composite contextual bands that triangulate between multiple metrics.

The Access Office records participant data on the individual, school and postcode level. This gives a range of proxy data to understand the disadvantages faced by pupils. Due to the project's scope, it has focussed in particular on POLAR4.

POLAR4 POLAR4 is a nationally calculated geographical metric that assesses the likelihood of progression to any HE institution for those in a given postcode region. POLAR classifies local areas into five quintiles based on the proportion of people who enter higher education aged 18 or 19. The 20% of areas with the lowest participation are designated Quintile 1.

POLAR4 does not necessarily converge with other ways of measuring disadvantage, particularly in London. Nevertheless, it remains a prevalent proxy measure of disadvantage across UK HE, and - crucially for this analysis offers widely available comparison between different UK universities.

POLAR4: The National Picture Q5 students outnumber Q1 students at university by a 3:1 margin.

The gap between Q5 students and all other quintiles becomes much more pronounced in top half institutions.

55% of Q1 students who go to university attend a top half institution, as opposed to 72% of Q5 students.

The Guardian University League Tables (2017-2020)

This graph tracks 'placed applicants’ (UCAS data) across the five quintiles in universities.

Unsurprisingly, students from less advantaged backgrounds make up a lower proportion of the student body at higher ranked universities. Only Q5 students make up greater proportions of the student body at higher ranked institutions than lower ranked institutions.

This graph displays this data on a trend level rather than individual institutions.

Programme Impact: POLAR4 Quintile 1

78% of tracked students on a St John's programme attend a top third institution, with 10% attending one within the top three.

On average, only 38% of Q1 students who go to university will study at a top third institution.

Understanding this analysis How does the graph work?

On the X axis are university rankings. The higher-ranked universities (Guardian League Tables 2017-20) are further right.

On the Y axis is the cumulative % of students who attend a university of this ranking or less, from Q1 based on the national data.

Each dot represents a university.

The red flag signifies the top third of universities; every dot to the right of this flag is a top third university. This is a group which has significant overlap with the Russell Group, but includes competitive, high-tariff institutions such as St Andrew's that are not otherwise included.

What do the two shapes show?

The upper row of dots represents national data available from UCAS. Red dots are universities at which students who have participated in St John's access initiatives have gone on to study. They are simply flagged but still use the national data for their position. This works as a loose control.

The lower, concave shape dots represents Inspire data. This represents the same universities as the upper row of red dots, but using the inspire data rather than national data.

Across the national average, only 38% of Q1 students who go to university will study at a top third institution. This is represented by the upper row of dots.

However, 78% of Q1 students who engage with St John’s access programmes attend a top third institution, with 10% attending one within the top three. This is represented by the lower row of dots.

We can see this through the point at which each line intersects the red flag.

Programme Impact: POLAR4 Q2-5

Understanding this analysis These graphs apply the same analysis to the remaining POLAR4 Quintiles There is a strong correlation between engaging with St John’s access programmes and attending significantly higher-ranked universities than peers from the same POLAR4 quintile.

The same curve is echoed across all five groups, which shows that the impact of the activities is not unique to students of any one background.

What else can we find out?

These graphs also demonstrate that a high proportion of participants go on to study at the country’s most elite institutions. A fifth of participants from Q2-Q5 backgrounds go on to study at a university ranked within the top three (Oxford, Cambridge or St Andrews in these years).

This analysis also shows that there is a high floor for student performance, with over 80% of students attending a top third institution across Q2-Q5.

What does this mean?

Proving causality is challenging, as it would rely on both complete base data and the ability to conduct randomised control trials.

This standard of evidence poses questions of feasibility and of ethics.

However, it is clear that ambitious and talented students (regardless of social background) were keen to get involved in the College’s programmes, and that those who did outperformed their national peers.

Whilst St John’s Access initiatives are not the only reason that programme participants significantly outperform their peers, it is reasonable to conclude that their work improves students’ outcomes and encourages them to consider elite universities as an achievable goal.

Contact us To discuss supporting St John's Access and Outreach Programmes, please contact: access@sjc.ox.ac.uk