Positive changes to Graduate Outcomes Official Statistics
In my last blog (‘Future proofing our approach to weighting’) in February, I updated data users on HESA’s schedule for publication of the results of the third year of the Graduate Outcomes survey (2019/20). I also explained our approach to ongoing assessment of non-response bias and statistical weighting. This further update is intended to provide more information for users on the upcoming release to be published at the end of May 2022.
Two become one
No, not a reference to the Spice Girls (for those of us of a certain age range) but a change in approach to publication of the survey results. During the first two survey years, this comprised the release of a statistical bulletin containing overall HE sector-level statistics, followed a week or so later by a more comprehensive release of open data which included data disaggregated by HE provider. This was partly due to the practicalities of work scheduling and partly a legacy approach that HESA has used for many years.
However, user needs have evolved and modern statistical production processes enable more flexibility. Our users have indicated that they would prefer access to the full content as early as possible and so, from the 2022 publication year (containing data from the 2019/20 graduate cohort), we will publish a single combined data product – ‘Graduate Outcomes 2019/20: data and statistics’. As previously announced, this is scheduled for the end of May 2022.
Design work is underway for the combined product and some limited rationalisation of data tables between the two existing products may be needed in one or two areas to reduce duplication of content. Users can be confident that no data content will be lost, but the table structures and/or formats may differ in a few places. Where this happens, we will include some information in the release to enable users to understand where previous content can now be located.
No longer experimental
Data and statistics releases derived from the first two years of the survey have been published by HESA under the Experimental Statistics label. As previously explained by my colleague, this indicated that the statistical publications were undergoing a period of evaluation and development. HESA has now determined that this process has been successfully completed and consequently that the Experimental Statistics label will be removed from this year’s edition onwards. This means that the newly-combined data product will revert to being a ‘standard’ Official Statistics product.
Why has HESA reached this conclusion? There are a number of factors.
Firstly, the survey methodology, quality-assurance, analytical and statistical production processes have reached a stable state. We believe the data source is appropriate to meet user needs, the methods used to generate the published data are sound and the quality processes in place provide assurance that statistics are accurate, reliable, coherent and timely.
Secondly the statistical and data publication content, which has changed and evolved over the past two years, is now also stable. Our intention to merge the two previous publications doesn’t affect the conclusion that the core content is stable, in terms of the treatment of data variables, preparation and aggregation methodologies, and supporting definitions with documented assessment of quality. Combining the products will merely change the presentation slightly.
Thirdly, user feedback suggests that the published statistics are meeting user needs. The data is often referenced in the media and it is serving an important role in supporting public understanding of the outcomes of higher education study. Survey data is also being actively utilised within policy and regulatory work within the four nations of the UK.
Finally, the first edition of the statistics from the survey has undergone formal assessment by the Office for Statistics Regulation (OSR). Their published findings indicated a “range of positive features that demonstrate the trustworthiness, quality and value of the statistics”. Their work also included three main recommendations, two of which were addressed in last year’s edition of the statistics (limiting pre-publication access and formulating a comprehensive user guide). The third – combining the two products into one – will be addressed this year, as stated above.
None of these areas of assessment indicates that our work is done however and we have no intention of resting on our laurels. The survey is subject to a programme of continuous review and improvement and this will continue. In addition, as with all HESA statistical products, we will undertake ongoing assessment and we will continue to improve and evolve the publication as user needs evolve over time. But from this point forwards the ongoing improvement follows a stable process, aligned with HESA’s overall approach to preparation and publication of data and statistics. Therefore, the experimental statistics designation is no longer appropriate.
Pandemic impacts
HESA has undertaken specific assessments of pandemic impact on the data and published our assessment alongside both previous editions of the statistics. We will repeat this process this year, aiming to understand whether the consequences of COVID-19 have affected the quality, characteristics and coverage of data. Although the direct social impacts of the pandemic appear to be reducing within the UK at the time of writing, the economic impacts on the experiences of graduates as they enter the labour market are likely to be felt for many years.
The Graduate Outcomes survey provides us with an invaluable longitudinal perspective on the life experiences and motivations of leavers from UK higher education, enabling us to better understand the long-term social and economic impacts of world events, among this sizeable proportion of the population. My colleague, Neha Agarwal, put it nicely that “It is only when we measure the impact, that the government, higher education providers and other users of the data can begin to take proportionate steps to address it in the future” in her blog from November 2020.
As in previous years, HESA colleagues will be talking users through the 19/20 statistics and the pandemic impact assessment in a webinar on the day of release. We will be providing booking details for this webinar in the coming months so please look out for that in the usual channels.
