Our first Researcher Experience post is from Matthew Iveson, Senior Data Scientist at the University of Edinburgh. Matthew has been working with Scottish administrative records for about four years. Data sets he has worked with include Scottish Morbidity Records, Scottish Census, Prescribing Information System, NHS Central Register, NRS Births, Deaths and Marriages, Scottish Stroke Care Audit. He has also worked with the Scottish Longitudinal Study, a set of pre linked administrative data sets. We asked Matthew to tell us a bit about his research and the routine data he has worked with, what he saw were some of the key challenges in accessing and using administrative records, and to offer his thoughts to early career researchers hoping to work with this kind of data.
Brief overview of Matthew’s research
My work has mainly focused around using data linkage to reconstruct the life-courses of individuals who took part in the Scottish Mental Survey 1947, a nation-wide survey of age-11 thinking skills conducted in Scottish Schools in 1947. These individuals, now aged over 80 years-old, have experienced a lifetime of changes in health and socioeconomic circumstances, and are an extremely important opportunity for examining how early-life circumstances can have a lasting impact on health and wellbeing across the life course.
So far, I have used linked data to show that individuals with higher childhood cognitive ability, better socioeconomic circumstances and more education are less likely to die, less likely to report a long-term function-limiting illness in older age, more likely to be economically active in later life, more likely to retire later and of their own volition, and so on. I’ve also tried to establish the mechanisms by which childhood advantage affects health and wellbeing. I am currently waiting for data to examine whether factors from across the life course can be used to predict whether someone will require care in later life (including the type of care required), how well individuals can recover from a stroke, and whether someone will respond to a given antidepressant medication.
Summary of challenges faced
One of the biggest issues I faced was in terms of timing. In some instances I have been waiting over 3 years for data. There have been several delays along the way, due to changes to the data access process (both over time and between organisations), queues for submitting forms to data controllers, changes to the legal landscape for data sharing (such as GDPR) and loss of submitted paperwork. The problem is that these delays are relatively common, and they result in a timescale that is not achievable under normal funding conditions. Since most early-career researchers find themselves on short-term contracts, they risk not getting data before their contracts expire, and since they are judged more than most on their productivity, these delays can seriously hamper a researcher’s career trajectory.
The delays also highlight the fragility of the data access process. Getting to know key people in each organisation is one of the best ways to get through the process smoothly, but if these people leave their expertise often go with them. One example is that, during my project, the lawyer in charge of reviewing requests for census data left. Their replacement was understandably less confident about data sharing, and decided to re-review the laws surrounding the use of census data for research. Data controllers and other involved organisations need to ensure that knowledge and expertise are distributed across their teams, and need to invest in the infrastructure and staff that can ensure a robust system for the future.
Thoughts for early-career researchers
While organisations need to make things easier, researchers themselves need to manage their own expectations – gaining access to routinely-collected data, especially linked data, takes a very significant amount of time and effort. It’s worth planning well in advance and making sure that you can stay busy and productive while you wait for data to arrive. It’s also worth thinking about pre-linked datasets such as the Scottish Longitudinal Study if you’re short on time. Regardless of how you engage with routinely collected data and how long it takes, bear in mind that you’re learning an incredibly rare and valuable set of skills. Things are slowly getting better, faster and easier, but organisations are still fine-tuning their processes and a lot of the data is still new to the research scene. If you do have the time – and the perseverance – then administrative data is an extremely powerful tool that will help you to answer the largest and most difficult questions faced by society.