With all that has been going on it has been a wee while since we heard from a researcher who is in the thick of working with administrative data in Scotland. In this Researcher Experience post, we hear from Dr Drew Altshul, Research Associate in the Department of Psychology at the University of Edinburgh, who has been navigating the administrative data landscape in Scotland for around two years. Drew works with a large linked data set of the Scottish Mental Survey 1947, 36 day sample, Scottish Longitudinal Study (census data), Prescribing Information System (PIS) and Scottish Morbidity Record for Mental Health Inpatient and Day Case (SMR04).
In Drew’s account of working with administrative data, the familiar challenges of timing, unforeseen circumstances and working in the safe setting, rear their heads. However, like the other researchers we have heard from, the ‘seeing the glass half full’ attitude and optimism for the need to press on in spite of these challenges endures. In particular, Drew points out the useful discoveries him and his colleagues made whilst waiting for data access, which would ultimately improve their research output in the long run. I think this point rings true for me especially, after all, eCRUSADers wouldn’t exist if it weren’t for the wait for data.
Over to you Drew:
Overview of my research
I’ve yet to do much of work with our main variables of interest, as we only recently were granted access to a few of the data sets we requested. However, while we were working on obtaining and waiting for access we followed some side avenues in part to prepare ourselves for working with the data, and in part because we thought of research questions that we thought were interesting in their own right. For example, we are interested in how early life socioeconomic conditions, commonly represented by the father’s occupational social class, relate to mental health later on in life. However, our data set is based on the participants of the Scottish mental survey 1947; these individuals were all born in 1936, and because of World War II, reports of fathers’ occupations from censuses carried out during participants’ early lives are unreliable, not representative, and often missing. In order to improve on our data set, we dug deeper into the data we were aiming to link, pulling out additional, historical occupation information, and coding these data ourselves. This in turn lead to a machine learning approach to classifying historical social class data, which can be used in the future by people working with historical social class data. So it goes to show how much interesting, useful work you can wind up doing along the way!
Summary of any challenges faced
The process is long and convoluted, and at seemingly every turn. I was fortunate because I joined the project relatively late, although when I joined we thought we would have access to the data in a few months’ time, rather than two years later. I did what I could to help with the application processes, but ultimately this work predominantly falls on the shoulders of a single person, and most of one’s time in this area is not spent working on forms, but waiting for other people to get back to you.
A large amount of time and effort goes into processing and preparing data before linkage, but that does not mean that the data are clean and easy to work with once you get a hold of them. You are likely going to need to spend significant time cleaning and otherwise processing your data before you can analyse them.
There are advantages to having to layout analyses in advance during the application process: essentially, this forces you to pre-register your work, which is an important step in doing reproducible science. However, a run-of-the-mill pre-registration has considerable flexibility, and this is not so much the case with the analyses we plan for our data. All output must be checked for privacy and security concerns, so if we want to tweak an analysis or run a sensitivity analysis, for instance at the request of a reviewer, every different analysis that we want to take out of the safe haven environment needs to be checked, and that process can take weeks.
Thoughts for fellow and future eCRUSADers
You ought to think very carefully about timing, in particular you ought to expect significant delays. If possible, try to plan for multiple scenarios, and make sure you have meaningful work you can do while you wait out the access process. The processes for accessing data are supposedly being streamlined and improving, but it is worth investing in your relationships with the people along the data access pipeline, as they are best served to help you manage your expectations.
It can be a difficult and frustrating area to work in, but there are big potential payoffs, including large sample sizes and long-term follow-up, sometimes across many decades. These are types of data that sometimes cannot be obtained in any other way, and this allows for novel, meaningful research questions to be asked and answered.
Public Benefit Privacy Panel Timelines
Preparation of PBPP application: 01/06-2018 – 21/08/2018 (about 12 weeks)
Submission to initial PBPP approval: 05/10/2018 (about 12 weeks)
PBPP approval to data access: 16/06/2020 (about 1 year and 6 months)