Programming in Mental Health Research

Mental Sharpness
237 followers
Comments are disabled for this video.
But for the next 20 minutes or so, I'm going to talk about programming in mental health research,
really thinking about it from a team science perspective. And when I'm talking about programming,
in this case, I'm mainly talking about research software engineering. So I know we've had a lot
of data science earlier today, and I'll be talking about the intersection of research software
engineering and data science. But thinking about some of the digital mental health interventions
that I've been working on over the last 10 years or so, thinking about software as a medical device,
so software that's heavily regulated software, that's an active diagnostic or therapeutic
intervention. I'll say a little bit about data collection, linkage, standards, interoperability.
Lots of discussions during the break about interoperability and some of the challenges
there. So I'll say a little bit about that. One of my roles is working in the NHS Trust,
where I lead a research unit thinking about digital mental health and how that can help
clinical services. So I'll say a little bit about operational mental health service improvements as
well. And tying it all back then to team science, and particularly some of the roles that I think
were sometimes invisible in some of the digital mental health research that we do. So UX, UI
designers, technical project management, comms, and so on. Some of the roles that don't get the
same spotlight that some of the academics and clinical researchers do. So at the University
of Manchester, for the last 10 years or so, I've been working with the digital health software team.
We're a team that builds web and mobile technologies for mental health informatics research.
Actually, as Louise mentioned, we build digital health research platforms generally, but we have
a particular focus on mental health. And we work with agile approaches. Is everybody here familiar
with agile? Is that a term that's common to people? Okay, some are, some aren't. I'll say a little bit
about that later on. And we always work with a very co-design and user research approach as well.
So working with the end users of the platforms that we're building, co-designing the approaches,
and trying to come up with software that's useful and usable and meaningful to people.
As a software team, we're very much embedded within the research team. So we are not like
a software consultancy that you might contract into a project. You might just come and kind of
dip in and out. We build the research from the ground up. So we work with the core research team
to write the grant application, get involved right the way through. In terms of tech, we can talk
about R and Python. We also have a whole spectrum of technologies that we use. You name it, we
probably use it. We can go in quite a long time. We use, depending on the project, lots of different
technologies. So I've just got a few slides on some of the projects that we've worked on over
the years to give you a flavor of the work that we do. LinTouch was a project that started out
about 10 years ago now. It's a smartphone app for symptom monitoring for people who experience
psychosis. So it's a smartphone app, Android, iOS. We built the apps. And it integrates into
the electronic healthcare record for clinicians so that within the clinical record, clinicians
can see the data coming through. And the idea is that patients, service users, can monitor
their symptoms more effectively and clinicians can see the data coming through and hopefully
intervene to prevent relapse. Actasys is another app that we've built. It's a CBT-informed smartphone
app for people with serious mental illness. It's led by Sandra Bucci. It's been through two trials
with the MRC, the Medical Research Council. And again, it's a smartphone app. It feeds into a
backend platform that allows the data analytics to function and then to be exported for further
analysis as well. Connect is a new project for us. So it's a Wellcome Trust-funded project,
12.5 million pounds, for the Wellcome Trust to try and use digital markers to predict psychosis
relapse. So it's a huge remote data collection project in the first instance, using active
symptom monitoring using smartphones and passive symptom monitoring using passive data from
smartphones and fitness trackers, pulling all the data together, building an analytics platform,
and trying to use those data to predict when an individual is at risk of relapse.
iMind is something slightly different. So it's an app, a multimedia interactive platform.
It's for young people who have experience of online sexual abuse. And it's really using a
therapy called mentalization therapy to try and improve young people's ability to understand
emotions of adults and peers, to try and feel more confident in difficult circumstances,
and to try and build some psychoeducational material around it. This is a slightly different
project for us. It came out of running lots and lots of focus groups and traditional
PPIE activities with young people for mental health. And although they were successful and
useful, we got lots of interesting feedback from the workshops and interviews and different kinds
of traditional face-to-face PPIE. Young people started to tell us that, you know, they found that
coming to the university site to sit in a two-hour focus group not really the most exciting use of
their time. And they wondered if there was a digital platform that we could build that would
enable PPIE to function a little bit better. We also noticed, as you're probably familiar with lots of
people in the room, that the same kinds of people were attending the PPIE sessions. We were missing
large sections of people from attending. So there was not really a great diversity of representation
in those groups. So PPIE apps like Tinder for PPIE, you can swipe left and right in different
projects. You can vote on different things up and down. It's not to replace PPIE with the kind of
rich feedback that you get from those sessions, but it's just to get some quick and dirty feedback.
So a slightly different kind of project then is the Core Mental Health Dataset project.
So this was an NIHR-funded project, and the idea is to try and understand the links between physical
health and mental health. So we worked with stakeholders, lots of public contributors came
to focus groups and sessions, trying to understand what kinds of mental health data people would be
willing to share with us. And then thinking about how could we collect that information in standardized
ways that would enable us to link it with physical health data later on. And really what we want to
do is to build up a national picture of mental health across the UK. So we know that at the moment
on the Clinical Research Network, the vast majority of studies are on physical health. There's very
little mental health data usually collected in those. If they are collected, they're not collected
in standardized ways, and it makes it really, really difficult then to analyze the data.
So this project started out as an NIHR grant. It's now being taken forward in the MRC DataMind
project, which lots of you will know about already. Another project that's linked to DataMind is
trying to support routine equity auditing of clinical trials. So at the moment, we know that
there's no routine equity auditing of clinical trials on the Clinical Research Network. There's
about a million participants a year. And we know that clinical research data is going to reflect
the underlying sample. So if we've got a very biased sample, then we've got a problem. So what
we wanted to do was to build an auditing tool, a very simple auditing tool, to display the
representativeness of participants in clinical trials across mental health studies. That sounds
really easy when you think, oh, we're just looking at sex, age, ethnicity. Actually, when we piloted
it in Greater Manchester, loads and loads of studies don't have sex, age, ethnicity captured
to begin with, so it makes it quite difficult. And then thinking about what are the denominators,
so what exactly are the expected samples for local, regional, and national populations and
by mental health condition. It's not quite as trivial as it sounds in the beginning. But the
overall goal then is to try and identify specific underserved groups within the NIHR portfolio as a
whole and then within individual studies so that we can target it at different touch points. It
might be during recruitment, we try to reach out to a more diverse sample, or it might be routine
auditing across the portfolio overall. This project is in collaboration with MQ, and it's
looking at the CAMS referral pathway. So when we looked at the research and we talked to people,
we discovered that about 25% of people who are referred to CAMS are rejected at the point of
triage. So you have lots and lots of young people who go through a very lengthy referral process,
it often takes quite a while for referrers, GPs, and school staff to refer people into CAMS,
and then they wait for eight weeks or more, and then they're rejected, and they're not always
sent anywhere else. So it's really distressing, as you can imagine, for young people, for the families,
and we wanted to understand what was going on and what we could do about it. So it's really
early stage discovery research in software terms, but what people told us was we just
want a digital platform to be able to tell us what are the options, where are we in the pathway,
can you give us some notification about when we'll hear about the next step, what the other options
are, and so that's where we're at in this project at the moment. We've identified lots of problems,
we've talked to hundreds of people about it, and now we're trying to think about how do we take
that digital platform forward, and what do we do to get to the next stage and support people better.
In research, we often talk about PPIE and software, we often talk about user research,
PPIE meets user research, I think, in building digital mental health technologies,
creating software that provides meaningful and relevant experiences for everyone.
We do an awful lot of user research in the work that we do, we always prototype before we program,
it's much less expensive to create design prototypes than it is to pay software engineers to code,
it gives users something to play with, gives them a working prototype they can click through,
they can give us quick feedback, we can build it in rapid iterations, it reduces the programming
time for us because we're not building things that's never used, and it reduces the floated
code so you're not building a function that will never be used. We do lots of usability testing as
well, right through the stages of software development to try and gain usability insight,
so we typically track where people are clicking on the screen to see if they're completing the
tasks that we give them, we see if they can complete the task, how long it takes them,
and so on. So it adds some quantitative data to the qualitative kind of discussions and think
sessions that we have as well. We know there's lots of people in the world with disability,
we try to program for accessibility, we comply with WCAG standards, that's kind of the base level
that we need to comply with anyway, but we try to think a bit more broadly about it and build
software that's usable for everybody. We try to support assistive technologies, we could do better
at that, but we do our best with the resources that we have, and we think about accessibility
in terms of content design and development. So agile technologies for people who aren't familiar
with them, really agile came about because the traditional scenario was that you would talk to
a client or somebody who wanted to build a piece of software, they would give you a list of
requirements, you'd write them all down, you'd agree to the plan, you'd go away for three, six
months, maybe build a software product, take it back to them and say there you go, there's software,
and they'd say that's not what I wanted at all, I want something totally different. So agile
really came about to try and reduce some of that overhead, so the whole idea is you build quickly,
you build lots of different iterations as you go, you change, you're really adaptive to change,
and you try and reduce some of the overhead that traditional software processes had, so you build
working software over comprehensive documentation and you try and do a very collaborative process
rather than working in isolation. So we do say we're agile in the team, we have daily stand-ups,
that's where you meet and say what you did yesterday, what you're going to do tomorrow,
and if there's any blockers anyway, and we have three weekly sprints and we plan in three weeks,
we set goals and we review the work and we see if there were any blockers or problems along the
way, and we use JIRA, which is technology just to project management basically, so you create
tickets and you estimate how long the work is going to take, but then you put agile technologies
into a research landscape and it doesn't quite go to plan, so typically we get involved in research
grant application maybe two years in advance, we specify and cost what we want to do, we report
the project milestones, our collaborators or the researchers, academics, depend on us delivering
what we said in the grant application, and for software as a medical device, change means
overhead, so you have all the documentation and so on to change if you change anything.
So it's not quite a perfect match and there's lots of challenges along the way.
Another problem is agile says, well, working software over comprehensive documentation,
if we're developing software as medical devices, we have a raft of documentation to prepare,
there's 82 documents for submission to the MHRA, your class two medical device,
we have quality management systems for software development, really lengthy process that's on top
of the usual research documents, data management plans, research ethics, so on, that you'll all be
familiar with. So it's not quite perfect in terms of minimizing documentation. We get these questions
which you probably are all familiar with as well, so you'd be sitting in a meeting and you've got
the project plan all agreed and then somebody has an idea and they say, oh, can you just link this
software to the electronic healthcare record, or can you just pull and link data from routinely
collected NHS data, or can you just link with an external third party, or can you just gamify it,
we get that a lot when we do build software for young people, people say, can you just gamify it,
can you just personalize it a bit more, can you add a simple button to, and that could be anything
from can you add a simple button to run a machine learning algorithm that nobody's ever invented
yet, to something entirely different, and can you write a sustainability plan into the grant
application, and I think in programming and building software, really difficult to build a
sustainability plan into a grant application. So we've done different ways in the past at
Manchester, we've spun out companies to take forward some of the research, we've extended
research grants, we've got multiple grants one after the other, really difficult to predict that
and plan that in advance. So I guess some questions for you all, how can we link data more easily,
how can we retain engineers, data scientists, programmers against industry competitiveness,
that's one of the challenges I face as somebody managing a team of software engineers,
I'm constantly losing people to industry because very competitive offer there, how can we be agile,
which is really desirable in software, we want to change quickly and yet we're constrained by all
these processes, how can we create sustainability plans that are realistic, and how can we adopt
the team science approach within what are effectively hierarchical institutions in a lot
of cases. I'm really interested to hear any solutions to these challenges. Okay, and then
just an invitation to people here to join us, so I run a digital health leading research group every
Tuesday morning, it's remote by Zoom, everybody here is welcome to join, dip in and out if you
like, and we have user experience meetings, UX meetings on a Wednesday just for 15 minutes,
everybody's welcome to join us as well where we talk about all aspects, not just digital mental
health but digital health generally, and just get in touch if you'd like to follow up on anything.
Thanks. Anyone with thoughts, comments, questions? Yes?
I'm actually curious as to how you organise your teams, so you're working in an engineering team,
but are you also closely working with other professionals like the user
research professionals and so on, or do you sort of work on your sprint and then
pass it on to them and then sort of have a review? Because I have found that we've recently changed
it where we have sort of project groups, we have sort of different members from different
sides of the fence, so you'll have a data scientist, data engineer, a user experience person,
a sort of project manager overseeing the sprint, and I have found that even though communication
is very chaotic at the beginning, a lot of the sort of issues that you come up with later on
do actually iron themselves out in those first few weeks of chaos. So yeah, I'm curious as to
your experiences with that and what are your arguments on each side, I guess? Yeah, so historically
we were a team of engineers and we didn't have any designers, we outsourced the design and the
usability, and some of the project management was sort of ad hoc from different people at the
university. Now the team has technical project managers, product owners, business analysts,
testers, quality assurance, UX designers in-house, so we work really closely with them, they join our
daily stand-ups, so we've got everybody together. Data scientists are a slightly different group
in our centre, but we work closely with them across different projects. They don't attend
daily stand-ups and they don't work quite the same way as us, but they're close enough that we
jump in a meeting with them if we need to. We definitely found that bringing it all into a
centralised team is much more effective for us than outsourcing bits and pieces of it or trying
to collaborate across different groups. That works better for us. Just to follow on, but to be able to do that
you need to have big grants or sustained grants for a long time, don't you? So for people who
are just starting, do you suggest that they start with outsourcing and then building? Or find a group
to connect to. There's other groups like ours, so find a group that you can embed yourself in, I guess.
I think, I mean, outsourcing is another option. We still occasionally outsource bits and pieces if we
have too much work in. We've been growing and growing the last 10 years and we can't keep up with the demand,
so I think that's a sign that if you're interested in, you know, build your own team, build the people
around you. And I would say that's worked for us a lot better than outsourcing than it has in the past.
It's just much more easy to control and keep on top and deliver time and target when you've got
full control over people. And to share knowledge across the team, because the design, past the
handover from the designer to the engineer is a lot smoother. If they're in-housing, you just bounce
back and forth and say that's not the file I need, it's the other one, rather than having to wait for
potentially somebody in a different time zone or whatever to get back to me.
I just wanted to say I was blown away by that whole thing because I'm so inspired,
particularly around the PPI app and the PPI app with co-design, so I thought that was amazing.
I've just got a couple of questions on here, but that's okay. So first of all, how is the
experience of how it's different of the PPI app versus like just a Zoom? Because I've tried
very hard to sort of make PPI and co-production much more accessible and much more national,
voices diverse as much as possible, and by Zoom essentially doing that. So that's my first
question, the differences between them. And the second one is in the actual app,
is it sort of an ad hoc sort of in and out sort of process? Is it all the way through the research
stages or is it just the design and sort of dissemination ideas and how does the process
work essentially? Yeah, great question, thanks. So first question, the PPI app is not supposed
to replace Zoom or those kinds of interactions. You don't get the rich qualitative data from the
PPI app that you will in those more traditional settings. It's not for that. It's much more for
kind of quick and dirty feedback. You know, we've got this question that we're not sure
there's going to be any interest in researching. What do you think about it? Do you think it's a
yes or a no kind of thing? One of the benefits of the PPI app, I think, is that because we're
pulling all the research together, we can see trends across time. We can see what works and
what doesn't work. So we're hoping that we'll learn what works and what people like in terms
of presenting PPI opportunities to people because we can see all the metrics coming in from all the
different projects that are on there. So we're hoping that we'll learn a bit over time as well
as to what works and what doesn't work. Did I miss a question? Oh, it's right through, sorry. Yeah,
it's right across the lifetime and the nice thing about the PPI app that people really like is that
you get updates on the project. So you say, I'm interested in this project and then, you know,
you hear two weeks later, oh, we've got funding or we're going to this event with it or would you
like to join this session? And that's one of the things people tell us. You know, you go to a PPI
workshop often and you never hear again about what happens to research. You might get a thank you for
attending the workshop, but then you don't tell us it made an impact in this way. So the idea is the
app will tell people how their research is progressing. Hi, yeah, thanks for an inspiring talk about
digital health. It's really cool. I just had a quick question, really. I was kind of attracted
to the last point there, talking about a team science approach. I was just wondering, because
that's a bit of a buzzword that I've noticed. I was curious, a few small questions. First of all,
what do you mean by that in terms of working within a research or hierarchical environment?
Second one, is it being considered or tried? And if so, what barriers did you face? And the third,
would there be scope to move towards that? Like if you're trying to affect the higher education
policy or something, would you then maybe go to one of the regulators and ask them to try and build
into things? I'm just curious how it looks on the ground. Yeah, great question. Thanks. So it's
something I think we've been doing informally in Manchester for quite a while, in that we all work
as a multidisciplinary team. So almost all the research I've presented so far is, you know,
multidisciplinary. So we've got the engineers working with clinicians and academics and so on.
We're adopting it formally within the Wellcome Connect project. So I'll keep you posted on how
that goes. And I think that's very much from the outset, establishing just some ground rules and
setting up a glossary of terms. You know, we work in different disciplines. Some of the words mean
different things in different disciplines, making everybody aware of it from the outset.
I added there within hierarchical organisations, because I think there is sometimes still a
challenge in that there's a, when we work in some of the bigger institutions, there is often
an academic, a clinician, and they're often seen as, you know, sort of the leaders of the project,
which effectively they often are. But then some of the other roles are just not recognised as
much as they should be, in my view. So people, you know, like the UX designers and things like
that, they're not always invited to events that they should be invited to, or not recognised in
the same way, not put on papers, even though they contributed, it's not. And I think we've
been really good about doing that at Manchester. And it's kind of on the radar now, but I'm not
sure that happens universally. And I think it is still a challenge in some of the more formal
grant applications where you, you can't even apply if you're under some contract.
And while you're thinking about your next question, and it's about the sustainability.
So can you tell me, do we really need to think about sustainability? Is it possible that, you
know, whatever kind of digital app or services that we create can be kind of short-lived? And
that's all right. Absolutely. I think some should maybe be shorter-lived than they are sometimes.
But I think one of the problems I see, and this comes from my trust role. So when I work
in the trust, and we're evaluating products, often things come through that have no evidence base.
And I know there's lots of products that have been through RCTs, really lengthy extensive RCTs that
do have an evidence base, and they're not coming through the same pipeline because they've fallen
after the grant has ended. There's no funding to sustain them. Digital tech needs to be hosted.
It needs some engineers on board to keep it alive. And so I see that gap then,
if we've got evidence-based things, it's not being funded. And I see commercial products
that look really flashy. People are like, oh, that looks great, or they've heard about it,
but actually there's no evidence base. And that's where the challenge is for me.
When you provide this kind of, say like, online therapy, or, you know, this private discussion
between the patient, or the participant, and the machine, you could say, how do you check
it doesn't go, you know, weird, or bad, or the machine doesn't say anything that shouldn't?
Yeah, I mean,
How do you make sure of that?
Yeah, it's a really good question. That's why we have six validation and testing protocols
to follow, especially if software is a medical device, then we have to prove it well in advance,
as well as working with the PPI group, the co-design groups. But there is formal stages
of validation testing along the way as well. Thank you. My question is about how we can
make it more inclusive of older people, like older users. In digital tech generally.
Yeah, it's a really interesting question. And I think it comes up a lot when we talk about
digital health and digital tech about which groups are going to be excluded. And there's
usually older people come up as the key group that will be excluded. One of the things that we do,
at Greater Manchester Mental Health, we've set up a digital navigator program, which is a trained
group of volunteers to support people if they don't know how to use tech. Not everybody wants
to use tech, because that's the starting point. So we can't assume that people want to use it.
But if it's skills or competence or devices, access to devices, they're things we can help with.
And so providing access to devices, training people or giving people support if they need it.
And that's something we try and provide through the digital navigator program that we set up. So
training people to support other people. Can you tell me about how you establish
kind of constructive relationship with the industry, given that you may not have the same
goals somehow. So how can young people reach out to people from the industry and get them involved
in their project, being confident that they, you know, both parties want to have the same outcome
or not always aiming for, I don't know, maybe I have a prejudice that I see industry profit.
Well, I think that most researchers want, you know, kind of treatment or research findings.
How do you manage it? Or is that your experience? Yeah, I think it's really difficult. So we have
worked with some really good industry partners in the past. We tend to get approached by industry
partners who want us to typically evaluate the products, they've built something, there's no
evidence base, can you evaluate it for us to say it's amazing, and then we can sell it much better.
And that's typically the research we don't want to do.
But we have had some really fruitful collaborations in the industry. So
we have evaluated some products that we think, you know, we've kind of done a pre-evaluation,
this is amazing, and we kind of want to support them, because they've got the same goals,
it's very ethical, they've got the same goals as us, but they don't have that academic research
background. And then I think you've got a, almost a duty to try and get those products,
you know, more widely used, because there's so few things out there that are,
so we do work like that. I think there's also quite a lot of skill and industry that we can
learn from. I talked about outsourcing previously, it wasn't the right solution for us, but we have
learned through outsourcing projects previously, you know, how they do it this way, that's quite
interesting. Could we learn some techniques and things from that? So their collaborations
have worked for us. Thank you. Thank you very much. Have a good day.
