Digital Technology for Mental Health PSP Protocol

Digital Technology for Mental Health: Asking the right questions #digitalMHQ: purpose and background

The purpose of this protocol is to set out the aims, objectives and commitments of the Digital Technology for Mental Health Priority Setting Partnership (PSP) and the basic roles and responsibilities of the partners therein. It is recommended that the Protocol is reviewed by the Steering Group and updated on at least a quarterly basis.

The James Lind Alliance (JLA) is a non-profit making initiative, established in 2004. It brings patients, carers and clinicians together in Priority Setting Partnerships (PSPs). These partnerships identify and prioritise uncertainties, or ‘unanswered questions’, about the effects of treatments that they agree are the most important. The aim of this is to help ensure that those who fund health research are aware of what really matters to both patients and clinicians. The National Institute for Health and Care Research (NIHR – www.nihr.ac.uk) funds the infrastructure of the JLA to oversee the processes for priority setting partnerships, based at the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton.

Digital technology has the potential to transform mental healthcare by connecting patients, services and health data in novel ways not previously available and through combination with existing treatments. Digital online and mobile applications have the potential to offer patients greater access to information and services and enhance clinical management and early intervention through access to real-time patient data. However, substantial gaps exist in the evidence base underlying these technologies. Greater patient and clinician involvement is needed to evaluate digital technologies and ensure they target unmet needs, maintain public trust and improve clinical outcomes (Hollis et all, 2015).

Therefore, NIHR MindTech Healthcare Technology Cooperative set out to build a consortium to undertake a PSP with a focus on the unanswered research questions about digital technology for mental health. Work is already underway with the NewMind network to identify and prioritise underpinning engineering and physical science research challenges to transform mental health care. However, further work is required to prioritise the research needed to address the uncertainties about the application of technologies in clinical practice and their effects. Despite the wide range of conditions and types of technology in mental health field, it is likely that there are a number of overarching unanswered research questions about the potential for technologies that apply generally. For example, these might include questions such as, do technologies increase access to services?; to what extent do technologies promote independence and self-management?; does using wellbeing digital tools prevent longer term mental health problems? Given that resources for research are limited, it is important for research funders to understand how patients, their supporters and clinicians prioritise these unanswered questions so that future research can be targeted accordingly.

Hosting the PSP within MindTech will bring the advantages of a national network of technology interested mental health service users/carers, clinicians and researchers along with high profile events in the field, such as the MindTech National Symposium (planned for 8 December 2016).

Aims and objectives of Digital Technology for Mental Health: Asking the right questions

The aim of this PSP is to identify current uncertainties about the use of digital technologies for mental health problems, and then prioritise those uncertainties that people with lived experience and health and social care professionals agree are the most important.

In this PSP we are specifically interested in uncertainties about the impacts, both positive and negative, of digital technologies being used with the intention of benefiting people's mental health. We are excluding from this PSP the impacts of the general use of digital technologies.

For the purposes of this PSP, ‘digital technologies’ refers to all interventions, tools and services (including, but not limited to, remote therapy, diagnostics, treatment support, monitoring and self-management, and potentially transformative approaches such as, virtual reality, avatars and robots) using information communications technology (ICT) e.g. internet, personal computer, laptop, tablet, social media, mobile phone applications (‘apps’) and wearable devices, that are intended to benefit/improve people's mental health whether they are being used in conjunction with, or independently of, mental health services. We are excluding from this PSP technologies which people with lived experience can not access directly and interact with themselves, such as, electronic health records (EHRs) and other healthcare administrative systems.

For the purposes of this PSP, ‘people with lived experience’ refers to people of all ages who self-identify as having any experience of mental health problems in their personal lives (some of whom may have experience of accessing treatments and services) and people who informally care for and support friends and family with mental health problems.

For the purposes of this PSP, ‘mental health problems’ refer to all conditions described within Chapter V (Mental and Behavioural Disorders) of the 10th revision of the International Classification of Diseases and Related Health Problems (ICD-10) with the exception of i. dementia and organic brain disorders, and ii. Intellectual disability. The scope includes physical health conditions with co-morbid mental health problems. The scope excludes primary prevention or the promotion of mental well-being in the general population.

Given the number of exclusions to the scope of this PSP, we will monitor the nature and volume of responses on these issues, and if required, will adjust the scope of the PSP (by agreement of the Steering Group) and/or work with project partners to identify ways in which these questions can be incorporated into other priority setting processes.

The objectives of the Digital Technology for Mental Health PSP are to:

• work with people with lived experience of mental health problems and health and social care professionals
  (including commissioners of mental health services) to identify uncertainties about the impacts of Digital Technology
  for Mental Health
• agree by consensus a prioritised list of those uncertainties for research
• publicise the results of the PSP and process
• take the results to research commissioning bodies to be considered for funding.

The Steering Group

Digital Technology for Mental Health: Asking the right questions will be led and managed by the following people and organisations.

Lead organisation:

• NIHR MindTech Healthcare Technology Cooperative

People with lived experience representative/s:

• Debbie Butler - Service user and MindTech Involvement Team
• Thomas Kabir – Public Involvement in Research Manager, The McPin Foundation
• Paul Radin - MindTech Involvement Team
• Mat Rawsthorne – Service User Consultant, Centre of Excellence in Mood Disorders, Institute of Mental Health
• Elizabeth Rye – carer and Chair of SUCRAN, De Montfort University, Leicester

Health and social care professional representative/s:

• Chris Hollis - NIHR MindTech Healthcare Technology Cooperative/University of Nottingham Digital Health Research Priority Area/Nottinghamshire Healthcare Foundation NHS Trust
• Julie Hankin/Chris Packham – Nottinghamshire Healthcare NHS Foundation Trust
• Geraldine Strathdee/Kathy Chapman – Mental Health Leadership and Intelligence programme
• Victoria Betton – mHabitat
• Louise Knowles/Leah Bijelic – Sheffield University Counselling Service

Additional support and expertise is provided by:

• Rachel Churchill, Cochrane Common Mental Disorders Group
• Sophie Dix, MQ Transforming Mental Health
• Iris Elliott, Mental Health Foundation
• John Loder/Allison Suddaby, Nesta
• Kate Cavanagh - University of Sussex

Administrative support and coordination of activities will be provided by Lucy Simons of NIHR MindTech Healthcare Technology Cooperative.

Data management, refining questions and uncertainties will be carried out by André Tomlin of Minervation and Rachel Churchill of University of York; Katherine Easton of CATCH; Lucy Simons of MindTech.

The Partnership and the priority setting process will be supported and guided by Sandra Regan (Oct 2016 - June 2017) and Toto Anne Gronlund (June - Dec 2017) of the James Lind Alliance, who will chair the Partnership.

The Steering Group will agree the resources, including time and expertise that they will be able to contribute to each stage of the process. The JLA will advise on this.

Partners

Organisations and individuals will be invited to be involved with the PSP as partners. Partners are groups or individuals who will commit to supporting the PSP by disseminating the PSP survey and helping the PSP to gather questions and uncertainties of practical clinical importance relating to the treatment and management of the health problem in question.

Partners represent the following groups:

• people who have lived experience of mental health problems (whether directly themselves or through caring for others)
• professionals with clinical experience of mental health treatment and services
• research groups with contacts with relevant patient, carer and clinical groups.

It is important that all organisations which can reach and advocate for these groups should be invited to become involved in the PSP. The JLA Adviser will take responsibility for ensuring the various stakeholder groups are able to contribute equally to the process.

A separate list of Supporting partners will be produced that will be an active contact list, added to and expanded by the Steering Group as the project progresses. Lucy Simons will hold the master copy and be responsible for maintaining the contacts.

Exclusion criteria

Some organisations may be judged by the JLA or the Steering Group to have conflicts of interest. These may be perceived to adversely affect those organisations’ views, causing unacceptable bias. As this is likely to affect the ultimate findings of the PSP, those organisations will not be invited to participate. It is possible, however, that interested parties may participate in a purely observational capacity when the Steering Group considers it may be helpful.

The methods the PSP will use

This section describes a schedule of proposed stages through which the PSP aims to fulfil its objectives. The process is iterative and dependent on the active participation and contribution of different groups. The methods adopted in any stage will be agreed through consultation between the Steering Group members, guided by the aims and objectives of the PSP. More details can be found in the Guidebook section of the JLA website at www.jla.nihr.ac.uk where examples of the work of other JLA PSPs can also be seen.

Step 1: Identification and invitation of potential partners

Now the Steering Group has been agreed, additional potential partner organisations will be identified through a process of peer knowledge and consultation and through the Steering Group members’ networks. Potential partners will be contacted and informed of the establishment and aims of the Digital Technology for Mental Health PSP and invited to take part. The Steering Group will approve the invitation and use this to invite identified contacts in their networks to join the PSP as a partner. This work will commence at the start of the PSP and continue during Step 3: Gathering
uncertainties.

Step 2: Inaugural Steering Group meeting

The inaugural Steering Group meeting planned for 24 October 2016 has several key objectives:

• to welcome and introduce members of the Digital Technology for Mental Health PSP Steering Group
• to present the proposed plan for the PSP
• to discuss and agree the scope of the PSP and ways to maximise the responses to step 3.
• to initiate discussion, answer questions and address concerns
• to identify additional partner organisations which may commit to the PSP and identify individuals who will be those organisations’ representatives and the PSP’s principal contacts
• to establish principles upon which an open, inclusive and transparent mechanism can be based for contributing to, reporting and recording the work and progress of the PSP.

Steering Group members will be active in raising awareness of the PSP among their networks and communities, in order to secure support and participation. This can take the form of conversations, mentions at events and talks, signposting people to the project webpages and using the #digitalMHQ hashtag on social channels.

Step 3: Gathering uncertainties

The primary method for gathering uncertainties of practical clinical importance relating to the treatment and management of Digital Technology for Mental Health will be via an online survey (produced in Survey Monkey). The survey will be created through the Institute of Mental Health’s Survey Monkey account and all responses handled initially by Lucy Simons. The survey will be designed with the involvement of the Steering Group and piloted with a small number of people with lived experience and health and social care professionals to improve its design.

Steering Group members and project partners will be invited to publicise and encourage participation in the survey throughout their networks and provided with resources to support this, for example, email invitation, project leaflet, Powerpoint slide, url to short animated film on YouTube.

Steering Group members can use a printed version of the survey for handing out at the events, conferences and workshops. These will need to be collected by the Steering Group member and posted to Lucy Simons for inclusion in the database.

Two other methods will be used to gather uncertainties.

Twitter:

A designated Tweet Chat will be used to (i) explore the value for collecting questions via this method and (ii) explore the value of using a Tweet Chat to promote participation in the survey. As MindTech have not hosted a Tweet Chat before, a partner who already uses this method will be sought to host the Tweet Chat. As this is the first time a PSP has used this method, two factors are considered important for evaluating this approach. First, any questions gathered by this route will be handled separately from the main database - this will allow an assessment of how similar or different the issues
raised in the Tweet Chat are from the questions being submitted through the survey. Second, it will be important to check the extent to which it is possible to attribute any questions to people with lived experience or health and social care practitioners.

Workshops:

Steering Group members will also be invited to identify groups and people that may not be reached by the survey methods and consider options for focus groups type workshops to gather uncertainties. A facilitation guide will be developed to aid the delivery of these workshops and ensure similar methods are adopted by different Steering Group members. The printed version of the survey will be used at workshops and these questions entered into the Survey Monkey database.

A period of 12 weeks between March and June 2017 will be given to complete this exercise.

Existing sources of information about uncertainties for people with lived experience and health and social care professionals will be searched. These can include outputs from other identification processes involving patients, carers and clinicians, such as workshops, hack days, innovation labs; research recommendations in systematic reviews and clinical guidelines; protocols for systematic reviews being prepared and registers of ongoing research. The starting point for identifying sources of uncertainties and research recommendations is NHS Evidence: www.evidence.nhs.uk .

Step 4: Refining questions and uncertainties

The Steering Group will need to have agreed exactly who will be responsible for this stage – the JLA can advise on the amount of time likely to be required for its execution. The JLA will participate in this process as an observer, to ensure accountability and transparency.

The consultation process will produce “raw” unanswered questions about diagnosis and the effects of treatments. These raw questions will be assembled and categorised and refined in a series of steps, with different project members taking different roles.

(i) developing a framework for organising the “raw” questions

André Tomlin, Lucy Simons and Kat Easton will develop a draft framework based on a sub-sample of questions, using a bottom up, iterative approach (akin to an inductive qualitative analysis). This will be reviewed and amended, through discussion and consensus by the data management sub-group (Chris Hollis, Liz Rye, Mat Rawsthorne and Debbie Butler) and then by the whole Steering Group at a face to face meeting.

(ii) applying the framework to the “raw” questions
Stephanie Sampson, an independent contractor to MindTech, will be responsible for applying the framework to the whole dataset. Checks will be put in place to ensure consistency of approach. This stage will include combining of duplicate questions, splitting responses that contain multiple questions and removal of those out of scope. Questions about specific digital products or branded digital services will be re-phrased to relate to the type of technology referred to.

(iii) Developing “collated indicative questions”
Once the data are all coded into the framework, the third stage will be formulating collated indicative questions which are clear, addressable by research and understandable to all. The data management sub-group will be consulted during this process, which will be carried out by Stephanie Sampson. Lucy Simons, Kat Easton and Andre Tomlin. These collated indicative questions will be then checked against the evidence base to see the extent to which these have been answered by previous research (see next stage)

(iv) Evidence checking
Systematic reviews and guidelines will be identified and checked by André Tomlin and Rachel Churchill with Stephanie Sampson to see to what extent these refined questions have, or have not, been answered by previous research. Appendix A contains a summary of the evidence checking approach to be used in this PSP. Because of the nature of the topic area it is anticipated that we will deal with questions that are broader than treatment uncertainties - once we know what these questions are, evidence other than systematic reviews can be searched and checked, e.g. primary research. We may also need to draw on wider literature such as Ofcom reports and sociological literature for some questions, for example, those related to digital exclusion.

There are three categories of questions that may emerge from the process that are not true uncertainties and the PSP will need to decide how to deal with these. As we can not anticipate the range and extent of questions in these three categories, the approach taken to dealing with them will be decided by the Steering Group in the latter stages of the project:

(i) Unrecognised unknowns - Sometimes, uncertainties are expressed that can in fact be resolved with reference to existing research evidence. If a question about treatment effects can be answered with existing information but this is not known, it suggests that information is not being communicated effectively to those who need it.

(ii) Questions asking for information and we know the answer to them e.g. Can an app be developed that can help people with homework between therapy sessions? (NB If individuals contact us directly with specific questions, we will provide the answer to them)

(iii) Questions which are out of scope of this PSP but are still good questions, for example, questions which are tractable and unanswered. These will be captured and passed on to others who may be in a position to address them.

Uncertainties which are not adequately addressed by previous research will be collated and recorded on a template (supplied by the JLA) by André Tomlin with Lucy Simons, Kat Easton and Stephanie Sampson. This will demonstrate the checking undertaken to make sure that the uncertainties have not already been answered. This is the responsibility of the Steering Group, which will need to have agreed personnel and resources to carry this accountability. The data should be submitted to the JLA for publication on its website on completion of the priority setting exercise, taking into account
any changes made at the final workshop, in order to ensure that PSP results are publicly available.

Step 5: Prioritisation - interim and final stages

The aim of the final stage of the priority setting process is to prioritise through consensus the identified uncertainties relating to Digital Technology for Mental Health. This will be carried out by members of the Steering Group and the wider partnership that represents patients and clinicians.

• The interim stage, to proceed from a long list of uncertainties to a shorter list to be discussed at the final priority setting workshop (e.g. up to 30), may be carried out over email or online, whereby organisations consult their membership and choose and rank their top 10 most important uncertainties. There are examples of how other PSPs have achieved this at www.jla.nihr.ac.uk in the Key Documents of the Anaesthesia and Perioperative Care PSP section and the Childhood Disability PSP section.
• The final stage, to reach, for example, 10 prioritised uncertainties, is likely to be conducted in a face-to-face meeting, using group discussions and plenary sessions.
  The methods used for this prioritisation process will be determined by consultation with the partner organisations and with the advice of the JLA Adviser. Methods which have been identified as potentially useful in this process include: adapted Delphi techniques; expert panels or nominal group techniques; consensus development conference; electronic nominal group and online voting; interactive research agenda setting and focus groups.

The JLA will facilitate this process and ensure transparency, accountability and fairness. Participants will be expected to declare their interests in advance of this meeting.

Dissemination of findings and research

Findings and research

It is anticipated that the findings of the Digital Technology for Mental Health PSP will be reported to funding and research agenda setting organisations such as the NIHR and the major research funding charities. Steering Group members and partners are expected to develop the prioritised uncertainties into research questions, and to work to establish the research needs of those unanswered questions to use when approaching potential funders, or when allocating funding for research themselves, if applicable.

Publicity

As well as alerting funders, partners and Steering Group members are encouraged to publish the findings of the Digital Technology for Mental Health PSP using both internal and external communication mechanisms. The Steering Group may capture and publicise the results through descriptive reports of the process itself in Plain English. This exercise will be distinct from the production of an academic paper, which the partners are also encouraged to do. However, production of an academic paper should not take precedence over publicising of the final results.

Agreement of the Steering Group

The following people agree to follow the Digital Technology for Mental Health Priority Setting Protocol.

• Chris Hollis, Clinical Lead and Director of NIHR MindTech Healthcare Technology Cooperative
• Lucy Simons, PSP coordinator and Research Fellow, NIHR MindTech Healthcare Technology Cooperative
• Andre Tomlin, Lead Information Specialist, Minervation and The Mental Elf
• Toto Anne Gronlund, The James Lind Alliance

June 2017

Appendix A: Evidence Checking Strategy

The Steering Group meeting on 22/2/17 approved the creation of a COPE (Current Overview of Published Evidence) database for the MindTech PSP. This database of published systematic reviews, looking at digital technology for mental health, will be our first port of call when checking for uncertainties.

Creating the COPE database

The database contains 1,772 references; the results of a literature search conducted by an expert health information scientist (Sarah Dawson from the University of Bristol) that was conducted on 5 April 2017. Our search aimed to find any systematic reviews, meta-analyses or scoping reviews about digital technology and mental health, published since 1 January 2012.

For the purposes of this search, our definitions of digital technology and mental health are in line with the PSP protocol. Full search strategies are available on request.

Summary of search results -
Date of search: 5 April 2017
1. Bibliographic databases:
a. MEDLINE, n=548
b. Embase, n=806
c. PsycINFO, n=668

2. Cochrane Library:
a. CDSR, n=27
b. DARE, n=134
c. HTA, n=50
d. NHS-EED, n=82
e. Other: 2 unpublished CCMD Reviews

3. Other review databases:
a. Epistemonikos, n=106
b. Health Evidence, n=31
c. DoPHER, n=49
d. PROSPERO, n=45

Total=2,548
After de-duplication, n=1,772

Manually checking and indexing the database

The next step is to manually check through all of the records in the database to ensure that they are in scope for this PSP. Irrelevant records will be removed and the remaining references will be labelled so that the database can be used as a resource for checking uncertainties that are submitted via the survey. References will be labelled depending on the health conditions, populations and interventions that they focus on. For example, a systematic review about Internet-based cognitive behavioural therapy (iCBT) for treating clinical depression and anxiety disorders in older people will be labelled: iCBT, depression, anxiety, older people.

Mendeley group for shared access

The COPE database has been created using a web-based reference management system called Mendeley, which allows a team of people to all access and edit the database. Members of the PSP team who wish to access the database should install the Mendeley desktop app on their computers:
https://www.mendeley.com/downloads and ask André for login details: andre.tomlin@minervation.com

Developing a top-level taxonomy for digital mental health questions

As far as we are aware, there is no existing taxonomy for this subject area, so we propose creating a simple way of indexing our COPE database and final list of uncertainties using the following broad categories:
1. Mental health problems
2. Digital technologies

Searching for answers/confirming that questions are uncertainties In scope questions gathered via the survey will have to be confirmed as uncertainties through two rounds of searching:
1. Searching of the COPE database
2. If no answer is found, searching of the wider literature

It is important to note that some questions may require a broader search, e.g. questions relating to digital exclusion that are not specifically about mental health, which may be answered by evidence other than systematic reviews and guidance.

Part of the process of checking a question and classifying it as a certainty will be an assessment of the reliability of the evidence in question. For example, if we are looking at a systematic review that claims to provide an answer to a specific question, we will use the JLA criteria to confirm the reliability of the evidence (see chapter 7, stage 5 (verifying uncertainties) of the JLA guide).