Involving children and young people in James Lind Alliance Priority Setting Partnerships

Background

Children and young people should be involved at every stage of research, starting with what the research is about.  In the past, paediatric research agendas were mainly developed by professionals and researchers.  Growing evidence suggests that research questions prioritized by professionals may not align with the perspectives of those experiencing the disease.  At its worst, this situation leads to research funds being allocated to studies that are not deemed important by patients, resulting in wasted resources.  This has prompted a call for collaboration with children and young people as equal partners in shaping research agendas. 

Up to this point, the involvement of children and young people in developing paediatric research agendas seems to be limited.  Only a small number of studies involved them in this process, with adults more commonly representing their views.  In 2022, Postma et al. showed that 22 studies involved children and young people in varying degrees in developing a paediatric research agenda together.  These studies were conducted in the United Kingdom, the United States of America, the Netherlands, Canada, and Australia.  The James Lind Alliance (JLA) Priority Setting Partnership (PSP) approach was the most frequently used method to involve children and young people.  Here children and young people were involved in the identification of research questions, the prioritisation of questions, the prioritisation workshop and three studies involved them in the steering group.  Most of the time, researchers adhered to the JLA guidebook without modifying the approach to the involvement of children and young people.  However, Verwoerd et al. and Grant et al. supplemented the JLA PSPs with additional focus groups to ensure the involvement of younger children.

Despite the widespread publication of outcomes resulting from co-creating paediatric research agendas with children and young people, there is minimal reflection on the methodologies employed for actually involving them, and there is a notable gap in the literature regarding their preferred methods of involvement in such processes.

This document offers a reflection on how PSPs involved children and young people in their own PSP initiatives.  Furthermore, it presents tips provided by young people themselves on effectively involving them in JLA PSPs.

Examples of how PSPs have adapted the JLA method to involve children and young people.

Mental Health in Children and Young People PSP (UK)

Our children and young people’s PSP ran from 2016-2018.  We aimed to identify the Top 10 unanswered questions about children and young people’s mental health.  We created a Young People’s Advisory Group (YPAG) at the very beginning of the project, formed of 10 young people aged 14-23 from across the UK, who had an interest in mental health research and included direct lived experience.  The YPAG were involved throughout the course of the research. They:

• Met four times a year throughout the project.
• Sat on the project Steering Group along with other key stakeholders, rotating attendance across the group of two young people per meeting and supported by a McPin team member (McPin Foundation).  Key decisions could only be made with the approval of the two YPAG members and this would be relayed back to the broader YPAG.
• Selected a name for the project: ‘Right People, Right Questions’.
• Selected the logo for the project.
• Co-produced recruitment materials for the surveys, including a video that they narrated and a social media strategy.
• Co-designed the survey, including the text for instructions, ensuring it was accessible and appealing for other young people aged 11-25 completing the survey.
• Were involved in grouping the questions into themes, and then selecting a theme to progress with on the project – ‘interventions and services’.
• Helped us condense questions further into similar questions using the Population, Intervention, Comparator, Outcome (PICO) method.
• Attended the final project prioritisation workshop along with additional young people outside of the YPAG (to provide a fresh perspective), and other key stakeholders.
• Contributed to dissemination, including presenting at houses of parliament on the final Top 10, and further conferences.

In addition, Rachel Temple was involved as a young person lived experience researcher in the project alongside another young person; they co-facilitated the YPAG and had a formal role in data analysis.  They were embedded in the process to ensure that the voices of young people had a key role in all stages of the research.

More information about the Mental Health in Children and Young People PSP can be found on the JLA website.

Children’s Cancer PSP (UK)

The focus of our PSP was on cancer and cancer-like conditions in children aged 0 to 15 at initial diagnosis (up to their 16th birthday).  We followed the well-established JLA methodology, collecting and shortlisting questions via online surveys with adult survivors of childhood cancer, carers, and professionals, and holding a final workshop.  Alongside this, a parallel process to collect and prioritise questions from children was undertaken.

A subgroup was established which included members of the PSP coordinating team and project Steering Group who wanted to be involved in the PSP's engagement with children.  The subgroup consisted of two researchers, a teacher, medical doctor, health play specialist, parent, clinical psychologist, and charity representative.

Our initial plan was to collect questions from children and young people using face-to-face methods.  We were unable to do this due to the COVID-19 pandemic.  The subgroup decided that the best way to reach children would be through their parents/carers. We wanted to help parents to explain the project and survey to their child(ren).  We thought that the use of age-appropriate animations would be a good way to do this.  We decided to make two different animations, one for younger and one for older children that would allow children and families to self-select what looked most applicable to them.  There was already an animation about the PSP process on the JLA website that was appropriate for teenagers.

Three different versions of the surveys were created, aimed at children of different ages (4-7 years, 8-12 years and 13-15 years) available on the Children's Cancer PSP page.  Children were invited to complete whichever survey version they preferred.  The surveys varied in the complexity of language used in the introduction section and questions, and the surveys for older children and teenagers contained more questions seeking demographic information.  A total of 71 respondents submitted 252 questions/topics.  Sixty-one respondents were children and young people who had experienced cancer (aged 3-21) and 10 were siblings (aged 4-19).

We grouped similar questions together into 24 summary questions.  Many children responded that their family, friends, and pets were important to them, but it was unclear what it was about these aspects that were important (several responses were one or a few words, such as ‘family’, ‘mum and dad’, ‘seeing friends’).  The subgroup decided it would be wrong to guess or presume what children meant and that further consultation with children was needed.  This discussion took place as part of the workshop with children and young people.

We held a workshop with eight children and young people aged eight to 16 years.  We discussed ‘family, friends, and pets’, to make some summary questions on these topics.  We then drew upon the methodology used by the Juvenile Idiopathic Arthritis PSP in the Netherlands.  We had seven envelopes, each containing questions on a different topic.  In total there were 31 questions - 24 summary questions from the children’s surveys, plus the seven new questions on family, friends, and pets. Each participant chose an envelope that corresponded to a topic that was important to them.  They could share envelopes if they wished.  The table was covered in red, amber, and green tablecloths.  The colours represented the importance of the question; green was most important, red was least important, with amber being medium importance.  The envelopes were opened, and the participants placed the questions on the table in the colour that reflected the importance of the question to them.   Participants looked at the questions on the table and were invited to add more questions if there was anything missing that was important to them.  The group decided together in which coloured section to place the additional questions.

Each participant was given three stickers which they could use to vote for their top three questions in the green area.  Before the voting took place, they could move any questions from the amber/red areas up if they wished, so that they could be included in the vote.  The questions were then placed in order of most to least votes and a discussion followed to agree the ‘Top 5’.  We were again guided by the Juvenile Idiopathic Arthritis PSP, who had recommended that a Top 5, rather than a Top 10 works better with children as it shortens the discussion process.

These questions were taken to the final project workshop.  This was a workshop with adults, including childhood cancer survivors, where the Top 10 priorities were decided.  At this final workshop, all participants paid attention to the perspectives of children and agreed that their voices should be heard as important stakeholders in this priority setting agenda.  This resulted in all five of the top priorities identified by children being reflected in the final Top 10.

More information about the Children's Cancer PSP can be found on the JLA website.

Juvenile Idiopathic Arthritis PSP (Netherlands)

The Dutch Juvenile Idiopathic Arthritis (JIA) PSP was initiated by two Dutch patient associations for JIA, the Dutch JIA patient and parent organisation (JVN) and the organization for young people (aged 16 - 30) living with rheumatic diseases Youth-R-Well.com (YRW), and two Dutch health care professional organisations.  The Steering Group (N=14) consisted of four young adult JIA patients (aged 18-25), three parents of JIA patients, and seven professionals.  The Steering Group chose to keep the scope broad to include questions regarding prevention, aetiology, symptoms, diagnosis, treatment, prognosis, health services, self-management and psychosocial aspects.  The PSP was not confined to children living with JIA (0-18 years) and included JIA patients of all age ranges.  The Steering Group explicitly made the commitment to include carers and patients with a range of literacy skills to be truly inclusive.

In PSPs, uncertainties are typically gathered via an online survey hosted on or linked to the PSP website.  Unfortunately, children up to 13 years might be excluded from these online surveys due to lack of access or skills.  In our online survey no more than 23% of responding patients were younger than 16 years, and only 16% attended primary school.  Some of these children were assisted by their parents, and therefore may have been influenced by what adults see as worthwhile or important.  To hear the voices of children, two focus groups and six small (group) interviews were held with children in an exploratory, creative and conversation-like manner, guided by an expert in child participation.  The two focus group discussions were held at the annual patient information day of the Dutch Juvenile Arthritis Association (November 2018).  Later, additional interviews were conducted at two paediatric rheumatology centers (March 2019) to involve more children and younger children, and to ensure engagement of non-members of a patient organization.  A cartoon was used to communicate the aims of the project.  We explained that researchers need to know what it means to live with JIA, what challenges children experience and what research they think needs to be done to improve their (everyday) lives, in order to conduct research that is relevant to them.  During both the focus group discussions and interviews, we first took a moment for the children to familiarize with the researchers and each other, with the help of a warm-up activity.  To ensure that children are not alienated from their own experiences and lives in the process, we used images that represent different domains of their lives as a starter, e.g. home, school, hospital and free time.  In total 24 children, participated.  Together, they formulated 81 uncertainties that were added to the over 500 uncertainties from the online survey.

The Steering Group formulated 53 summary questions.  This number could be hard for children to manage.  Moreover, some of these summary questions were rather technical questions.  Therefore, we selected the summary questions based on the input of children in the first phase; 23 of these 53 summary questions were based on at least one of the 81 uncertainties the children had brought up in the focus group discussions and interviews.

Furthermore, the small number of children who had participated in the online or hardcopy survey also delivered uncertainties which had been categorized under five extra summary questions.  This led to a total of 28 summary questions (53% of all summary questions) based on concerns raised directly by children.  These 28 summary questions were rewritten in less abstract and more child friendly language and clustered into 12 themes.

Two focus groups, and a subsequent group priority setting discussion, were held at the annual patient information day of the patient organization (November 2019).  The 12 themes were written on envelopes and presented in two separate meetings to the children.  First, each child shared in the group what their personal preferred theme was and why.  The children were then able to open envelopes that contained the corresponding summary questions to help them to narrow down their preferred research questions.  They were then invited to jointly discuss and decide the Top 5 themes of the whole group, leading to a final Top 5 of research questions for each group.  Next, the two groups compared their respective Top 5 and then, through facilitated discussion, formulated a final list in order of preference.  Briefly after the focus group discussions, the children physically handed this final list in a golden envelope to the PSP Steering Group.  Some children found it hard to let go of their personal priority list, which was at times different from other children's priorities during the same session.  It therefore needed sensitive guidance, despite the limited time and the children's impatience, to come to an agreement.  The fact that we had chosen to use a a Top 5, instead of the JLA's usual Top 10 turned out to be successful. A Top 10 would have required more discussion time.

The advantages of interviews and focus group discussions were multiple.  First, parents were not involved in the discussion, whereas for the survey, they were likely to be supervising the children's participation.  Second, through the discussions, the children had the opportunity to learn more about the broad field of research.  Third, in the focus group discussions, they could reflect together with peers, which strengthened their confidence in articulating their priorities.  Fourth, both methods were deliberately designed to start from their lifeworld - ensuring genuine participation to share their experiences.  Finally, carefully listening to them provides context for their prioritised topics.

Afterwards, most of the children were pleasantly surprised that they were asked to think about what should be researched for children with JIA.  It made them feel important and recognised as serious stakeholders.  Although some spontaneously shared some first thoughts, most needed time to get acquainted with the purpose of the consultation.

Creating an informal and relaxed atmosphere, starting from their own experiences and lifeworld, the supporting tools (the cartoon and the template) and the interaction with the experienced facilitators and their peers, helped them to get started.

The Steering Group added this Top 5 to the Top 10s of patients, carers and clinicians as input for the final priority setting workshop.  Three questions in the children’s Top 5 – numbers 3, 4 and 5 - were ranked much lower by the other three groups.  This illustrates the added value of engaging children themselves.

The children were not invited to the workshop, as the Steering Group at that time felt they would not have an equal say there.  We hoped the young adult patients would advocate for them.  And they did. One question of the final Top 10 made it into the list because a young adult patient advocated for it during the final discussion, using the argument that it was the children’s top priority.

More information about the Juvenile Idiopathic Arthritis (Netherlands) PSP can be found on the JLA website.

The picture above shows children during the interim priority setting for the Dutch Juvenile Idiopathic Arthritis (JIA) PSP.  They are viewing the envelopes with the main themes and writing down their personal preferences.

Pediatric Inflammatory Bowel Disease PSP (Netherlands)

The Dutch PSP for children and young adults with inflammatory bowel disease (IBD) was established in cooperation with the patient association Crohn & Colitis Netherlands (CCNL), the Dutch working group Kids with Crohn's and Colitis (KiCC), and the JLA.  In April 2021, a Steering Group was set up, including four children and young adults with IBD, four parents, and eight healthcare professionals, who all had an equal voice.  They agreed on a broad scope of research topics, including the cause, diagnosis, symptoms, treatment, prognosis, self-management, and the influence on daily life regarding IBD in children and young adults (age up to 21 years).

Research uncertainties were gathered through online and in-person surveys.  We decided to create three different surveys for children and young adults of different ages: up to 12 years, 13-16 years, and 17-20 years available on the Pediatric Inflammatory Bowel Disease (Netherlands) PSP page.  The children and young adults on the Steering Group were involved from the beginning in designing these surveys.  They adapted the language of the introduction text and questions for each age group.

Additionally, they selected graphics that could be used in the different surveys.  As promotion material for filling out the surveys, the children and young adults also co-designed flyers that were distributed online and in the participating hospitals.  To ensure inclusivity for children and young adults of all age groups, we added demographic questions to the surveys, asking for participants’ age and educational status.  Of the 222 respondents who filled out the survey, 74 were children and young adults: 13 children up to 12 years, 23 children aged 13- 16 years, and 38 young adults aged 17-20 years.  Together with the Steering Group we discussed about organising focus groups for the youngest age group. However, due to the COVID-19 pandemic, we were unable to do so.

The children and young adults of the Steering Group were subsequently involved in categorising the different themes and summarising the uncertainties into summary research questions.  To accomplish this, we divided the Steering Group members into subgroups.  In each subgroup, a child or young adult was included who specifically ensured that the summary questions were comprehensive for children and young adults of all age groups.  Furthermore, the children and young adults reviewed and adjusted the introduction text of the Shortlisting Survey in order to make this survey accessible for everyone.

We decided to bring healthcare professionals, caregivers, and children and young adults all together in the final workshop.  Ten children and young adults of different ages (12 to 19 years) who were not members of the Steering Group participated in this workshop.  We approached them via the communication channels of the patient association Crohn's & Colitis Netherlands and via the interest form included in the Shortlisting Survey.  During the final workshop, we especially paid attention to the equal voice of the participating children/young adults and created a safe and comfortable atmosphere.  We aimed to do this by starting with a first round in subgroups in which everyone could introduce themselves and share their perspectives on the most and least important research questions in small subgroups.  Furthermore, the Chairs of the small group discussions ensured that the children and young adults' perspectives were as much heard as those from healthcare professionals and caregivers.  In our experience, healthcare professionals and caregivers were truly willing to listen to the voices of children and young adults, resulting in questions ending in the Top 10 priorities that were specifically important for them.

We presented our Top 10 research priorities in-person to the Dutch Paediatric Association and to ZonMw (a major national health-care funding agency).  Two young adults (from the Steering Group and the final workshop) joined these and handed over the priorities (as shown in the image).  The shared their personal views on the final Top 10 and the process.

More information about the Pediatric Inflammatory Bowel Disease (Netherlands) PSP can be found on the JLA website.

The image above shows the Top 10 priorities being presented by one of the young adults to the Chair of ZonMw.

Young people’s views of being involved in PSPs

In developing this guidance, we sought the views of young people who had taken part in a JLA PSP. Two online workshops were held, one in the UK and in the Netherlands. Each was facilitated by two researchers (UK – Susie Aldiss and Faith Gibson; Netherlands – Laura Postma and Jasmijn Jagt).  Four young people aged 12 to 18 years participated in the UK workshop, they had all taken part in the children and young people’s workshop for the Children’s Cancer PSP.  Three young people took part in the Netherlands workshop.  They were aged 13 to 19 years and were involved in the Steering Group of the IBD PSP.  This meant they were involved in both the shortlisting of questions and the final priority setting workshops.  One young person who was unable to attend the Netherlands workshop took part in an interview.  She was aged 21 and had been involved in the final workshop of the JLA PSP about Glycogen-Storage Disease.  All young people were under 18 years at the time of participation in PSPs.

Our primary focus was on exploring the young people’s experiences of being involved in PSPs.  We asked them about their perceptions of what worked well and for their constructive insights into areas for refinement in involving children and young people in developing a research agenda, to develop ‘tips’ on how best to involve them.  The different elements of a PSP project were discussed, with 'tips' shared for involving children and young people in the Steering Group, surveys, and final workshop. These tips were collected and documented by the researchers who facilitated the workshops and sent back to the children and young people for verification. 

Listed here are the 29 tips the young people produced.  We hope these will guide future partnerships seeking to involve children and young people.  These offer guidance to teams during the planning, execution, and completion of the project.

Tips from the young people involved in the workshops in the Netherlands and the United Kingdom

Steering Group

1. Have two groups, one for adults and professionals and one for children and young people but bring them together sometimes so that they can hear each other’s views, interact and collaborate.
2. Allow time for young people and professionals to get to know each other and for professionals to explain their roles to the young people involved.

Survey

3. Make sure the survey looks interesting, use images and colour.
4. Online surveys are good for young people.  Younger children may prefer to have the option to fill in a paper survey.
5. Sometimes it is more enjoyable to express your opinion through a workshop rather than a survey.  However, a survey is quicker to fill out, so offer both options if possible.
6. Generating questions is hard but writing what is important to you is easier.  Ask participants to write what is important to them, do not ask them to send in their responses as questions.
7. Approach young people through social media, email, and their own health and care professionals.  For example, questionnaires could be distributed in the waiting room since the children and young people are already waiting there.
8. Try linking a reward to completing the survey.
9. Let the participants know what will happen to what they say in the survey, what the impact will be.

Final priority setting workshop – preparing the workshop

10. Discuss with young people what time would be suitable for them to schedule the workshop.
11. The workshop should be planned in one day instead of multiple sessions.  This is easier for the children and young people as they have to travel only once.
12. Have an activity or questions for participants to answer before coming to the session, and then at the start everyone can share something they have already prepared.  This might help everyone to feel more confident as they have something ready to share, rather than having to think about what to say when they might be feeling nervous.
13. Ensure more than one child or young person is involved.  No one needs to feel alone.  For example, being able to attend with an older sibling may be especially helpful for younger participants.
14. Involve siblings in the workshop.  The experiences of family members of someone with a health condition are also important.
15. Provide information about what will happen at the workshop, step by step, so participants know what to expect and can feel more prepared in advance for what they might like to say.

Final priority setting workshop – during the workshop

16. Play music as people arrive to help everyone feel a bit livelier and more open to sharing.
17. Allow time for everyone to get to know each other before moving on to discussing the questions as this will make it easier for people to share what they think.
18. Have a fun activity which involves moving around to start off.
19. Have colouring or a craft activity to do to occupy your hands during the discussion. 
    Sometimes people feel more confident to open up when they have something to do.
20. Ensure a person is involved that specifically monitors and advocates for the involvement of children and young people, to help make sure that they are able to express their opinions and be listened to.  Children do not necessarily need to speak a lot. If their opinions are listened to, that is what matters.
21. Reassure participants that everyone’s opinions matter, there are no right or wrong answers, and no-one will be judged.  Young people may worry more about being truthful and feel under pressure to 'get it right' and not give the ‘wrong’ responses.
22. Have all young people agree on a signal or word to use if they want to leave the room.  Have some play stuff in the room, that they can then go to if feeling nervous.
23. Begin by having everyone explain why certain questions are important to them.  This way, everyone's motivation is clear, allowing you to empathize with them and making it easier to vote.
24. Make sure the questions being discussed are easy to understand and not too long.
    It is easier to contribute if the questions are clear, and you do not have to think too much about what they mean.
25. Make sure the questions being discussed do not contain scary terms.  Sometimes professionals share a lot of information that young people might find daunting.  Try to prevent this.
26. Ensure there is time planned for breaks.  During the breaks people might get to know each other better and this might increase the collaboration.
27. Offering a reward (or lunch) as a thank you for participation makes children and young people more likely to join in.

Final priority setting workshop – after the workshop

28. Follow up with everyone after the workshop and ask if there was anything young people thought of afterwards that they would like to add to the discussions.  Sometimes participants might think of something afterwards that they wished they had been able to add.
29. Evaluating the workshop is important.  When asking for feedback after the workshop, make sure this is done soon afterwards, do not wait too long.

This guidance was produced by:

• Laura Postma - University of Groningen, Beatrix Children’s Hospital University Medical Center Groningen, Groningen, the Netherlands.
• Faith Gibson - School of Health Sciences, University of Surrey, Guildford, United Kingdom; Centre for Outcomes and Experience Research in Children's Health, Illness and Disability (ORCHID), Great Ormond Street Hospital for Children NHS Foundation Trust, London, United Kingdom.
• Jasmijn Z. Jagt - Department of Paediatric Gastroenterology, Emma Children's Hospital, Amsterdam University Medical Centre, 1105 AZ Amsterdam, the Netherlands.
• Karijn Aussems - Department of Ethics, Law and Humanities, Amsterdam UMC (Location VUmc), Amsterdam, the Netherlands.
• Helen Barrett - The James Lind Alliance, National Institute for Health and Care Research, School of Healthcare Enterprise and Innovation, University of Southampton, Southampton, United Kingdom.
• Malou Luchtenberg - University of Groningen, Beatrix Children’s Hospital University Medical Center Groningen, Groningen, the Netherlands.
• Casper G. Schoemaker - Department of Paediatric Rheumatology, Wilhelmina Children's Hospital University Medical Centre, Utrecht, the Netherlands.
• Rachel K. Temple - The McPin Foundation, London, United Kingdom.
• Susie Aldiss - School of Health Sciences, University of Surrey, Guildford, United Kingdom.

Young people based in The Netherlands

• Quinten Eppenga
• Fay Elise Harrewijn
• Annabel Loman
• Jasmijn Terlouw

Young people based in The United Kingdom

• Emily Gilbert
• Max Kyffin
• Daisy Leppard
• Florence Leppard