JLA Guidebook

Chapter 5: Gathering uncertainties

Uncertainties will usually come from four sources:

  • patients/service users with experience of the health or care area or setting
  • carers
  • clinicians
  • existing guidelines and systematic reviews.

In the definition of 'patients', Priority Setting Partnerships (PSPs) may wish to include those at risk of the condition. In 'carers', PSPs usually include wider family members or other stakeholders who may have a viewpoint.

PSPs should try to gather their uncertainties from as wide a range of contributors as possible. They must ensure patients are as confident and empowered as clinicians to submit their evidence uncertainties.

Uncertainties are typically gathered via an online survey hosted on or linked to the PSP website, with paper formats available for those who are not able to contribute electronically. In some cases, people may be consulted face-to-face, for example through interviews or discussion groups, as well as via a survey. Reaching patients does not need to be costly or time-consuming. Most patient organisations can use existing communication mechanisms, including newsletters, meetings, email networks and online message boards to help communicate the survey.

Steering Group members should draw on their knowledge of the clinical area to work out who their target groups are, how to reach them, who might face barriers to involvement and how best to communicate. It is important to consider how to reach patients and carers where there is not a specific support group or charity through which a PSP might typically be able to engage people and promote the opportunity to participate (for example, the PSPs for Elbow Surgery, and Foot and Ankle Surgery).

The JLA Adviser can offer guidance on gathering uncertainties, but PSPs should draw on the knowledge and experience of their Steering Group members and use a method that is suited to their context, resources and infrastructure. While patients and carers are asked to share unanswered questions relating to their experience of a healthcare area (which could come from a problem or concern), clinicians are requested to identify uncertainties that are immediately relevant to treating a patient or delivering an intervention or service. They are asked to recall and share any issues or questions that they have encountered during discussions or consultations between patients and those caring for them.

It is worth noting that this is not the same as recalling an area where research is lacking. What the JLA process is looking for is the uncertainty during a consultation where the doctor thinks "How am I going to treat this? I'm not sure. I wish I knew the answer."

A challenge for some clinicians may be admitting to clinical doubt. It is important that respondents know that uncertainties that they raise will not be attributed to them as individuals, or their admission of uncertainty seen to reflect unfavourably on them.

Approaching the relevant Royal College for contact details of an individual or small group of people who can coordinate the dissemination of information is one way to gain access to a group of clinicians. Many Colleges have smaller committees or associated societies made up of clinicians with specific expertise. For example, for the Urinary Incontinence PSP, these included the British Society of Urogynaecology within the Royal College of Obstetricians and Gynaecologists and the British Association of Urological Surgeons within the Royal College of Surgeons.

Groups that support people with a particular health problem will often have good relationships with interested clinicians and can provide contact details if required. As clinicians are often asked to participate in consultation exercises, it can be helpful if Steering Group members are willing to make personal contact with their networks of clinicians, to encourage participation by endorsing the process.

At the planning stages, PSPs should consider how to reach out to relevant but often under-served groups who should ideally have a say in the research priorities. It is important to be clear about this at the budget planning stage as reaching out to these, particularly marginalised or vulnerable groups, may have cost implications.

Each PSP is different, but examples of groups that may be important in certain health areas could be:

  • those whose first language is not English
  • those who face cultural barriers to discussing certain health issues or research
  • children and young people (thinking about the rules to consider when involving them in surveys, Steering Groups and workshops, and the different social media that they might use to communicate)
  • anyone who might be excluded from online surveys due to lack of access or skills.

The Steering Group's knowledge of the communities it represents will be crucial in designing an effective outreach strategy.

“We adapted the process for our PSP and I do think these additions to the method helped us: For both the first survey and the interim survey we supplied paper and pencil versions to the outpatient clinics, and this resulted in a lot of response from people with lower education levels. On the online surveys the people with higher education levels were found to be overrepresented. Young children under 15 years of age were underrepresented in the first survey as well, so in the first round we added two focus groups of young children (10-13 years old) on a meeting of the patient and parent organization in November 2018. The focus groups were organized and facilitated by someone very experienced in involving children in research. This resulted in about 90 questions that we added to the others. For the interim survey, we added two prioritizing focus groups of young children (10-13 years old) on a meeting of the patient and parent organization in November 2019. Together, they formulated a Top 5. This was added to the Top 10s of patients, parents and clinicians. The Top 2 from the young childrens focus group are part of the final Top 10. In the final discussion in the priority setting workshop, the fact that the children rated one question as highest, was an important argument for it re-entering the Top 10. Casper Schoemaker, Juvenile Idiopathic Arthritis (Netherlands) PSP. This article gives more information about the PSP”

PSPs should allow time to pilot the survey, including with seldom-heard groups. JLA Advisers rely on the PSP Steering Group to identify relevant groups and to advise on how to reach them. PSPs should however keep in mind what is reasonable, relevant and practical within their clinical area.

“in retrospect we probably should have planned to spend more time with some of the seldom heard groups who wont necessarily respond to an online questionnaire. From PSP feedback survey to the JLA”
“I am conscious that peoples experience differs depending on where they live - region by region and city contrasted with rural communities. It does concern me that research is often focused around cities and universities but to truly represent the priorities of the country we need to focus on strategies to reach out as far and wide as we can. From PSP Steering Group member feedback survey to the JLA”