Summary of data management steps

• May be completed by the Information Specialist or the PSP Coordinator.
• All survey responses to be in included in SurveyMonkey or similar survey software - add any paper responses into the survey software before downloading, for ease of monitoring demographics.
• Download into Excel spreadsheet.
• Add in any relevant uncertainties from other sources, such as discussion groups, published research recommendations or healthtalk.org.
• Format Excel according to suggested minimum fields in JLA data handling template.
• Remove blank rows or consider how to manage incomplete responses.
• Each respondent will have a unique identifier. Where one respondent has submitted more than one uncertainty, ensure these are separated so there is one submission per row, always ensuring demographic data is copied across so the origins of each submission are retained.
• Anonymise the data by removing any identifying personal details.

• Responsibility of the Information Specialist and Steering Group.
• Remove submissions that fall outside the scope of the PSP, and those that are not uncertainties. Refer to the PSP Protocol for the agreed scope.
• Ensure that those removed are kept separately, with all the associated details, e.g. number of patients, carers, clinicians who submitted the question.
• Pass the list to the Steering Group for review, confirmation of exclusion, and decision about how to handle, e.g. passing to other organisations/PSPs.

• Responsibility of the Information Specialist and Steering Group.
• Decide how to categorise the submissions. This might be an existing taxonomy, e.g. UK Clinical Research Collaboration Health Research Classification System, or Cochrane. Or, it might be done iteratively as the data is processed.
• Allocate submissions into categories, as a first step to organising the data.

• Responsibility of the Information Specialist and Steering Group.
• Submissions that fall within the scope of the PSP to be rewritten or rephrased to help clarify the precise uncertainty. Where possible, use PICO format (Patient/Population, Intervention, Comparison, Outcome). Lay language to be used. Not all submissions are suitable for PICO structure, but PSPs should aim to ensure that the uncertainties are in a format that will ultimately be of value to the research community.
• If 'Outcome' variable is difficult, the Steering Group to consider generic outcomes, e.g. efficacy, adverse effects of complications, quality of life.
• Produce a list of those eligible submissions that cannot be standardised. The full list will be circulated to the Steering Group for confirmation and discussion of any issues. Once approved, these submissions will be combined with other eligible submissions.
• Eligible submissions (now in standard format) in duplicate are combined, with their frequency recorded. Frequency is dependent on the number of times one uncertainty is submitted by a particular participant or group, e.g. patients, carers or health professionals, and the number of times it is submitted by different groups.
• Ensure all original information is combined with each set of combined questions, e.g. original survey submissions.

Note:

• Some steering groups may choose to do the formatting work after combining duplicates if the number of questions is too high, although the process of formatting submissions may help with identifying duplicates.

• Responsibility of the Information Specialist with input from Steering Group where required.
• Check the evidence base for relevant, up-to-date systematic reviews.
• Check the list of formatted questions against the evidence base to identify whether any have already been answered ('unknown knowns').
• Agree with the Steering Group how to treat any questions that may have been partially answered by research.
• If there are any 'unknown knowns', extract them and pass to Steering Group for a decision on how to handle/who to inform.
• Keep the true uncertainties, i.e. where no current evidence exists, in the Excel spreadsheet format supplied by the JLA.
• If the Steering Group has agreed this is in scope, then add to the list any further uncertainties identified from checking the existing research evidence, noting the source.
• For ease of data review, if possible, the uncertainties should be put into broad categories such as care pathway, or area of interest, e.g. sepsis, communications.
• The list of true uncertainties to be reviewed and agreed by Steering Group.

Notes:

• This stage should take place once the list has been reduced to a more manageable length, e.g. 100, prior to the interim process.
• JLA definition of uncertainty: no up-to-date, reliable systematic reviews of research evidence addressing the uncertainty about the effects of the intervention exist; or up-to-date systematic reviews of research evidence show that uncertainty exists.
• Reviews to check should include, but not be limited to, the Cochrane Database of Systematic Reviews, NICE guidelines, SIGN clinical guidelines and relevant Royal Colleges' guidelines. Other sources of evidence to be used and recorded if appropriate.

• Responsibility of the Information Specialist and the Steering Group.
• The verified uncertainties and research recommendations become the long list which will go forward to the community for interim prioritisation.
• If there are a large number of questions on the list, the Steering Group will need to do an initial prioritisation process to reduce the list to a more manageable number for interim prioritisation, e.g. no more than 70. Steering Group to be mindful of no one perspective or single agenda dominating this process.

• Responsibility of the PSP Coordinator or the Information Specialist.
• Questions will be scored and ranked (as agreed by the Steering Group), and should be tracked accordingly, again keeping all relevant details.
• The top 25/30 will go forward to the final prioritisation workshop. Question cards may show details of interim rankings from patients, carers, clinicians on the reverse to help with discussion at the workshop, and may need examples of original questions and numbers of contributors of different types if questions have been combined.

• Responsibility of the Information Specialist.
• The Top 10 will emerge from the workshop. Any changes to the questions that were agreed at the workshop should be tracked accordingly on the spreadsheet with all relevant details.

• Responsibility of the Information Specialist.
• Send the final spreadsheet of data, after any changes made at the priority setting workshop, to jla@soton.ac.uk
• Also submit completed Question Verification Form to explain the evidence checking process.
• Publication on the JLA website is an important step in making sure that researchers and research funders can find information on all of the uncertainties, what the key areas of the questions discussed at the workshop were, and what people were concerned about in their original survey submissions.
• An Excel spreadsheet template is available from the Templates and useful documents section for PSPs to use to manage their data. The completed spreadsheet will be published on the JLA website once the PSP is ready to announce its results.