As we explain in What is patient and public involvement, the aims of involving people in research can be thought of as making sure the right research is done and making sure research is done right. But understanding the impact of involvement (the difference it makes) has become a contentious issue for researchers and people who get involved. We asked people what they thought about the value and impact of involvement, including examples where it had made a difference, and how to improve the way we measure and record such impact.
Generally people agreed that we need good evidence of what difference it makes, but had no clear answers as to how best to measure this. Some researchers have designed tools for recording involvement, including the GRIPP Checklist (Guidance for Reporting Involvement of Patients and Public) and the PiiAF (Patient Involvement Impact Assessment Framework), which may help for the future in recording involvement at different stages of the process. Derek suggested this needed to be a collaborative process.
Kath thinks it’s important to assess the impact of PPI because it’s a big investment.
Patients who are involved and researchers need to keep a record of the impact throughout the trial. Patients who are involved and researchers need to keep a record of the impact throughout the trial.
In general, the people we spoke to were convinced that involvement has a positive impact on research. It was argued that involving patients and members of the public appropriately made research more worthwhile, more relevant to patients, and more successful, even though this might not have been evaluated.
Derek explains how patients can change the design of studies by bringing their experience to the table.
Involvement can put researchers in touch with the real world. It would be satisfying to know research made a difference to health care.
Margaret thinks involvement can have a positive impact on every part of the research process.
They gave examples of the impact of involvement on many different stages of research projects, from initial research questions and ideas, designing research studies, getting funding, and gaining approval from bodies that govern research, through to recruiting participants, data collection and analysis, sharing research findings and helping it to make a difference to health care. Richard suggested the patient perspective can help researchers ‘keep focused on whether or not the questionwill actually benefit patients in the long run’ and David Z gave an example where patient involvement led to a study being abandoned.
Rosie describes how patients can improve recruitment to research
Research can be made more relevant to patients by involving them in deciding which outcomes to measure.
David Z gives an example where patient involvement led to a decision not to conduct a study that they didn’t feel was worth pursuing.
But the changes people make to research aren’t always easy to identify. They may change the course of a discussion or attitudes to an issue by saying one thing or even just by being present.
Sometimes patients can make a positive difference just by being there.
Maggie thinks that lay people can make a difference by asking the elephant in the room question.
Lots of things could affect the impact of involvement in research, including the extent of involvement, the expertise and skills of those involved, the type and stage of research, the reason for involvement, and people’s knowledge of research. Researchers’ attitudes were also seen as important because if they see involvement as tokenistic and don’t involve people in the right way or offer them appropriate training and support, then it’s more likely to fail and show no impact. The fact that we don’t yet have much evidence about what is ‘the right way’ to involve people makes it even harder to be sure of what we are measuring and why. We need to understand what doesn’t work too.
If involvement is done well and people are treated with respect, then it will lead to improvements.
Involving people at the last minute is bound to be ineffective and unsatisfying for everyone.
Evidence for involvement needs to be more than anecdotal. We need to know what doesn’t work and a recognised scale for measuring impact.
Involvement may make a bigger impact on research conducted with human participants rather than laboratory based research. But people were divided on this. Beryl said ‘Not everything needs a regular human being to be involved’, but Charles felt that ‘there would be a role for PPI actually in every possible field’.
Some felt that isolating the impact made by individual people or comments in a discussion was challenging. Peter said, ‘It’s more difficult to see a direct result of what you’ve done the more sophisticated you get with your PPI activity’, and Carolyn pointed out that ‘if you’re working really collaboratively and you’ve got ten people sitting round a meeting table it’s often hard to pin down where particular ideas come fromIt’s sometimes hard to pin down exactly what the difference is that you’ve made’.
Simplistic measures won’t capture the full complexity of involvement and could be damaging.
Lay people are now accepted without question but the more closely involved they are the harder it is to identify what difference they’ve made.
Alan doesn’t think we can measure the impact of PPI because involvement is just a little cog in the research machine.
Being able to demonstrate impact is important. Current evidence about impact is often dismissed as anecdotal but Mary thinks practical examples are useful.
However, not everyone was certain that they had had an impact on the research. Jennifer wondered, ‘Well I’m doing all this work, are they really listening to what I’m saying here?’ Ben, who’d been involved in research for 6 months, felt it was ‘far too early’ to see the impact of his involvement, but hoped he would see it a few months down the line. Helen, Marney and Carolyn voiced a common concern about the need for feedback from researchers to be sure whether you have made any difference. (See also ‘Difficulties and barriers to involvement‘).
Helen doesn’t know if her involvement has made a difference. If people don’t get feedback on how they’ve contributed they may not feel motivated to continue.
Marney receives feedback on how she has made a difference to research.
Learning by asking questions in meetings is not easy, especially if taking part by phone. Hazel feels meetings are often not chaired well.
Further suggestions included:
It would be interesting to test the impact of involvement by trying to recruit for a study using information sheets designed with patient involvement and without patient involvement.
Involvement improves recruitment because it leads to better research, but there is still a lot to be done to understand impact.
Roger thinks that that the impact of PPI can only be measured by asking researchers.
Charles thinks that measuring impact is important but very difficult, and we need to be careful what we measure.
Making sure that research had an effect down the line on the care and treatments patients receive was important to people, not just whether it increased recruitment to a trial or the chances of a study getting funding or getting ethical approval. Stephen said by being involved he aimed to reduce ’emotional concerns for patients in the future’ and Richard said that research ‘actually needs to change clinical practice in our hospitals for the benefit of patients’. But measuring such long-term impacts would be even harder.