- Case study #6: Evaluating a TB awareness animation
What did you do?
We secured a £1,000 grant from the British Society for Immunology to develop an animated film about latent tuberculosis (TB) infection, which was matched by funding from the NIHR Health Protection Research Unit (HPRU) in Respiratory Infections. So together with my colleagues, Mica Tolosa-Wright and Ishita Marwah, we developed the script and initial version of the film over a period of about 3 months (Feb–April 2017).
The majority of new TB cases in the UK are a result of reactivation from latent infection, so the main aim of the film was to explain the immunological basis of reactivation, in order to empower viewers to make informed decisions relating to latent TB screening and treatment, and help reduce disease burden as a result.
We were able to produce a high-quality film and screen it to a large audience at the Imperial Festival in May 2017, where we received positive feedback. However, we still lacked specific feedback about how viewers understood latent TB, testing and treatment options, and how the film could be improved to make the message clearer for our target audience. To address these gaps in our evaluation, we decided to carry out a screening event combined with pre- and post- event questionnaires with a focus group made up of patient and public representatives.
What were you trying to achieve?
From the focus groups, we aimed to get detailed feedback about how effectively the film communicates messages about TB to the general public. Specifically, we wanted to assess whether the film was able to:
- Educate the viewer about what latent TB infection is, and how it is different to TB disease
- Inform the viewer that there is a free test for latent TB, treatment is available for individuals who test positive and that more information can be obtained through GPs
- Clarify to the viewer who the target population for latent TB testing is (i.e. those who would be offered the free test)
By obtaining this detailed feedback, we hoped to identify areas for improvement, which could be addressed in the next stage of film development.
Who did you involve and how did you find the right people?
For the Imperial Festival Event, we had a small stall within one of the event rooms (a hall) where hundreds of members of the public, mostly families, were circulating and visiting multiple stands. We invited passers-by to watch the animation, and then fill in a brief questionnaire after their viewing.
For the focus group, the individuals involved were part of our HPRU in Respiratory Infections Patient and Public Panel – a panel of seven members of the public are already actively involved in research within the HPRU (see ‘Case study 5: Establishing a public panel in research’ for more). Five of the seven panel members participated in the screening event, completed questionnaires and contributed to the focus group discussions.
However, while both events gave us invaluable feedback and informed the development process of the animation, the individuals involved didn’t meet the profile of our target audience, i.e. individuals from countries of high TB incidence who have recently moved to the UK, so that was something to work on going forward.
Did you achieve what you set out to do?
The focus group was very useful for us because we were able to identify areas of weakness in the animated film, which we had not fully identified previously. One of the most important things we learnt was that the film didn’t make it clear enough that only some high-risk individuals would be considered at risk of having latent TB, and therefore be eligible for testing.
One of the panel members shared her experience of how her GP had refused to give her a TB screening test when she requested one. We clarified that this was likely to be because she wasn’t in a high-risk population or hadn’t had direct contact with someone with TB. And that, for example, being on public transport with someone doesn’t qualify as direct contact. As a result of this though, the panel suggested that we should clarify who is eligible for testing and who is not.
The panel made additional comments and recommendations. For example, they suggested that not everyone would recognise the countries on the map that appears at the end of the animation to denote countries which were high risk for TB, and that there should also be link to a list of the countries affected. We also discussed the stigma associated with having tuberculosis infection, and the barrier to accessing health services that it results in. One of the panel members initially thought we should address the issue directly in the animation by using the term ‘stigma’ and informing people that this needn’t be a barrier. However, after some discussion amongst the group, they decided against this suggestion.
What impact did it have on the people involved?
Impact on the researchers: We found it useful because some of the reactions to the film and topic were unexpected. Sometimes, we think we can predict how the public understand a specific topic. In particular, holding the focus group helped us realise that our understanding of the publics’ conception of infections and disease is actually quite limited.
Impact on the PPI panel members: The members reported gaining new knowledge and confidence about the requirements for latent TB screening as a result of the animation, and their direct conversations with the researchers.
They also seemed to enjoy their involvement in the focus group:
“What I liked most about the meeting is the interaction with colleagues and research staff who obviously value our input.”
“Partnership working is the way forward.”
What was the most challenging part of doing PPI and how did you overcome it?
The most challenging part was keeping the focus of discussions on the aims of our project. In the focus group, the panel members had many things to say about the topic, which was interesting and enlightening. At future public involvement events, we will make sure that the aims of the task and research are really clear from the beginning, so that we can better manage peoples’ expectations and understanding of their involvement.
What advice would you give others interested in doing something like this?
Just do it! It’s a useful way to provide feedback on the effectiveness of research projects. We’d also recommend engaging with patients or members of the public at an early or mid-way stage (i.e. not at the end of a project). It benefits both sides: it helps those you recruit feel they’re truly playing a part in the development of the research, and the researchers have time to fully incorporate their comments and suggestions into the project.
So, what’s next?
Luckily, we were able to get additional funding from Imperial’s Societal Engagement SEED fund to further develop our film, as well as translate it into the eight key languages spoken by our target audience. Thanks to this, we have implemented (where possible) the panel’s suggested changes into the final version of our film, and held an event with other members of the public to evaluate the translated versions. We plan to carry out a final evaluation event with members of the public who do meet the criteria for latent TB testing, and then we plan to publish our findings. The film is now available online and we are working to screen it in GP clinics nationwide – we hope to distribute the final product to more locations by the end of this year.
If you’ve got a public involvement story you’d like to share, please get it touch.