Southwell, B., Blake, S., & Torres, A. (2005). Lessons on Focus Group Methodology from a Science Television News Project. Technical Communication. (Online), 52(2), pp. 187-193.
This paper highlights a case study about the use of focus groups in an evaluation study about science-based television news programs. The overall goal was to address knowledge gaps in science by finding out which science subjects members of the public wanted to see on these programs and how they wanted that information delivered.
The context
Discoveries and breakthroughs inside science (DBIS) syndication resource that presents science & technology news via local television channels
Purpose of the evaluation
- to inform DBIS story development, adoptions and impact
- part of a wider formative and summative evaluation – focus groups are used at the first stage of the formative process
Questions
- how do our viewers engage with the news content?
- how do our viewers react to the stories?
Use of focus groups
- No existing qualitative data on viewer reaction
- Wanted raw commentary from viewers about what they wanted or expected to see via DBIS, rather than “pigeonholing respondents into response categories” created by DBIS
Procedure
- 9 focus groups from three US cities
- Small financial reimbursement for participant time
- 1-2 hour sessions
- Participants chosen from sampled from quota of respondents who reported watching television from market research data, resulted in diverse group
- Session involved watching news sessions and discussion supported by moderator
- Notes and video tapes of sessions analysed by two researchers
- Themes, agreement with themes (visual body language such as head nodding), comments
- Researchers cross referenced results and discussed discrepancies
Lessons reported
- Use focus groups for ideas, not for testing hypothesis
- Successfully generated new ideas that DBIS staff had not considered within their own framework for science news
- Mapping the diversity of the group to the wider audience e.g stratify group in some way to enable deeper analysis (e.g education, geographic location, gender, ethnic background)
- Be aware of limitations of the data based on the qualities of people likely to have volunteered for the group – they are not formed by random sampling
- Ensure participants are equally heard – take into account more dominant characters. A technique used in this process was to use both verbal and written contributions.
- Importance of multiple groups to gain a true reflection that is not skewed
Recommendations
- Consider how your evaluation setting (particular the use of a group) relates to the real world setting (in this case, individual television viewing) and how this might change the participant feedback
- Don’t jump to conclusions based on individually strong comments, use several techniques to validate the data
- Moderator should not be sole note taker, use two researchers who then cross-reference transcripts and videotapes.
- Use both quantitative and qualitative methods for data analysis
Outcomes of the focus group
- Useful as a way of a removing the emphasis on idea generation as an internal process and getting real audience ideas
- Sampling constraints and the groups as a conversational medium limit focus groups to formative assessment of problems and ideas – not useful as summative evaluation tools
Comments (0)
You don't have permission to comment on this page.