by I’ve generally taken a pretty dim view of focus groups, which seem to be used mostly to detect and amplify unthinking prejudices. But, I have to admit, that probably is due at least in part to the fact that my prejudices aren’t very close to those of the median focus group participant. So, it wasn’t until I saw focus group results matching my own thoughts that I paid any attention to the question of whether the results actually meant anything. According to the SMH report, participants in two Ipsos focus groups almost uniformly saw Morrison’s response to the bushfires as “pathetic” and “lacking empathy”. In each case, eight of nine respondents gave such negative views. Ipsos also concluded there was “little confidence” Labor leader Anthony Albanese would have provided better
Topics:
John Quiggin considers the following as important: Uncategorized
This could be interesting, too:
Merijn T. Knibbe writes Christmas thoughts about counting the dead in zones of armed conflict.
Lars Pålsson Syll writes Mainstream distribution myths
Dean Baker writes Health insurance killing: Economics does have something to say
Lars Pålsson Syll writes Debunking mathematical economics
I’ve generally taken a pretty dim view of focus groups, which seem to be used mostly to detect and amplify unthinking prejudices. But, I have to admit, that probably is due at least in part to the fact that my prejudices aren’t very close to those of the median focus group participant. So, it wasn’t until I saw focus group results matching my own thoughts that I paid any attention to the question of whether the results actually meant anything.
According to the SMH report, participants in two Ipsos focus groups almost uniformly saw Morrison’s response to the bushfires as “pathetic” and “lacking empathy”. In each case, eight of nine respondents gave such negative views. Ipsos also concluded there was “little confidence” Labor leader Anthony Albanese would have provided better leadership, with descriptions like “weak” and “bland” being offered. Regular readers won’t be surprised to learn that I agree with both judgements.
But how much can we learn from an exercise involving only eighteen participants? Surprisingly, the answer is, quite a lot. Use q to denote the proportion of the population who approve of Morrison
If the population from which the group was drawn was evenly divided between approvers and disapprovers, the chance of getting results like this would be tiny. This table (look at N=18, x = 16, p 0.5) gives it at 0.000584. A classical hypothesis test would reject the null hypothesis (or, in the usual jargon, find a statistically significant effect) for any chance below 0.05. Using a more sensible, Bayesian approach, with just about any prior distribution, the updated estimate of for the distribution of q would lie entirely below 0.25 (25 per cent approval). We could do something similar for Albanese, but the exact numbers aren’t given.
These numbers are substantially worse than the approval/disapproval numbers given by Newspoll. So what is going on?
First, the focus groups were drawn from Sydney and Melbourne, where Morrison is less popular than in Australia as a whole (though that makes things look even worse for Albanese).
Second, the focus group was not a random sample, though the members were chosen to be representative of the population. With high refusal rates this is also a problem for standard polls, but maybe not as severe.
Third, for both focus groups and standard polls, the pollsters may ask questions in ways that encourage particular ways of framing the issues, and therefore encourage particular respondents. My guess is that the poll setting encourages people to answer more in line with their party preference, which limits extreme results like those in the focus group. To spell this out, Liberal voters might be critical of Morrison in a focus group discussion, but still tell a pollster they approve of his performance.
Finally, the focus group answers emerge from discussion and may therefore be subject to ‘groupthink’ effects. With an election two years away, it’s hard to know whether this is a plus or a minus. If a negative view of Morrison is widespread, people who were initially on the fence, and not closely interested in politics, are likely to be exposed to it over time.
That’s a lot of analysis for one little focus group. But it’s better than trying to read the tealeaves from my Twitter feed which is, of course, uniformly hostile to Morrison and mostly unenthused by Albo.