Your location:

Bias at the Beeb: addressing some questions

    My report, “Bias at the Beeb? A Quantitative Analysis of Slant in BBC Online Reporting” is published this week. The goal of this report was to try to bring a more dispassionate approach to a debate that has become extremely heated in recent years.

    In this blog post I’ll quickly outline the results of the two main approaches I use and then respond to some of the most common questions/critiques.

    Results on Quantity of Coverage

    The first approach I use compares the quantity of coverage given to a set of think-tanks by the BBC news website with the amount of coverage these same think-tanks received in the Guardian and Telegraph newspapers. The goal is to examine whether the BBC’s coverage is “more similar”, in a statistical sense, to one newspaper or the other. The key results are as follows:

    1. The Guardian’s coverage of the think-tanks in our sample is a much stronger predictor of the BBC’s coverage of these same think tanks during the period than is the Telegraph’s coverage (formally, the Guardian citations are more highly correlated with the BBC citations than the Telegraph’s and across our regression analysis the R2 is always higher when Guardian citations are used as an explanatory variable rather than Telegraph citations).
    2. A regression analysis suggests that, if the Guardian prints 10 extra articles about a think-tank during the time period of the sample, then that think tank can expect to receive 7 additional hits on the BBC website during the same time period. In contrast, 10 additional articles in the Telegraph translates to just 3 additional articles on the BBC.

    These results are marginal effects: the question they attempt to answer is “what would be the effect on the level of the BBC’s coverage of a given think tank if one of the newspapers we look at increased its coverage?” The results we find then suggest that BBC coverage is twice as sensitive to Guardian coverage as Telegraph coverage. This is not equivalent to saying that right-wing think tanks receive twice as much coverage as left-wing ones. Unfortunately the discussion of the results in my article in the Sunday Times was not sufficiently clear on this point and even more misleading descriptions of the results have appeared in one or two places since.

    3. The regression results shows that, once we control for the number of hits a think tank received in the Guardian during the time period, the number of hits it received in the Telegraph does not have any statistically significant effect on coverage by the BBC.

    The important thing to note about this first set of results is that they require very few subjective judgements on our part: we are not making assumptions about the ideologies of the think tanks or even reading the content of the BBC’s articles beyond checking that they are actually about the think-tank of interest.

    Results on Tone of Coverage

    The second set of results looks at the way the think-tanks in our sample are introduced by the BBC, through examining whether they are given “health warnings” by being labelled with an existing view on a subject (e.g. “which opposes the…”), an ideological position (right-of-centre, left-of-centre, conservative etc.) or a link with a party political figure (e.g. “founded by Margaret Thatcher”, “whose director is a former Labour cabinet minister” etc.).

    This approach requires some more in-depth reading of the BBC’s articles to find any instances of health warnings and so I restrict the sample to five right-of-centre think-tanks (the Centre for Social Justice, the Institute of Economic Affairs, the Centre for Policy Studies, Policy Exchange and the Adam Smith Institute), four left-of-centre think-tanks (the Social Market Foundation, Demos, the New Economics Foundation and the Institute of Public Policy Research) and CentreForum, which has links with the Liberal Democrats.

    1. I find that only one of think-tanks categorised as left-of-centre received a health warning more than 10% of the time (IPPR) and one actually received none at all (the SMF). In contrast, the minimum number of health warnings for the right-of-centre think-tanks was around 25% and the highest was above 60%.
    2. The results are similar when we look at how often the BBC referred to these think-tanks as “independent”. Three of the four left-of-centre think-tanks in our sample received this treatment at least once, whereas only one of the right-of-centre think-tanks did (the exception being the Centre for Social Justice, discussed below).

    Responses to Common Critiques/Questions

    General Questions

     1. “So What?”

    Firstly, I think that the results are interesting in their own right: they provide objective, quantitative evidence of slant in both the amount and tone of the BBC’s coverage of think-tanks. Secondly, the results reinforce the existing anecdotal allegations of bias at the organization and so suggest that these views should be taken more seriously.

     2.    “Isn’t the CPS even more biased than the BBC? How can we trust you?”

    It is inevitable that, if an organization like the CPS criticises the BBC, it will be accused of ulterior motives. That is why I tried to use methods with clear (and I hope) uncontroversial assumptions. By removing much of the subjective element from measuring slant, my hope is that people will read the report, assess the assumptions and then draw their own conclusions.

    Questions About the Comparison between the BBC and the Telegraph/Guardian

    1. The BBC’s coverage should be more similar to the Guardian than the Telegraph, one is a reliable news source, the other is outrageously right-wing.”

    I think that my choice of reference newspapers is reasonable. The Guardian is the most prominent quality newspaper with an openly left-of-centre editorial line and the Telegraph is the most prominent quality newspaper with an openly right-of-centre editorial line. I would also point to the fact that the Telegraph has a circulation three times that of the Guardian as evidence that it is hardly a fringe publication.

    I would also ask anyone who has the reaction “You’ve convinced me that the BBC’s coverage is more similar to the Guardian’s, but that’s how it should be” to acknowledge that this probably means that both the Guardian and the BBC are producing output that fits with their worldview. This may be okay from their point of view, but surely they should respect that that those of a different political persuasion might think differently.

    2. “You generate a ranking of think-tanks using their relative coverage in the Guardian/Telegraph. However, some of the think-tanks aren’t where I expect them to be, why should I trust your results?”

    We rank the think-tanks in our sample by the relative coverage they receive in the Guardian versus the Telegraph. We do this to check whether the resulting ranking looks plausible so as to test whether it really seems to be the case that newspapers give more coverage to sources they agree with. There are some outliers (in particular, IPPR appears rather right-wing by this measure). However, given that this ranking required no assumptions on our part about the think-tank’s ideological persuasion, we think that it does a pretty good job.

    Also, even if the ranking of think-tanks was completely mad, the results would still suggest a similarity between the BBC and the Guardian citations, which would be interesting in itself.

    Questions About Results on Health Warnings

     1. How can you equate the Social Market Foundation to some of the right-of-centre think-tanks? The SMF are far less extreme.”

    The treatment of the SMF has been the biggest source of contention so far. I would like to first point out that the classification of think-tanks as left-of-centre/right-of-centre is only necessary for our results on health warnings and has no bearing for the first set of results looking at the level of coverage a think-tank receives from the BBC.

    Even if one disagrees with the classification of the SMF, and I can see how people might, I think that the other choices about which think-tanks to classify as left-of-centre are substantially less contentious. Most would agree, for example, that the New Economics Foundation is more left-wing than Policy Exchange is right-wing. The broad picture, that right-of-centre think-tanks get more health warnings, remains the same when we exclude the SMF. The results are robust to their exclusion.

    This debate also underlines one of the points I make in my article for last week’s Sunday Times: when trying to measure the extent of bias, subjective judgments are best avoided wherever possible in favour of a more value-neutral approach.

     2.    “Maybe everyone treats right-of-centre sources in this way and so you’re singling the BBC out unfairly? How do the newspapers treat the think-tanks in your sample?”

    We took Demos and the CPS as representative left-of-centre and right-of-centre sources and did a quick check of the Guardian and Telegraph’s use of health warnings. Demos received a health warning 4% of the time in the Guardian and 20% of the time in the Telegraph. In contrast, the CPS received a health warning 71% of the time in the Guardian and 22% of the time in the Telegraph. It would be foolish to take too much from this observation, but it is fair to say that it does not support the idea that the BBC’s asymmetric treatment of think-tanks sources is something that is common to all news sources.

    3.    “The right-of-centre CSJ was referred to as “independent” more often than any other think-tank. Doesn’t this suggest that the BBC is being fair?”

    The CSJ was a tricky think-tank to deal with because the BBC nearly always referred to its foundation by Iain Duncan Smith, but then often combined this with another health warning like “right of centre”, or a reference to its “independence”. This led to introductions like “The CSJ, an independent think-tank founded by Iain Duncan Smith”. The result was that it received both more “independents” and more health warnings than anyone else. References to its independence were probably undermined somewhat by the accompanying health warning, but it is impossible to know by how much. 

    Be the first to make a comment

    Centre for Policy Studies will not publish your email address or share it with anyone.

    Please note, for security reasons we read all comments before publishing.