Trump and the Media: A Text Analysis

He’s even gone so far as to ban several major media outlets from his campaign events, including the Huffington Post, BuzzFeed, Politico, and, most recently, the Washington Post.

Here at The DataFace, we decided to do a bit of investigation in hopes of unpacking Donald Trump’s complicated relationship with the media. Which media outlets have written about him the most? Which publications are the most anti-Trump? And how has the media's opinion of Trump shifted over time? So we gathered a dataset consisting of 19,637 articles written about the campaign between July 1, 2015 and July 31, 2016. The articles were scraped from the websites of eight media outlets - the New York Times, Wall Street Journal, Washington Post, Chicago Tribune, Fox News, Slate, Politico, and the Weekly Standard. Articles had to include reference to either Donald Trump or Hillary Clinton in the headline to be included in our sample (but not both, to avoid questions of who to attribute such articles to). Finally, we fed the actual text of the articles through a sentiment analysis package to determine how positively/negatively each article treated the candidate. But more on that later.

First, we have to acknowledge the sheer number of articles that have been written about Trump’s candidacy over the past year. MediaQuant - a Portland-based firm that analyzes media mentions across TV, print, and social media - recently pegged the value of Trump's free media coverage at $4.3 billion through the end of July. In our sample of eight major media outlets, we found that the number of articles written about Trump on a monthly basis has continued to increase over the course of the campaign. The peak of Trump’s mentions came in March 2016, when 1,686 articles across the news and opinion sections included his name in the headline. Clinton, meanwhile, hasn’t enjoyed the same level of coverage; even in her peak month, July 2016, Trump's name appeared in 1.8x the number of headlines that Clinton's did.

One potential explanation for Trump’s pervasiveness in the headlines is that conservative media outlets are repeatedly touting their own candidate. However, we found just the opposite is true. We divided our media outlets into groups of four based on conventional wisdom - one group representing the more left-leaning publications (NYT, Washington Post, Slate, and Politico) and the other more right-leaning (Wall Street Journal, Chicago Tribune, Fox News, and the Weekly Standard). When we looked at the number of opinion articles written about Trump and Clinton in each group, we found that it was actually the liberal outlets who tended to write about Trump more often than their conservative counterparts.

So are liberal media outlets actually doing Trump a favor by writing about him more? If one believes the old adage “there’s no such thing as bad publicity,” then perhaps. Several academic studies have linked bad press to increased sales and success in congressional campaigns. Yet, in both those studies, it was noted that the brands and candidates that benefited from bad press suffered from low public awareness. Typically, when awareness is higher, the effect of bad publicity is detrimental. And we can confidently say that the host of Celebrity Apprentice wasn’t lacking in brand awareness coming into the election.

So media outlets write about Trump a lot, but how do they actually view him? In order to explore this question, we employed a technique called sentiment analysis, which takes a block of text - in our case, an article in our dataset - and assigns a polarity score to it based on the nature of its language. If a high proportion of the words carry a positive meaning, the text will be scored more positively; if more of the words are negative, the text is scored more negatively¹.

We used a Python text processing library with sentiment analysis capability, called TextBlob for our analysis. The following visuals are based on the polarity values (scaled from -1 to 1) extracted from our set of articles using TextBlob. Most sentiment scores on individual articles hovered between 0 and 0.2, meaning that they were deemed relatively neutral. Given that we're analyzing a large batch of news articles, we think the observed impartiality makes sense.

The table below displays the raw sentiment scores derived from our data. We decided to include only opinion articles for this portion of the analysis, believing that each publication's true political "colors" would be most clearly expressed in the opinion section². Each outlet is listed below with the median sentiment it demonstrated towards each candidate over the past year. Remember, a more positive score means a more favorable treatment of a given candidate.

Based on this table, we can see that the news sources aligned, for the most part, according to their general leanings. The New York Times, Washington Post and Slate seemed to treat Clinton more favorably, while the Wall Street Journal, Fox News and Weekly Standard expressed more positive sentiment towards Trump. Politico stands out here for appearing to be pro-Trump, though conventional wisdom would say that it’s a very liberal publication. Our hypothesis for the discrepancy is that, during the race for the Democratic nomination, Politico might have treated Bernie Sanders favorably at Clinton’s expense. More investigation, including sentiment analysis of Politico articles mentioning Mr. Sanders, would be required to further explore this point.

We took the median sentiment values in the preceding table and subtracted Clinton’s score from Trump’s for each outlet. We then ran a statistical test³to determine if the differences in demonstrated sentiment were statistically significant. Our results are displayed below.

Four of the eight outlets in our sample displayed significantly more positive sentiment for one candidate than the other. Yet, three of those sources were pro-Hillary, whereas only Fox News was pro-Trump. That the Wall Street Journal, Chicago Tribune, and Weekly Standard are not statistically more favorable to Trump than Hillary has to be cause for concern among his supporters. We can't really say that the media is "biased" against the Donald (as his Twitter feed seems to suggest), but it does seem that Clinton has managed to win over the liberal media more definitively than Trump has conservative outlets.

Finally, we looked at sentiment in Trump opinion articles over time. Again, we bucketed the publications into “liberal” and conservative” based on convention. In the graph that follows, we plot the median sentiment of the two groups on a monthly basis.

A few interesting patterns emerge here. First, the liberal media's perception of Trump has been lowly from the start. In fact, the liberal media's lowest median sentiment score for a single month occurred in July 2015, just after Trump officially declared his candidacy. The conservative media, on the other hand, has followed a different trajectory in their treatment of Trump. They started out higher on the Donald, with more positive median sentiment scores than the liberal outlets' for the first five months of his campaign. Aside from aberrations in December 2015 and January 2016, it stayed that way until Trump became the presumptive nominee. Then, starting this past April, the conservative media's median scores have aligned closely with those of the more liberal outlets. 

And herein lies the rub for Mr. Trump - despite his repeated bashing of the New York Times and the Washington Post, it doesn't appear as though the conservative media has been treating him much more favorably than liberal publications these past few months. July 2016 did provide Trump a bit of a rebound in the court of conservative opinion, perhaps as a result of a post-convention "bounce." Still, it seems as though the conservative media has been generally laggard in their efforts to defend their nominee.

Trump now has just 84 days until his November 8th showdown with Hillary Clinton. In that length of time, we've seen polling deficits of the size that Trump now faces erased (looking at you, Michael Dukakis). And surely, Trump will have no problem continuing to garner the sort of media coverage observed in our sample. However, our sentiment analysis shows two worrisome trends for the Trump camp. First, that three of the four more conservative outlets in our sample - the Chicago Tribune, Wall Street Journal, and Weekly Standard - did not express more positive sentiment towards the Republican nominee across our sample articles. And second, that conservative sentiment seems to be more closely aligning with the liberal media's long-held distaste for their candidate. We'll see if Trump can find a way to correct these trends, if he even wants to. If we learned one thing from the primary season, it's to not count the man out.