Back in the summer of 1983, David Bowie was enjoying a spell of incredible commercial success. His pop rock album Let's Dance had dropped in April of that year, spawning three singles that reached the top 15 in the U.S. Bowie was just one of numerous rock acts at the time wielding an enormous influence over popular music. Over 60% of the songs that reached the Billboard Hot 100 in 1983 can be categorized as some form of rock1.
Fast forward to 2016 and David Bowie is once again on the charts. After he passed away in January, four of his hit records reemerged on the Hot 100 for a brief period. This time, however, Bowie's resurgence represents a nostalgia for a bygone era in music. Only five other rock musicians have broken into the Hot 100 at all this year (through August). Rock’s popularity has fallen dramatically in America, replaced by pop, rap, and country.
Rock's fall from grace is the impetus for this project on the evolution of music. We aim to look at the trends of music genres that have broken into the mainstream - the years that genres like rock or rap or country spent climbing onto America's radar, the golden era of each genre when it was adopted and enjoyed by the masses, and the decline older genres suffered as they fell out of favor. In this way, we hope to capture the "lifecycle" of major genres in American popular music.
Each week since 1958, Billboard magazine has released its Hot 100 chart - a list of the 100 most popular songs in the U.S. according to airplay, streams, and physical sales. We've paired each of the 27,000+ songs that have appeared on the Hot 100 with an appropriate genre. To determine a genre for each song, we leaned heavily on the Spotify API, with supplemental data from EveryNoise.com, AllMusic.com, and Wikipedia for songs missing from the streaming service. We think that this is the most complete dataset of Hot 100 songs matched to their respective genres publicly available.
The Evolution of Music Genres Over Six Decades
Here's a look at every single song that has reached the Hot 100, stretching back to 1958. Each tiny rectangle denotes a different song. The larger the rectangle, the longer the song stayed on the Hot 100. Click on one of the rectangles to pull up the song in the Spotify player (if it's on Spotify that is).
A few notes here:
- We grouped the 27,000+ songs into 16 major genres. Six of the genres - rock, pop, soul, country, R&B, and rap/hip-hop - account for over 77% of the tracks represented.
- "Rock & roll" is separated from traditional "rock" in our genre schema. We felt that the two were disparate enough to warrant their own categories.
- Pop music is treated similarly. We've designated the swing music of artists like Frank Sinatra and Dean Martin - often described as "pop" in the late '50s and early '60s - as "pop standards." The pop music more familiar to today's listeners just falls under "pop."
- We took a different approach with "soul" music. Our definition of soul encompasses several sub-genres that could arguably have their own buckets like funk, Motown, and disco. However, we found substantial enough overlap between these genres in the data from Spotify to persuade us to lump them together.
When was your Favorite Music Genre Most Popular?
Once each song was categorized, we looked at how a genre's representation on the Hot 100 chart changed over time. To do so, we calculated the total number of Hot 100 spots occupied by each of our 16 major genres in a given year. Consider just 2016 for a second. Our dataset stretches through August 13, meaning that we have 33 Hot 100 charts for this year. Each chart has 100 spots. Thus, there have been 3,300 spots up for grabs so far in 2016. So what percentage of those 3,300 were nabbed by artists from each genre?
Performing the same calculation for each year since 1958, we produced this chart:
There's obviously a lot to digest here. You can toggle genres on and off the graph by adjusting the "Filter Genres" option in the upper left corner.
Given the trajectories that various genres have followed, it's clear that there's no "one-size-fits-all" genre lifecycle. Here's a brief look at how a few music genres have evolved:
- Pop standards seem to have been the standard in the late '50s. The genre provided the largest percentage of Hot 100 entries for the first three years of the chart's existence. Given that we only have data stretching back to 1958, we probably haven't even captured the genre's peak years.
- Rock and soul were the most popular music genres from the mid-60s to mid-70s. But as soul peaked in 1974 and slowly began to fade, rock continued to climb. Its run from 1982-86, when rock musicians occupied nearly 60% of available Hot 100 spots, is by far the most dominant stretch for any one genre.
- Despite all the attention paid to boy bands in the late '90s, it seems like R&B had no problem flourishing. Acts like Boyz II Men and Janet Jackson propelled the genre's popularity and ingratiated it with the masses.
- Country has had a tumultuous ride in the history of popular American music. It enjoyed middling popularity through the mid-'80s, when it all but dropped off the charts. Since 1999, however, it's seen a noticeable resurgence.
- Music trends have swung in favor of pop and hip-hop in the 2010s. Pop has owned the largest share of Billboard spots dating back to 2006, but has seen its popularity decline slightly since 2011. Meanwhile, rap has come on strong in the last two years. In fact, rap is on pace to occupy more than 30% of Hot 100 spots this year, higher than its previous peak in 2004.
A Look at Music Trends in 2016
So if 2016 really is the year of rap, then which rappers are driving a cultural shift towards the genre and away from pop and country? First and foremost, there's Drake. The Toronto MC has 24 Hot 100 songs to his name in 2016 alone, not even counting an array of guest features that have landed on the charts too. Kanye West, Fetty Wap, and Future are other top contributors from rap with upwards of five solo Hot 100 hits apiece this year. And it's not just established acts getting in on the fun; there are fourteen hip-hop artists with Hot 100 hits in 2016 that have never charted before.
To explore the data by year, we've created this interactive bubble chart. In its preset format, each bubble represents a song that has charted thus far in 2016. You can toggle the year parameter to change the year shown. Again, click on a bubble to pull up the corresponding song in the Spotify player.
A final point: 2016 has provided a little bit of music nostalgia. Even as pop, rap, and country have collectively occupied 86% of the Hot 100 spots this year, there have been flashes of music from bygone eras. David Bowie has had four songs touch the Hot 100 in 2016, more than any other active rock band can claim. Then there's Prince, the Minneapolis icon who died in April, who has eight Hot 100 songs to his credit this year. That's tied for the most among everyone not named Drake or Beyonce. Though rock and soul have fallen out of the mainstream, Americans continue to hold onto artists from each genre's peak years.
Footnotes:
- Refer to the line chart for confirmation of rock's dominance in the mid '80s. ↩︎
Methodology:
The data collection process for this project was intensive. We had already scraped data from Billboard's website for this DataFace post about the hometowns of popular American musicians. So after refreshing the dataset compiled for that post in October 2015, we had every Hot 100 list dating back to 1958. All we had to do was find a way to categorize the songs into their respective genres. Surely there was a website, an API, a random spreadsheet floating around somewhere on the Internet that would have this data for free. Yet despite our best efforts, we couldn't find anything that provided genre categorization for every song in our dataset, let alone even close.
We ended up using the Spotify API as our primary source of genre data. We grabbed Spotify data about 79% of the songs in our dataset using this Python project (shout out to Allen who maintains the GitHub repository). We filled in the blanks with information from EveryNoise.com, AllMusic.com, and many Wikipedia searches.
Spotify's genre classification system provided additional challenges. The streaming service categorizes artists into over 1,300 specific, and often unheard of, music genres (anybody familiar with "zydeco"?). Moreover, a given artist may fall into as many as 23 different Spotify genres. We used a two-step process to translate Spotify's genres to our own genre definition. First, we mapped each of Spotify's genres to an "overarching" genre -- "atmospheric post rock" became rock, "deep northern soul" turned into soul. Then, for any artist that fell in multiple Spotify genres, we wrote a short Python script to "vote" on which genre to place the artist in. For instance, Spotify classifies Drake as "pop rap", "indie r&b", "alternative hip hop", and "hip hop". According to our mapping system, three of those genres fall under rap/hip-hop and one under R&B. Thus, Drake goes under rap/hip-hop.
This is by no means a perfect system. Spotify categorizes music into genres at the artist, not the song, level. In other words, for a musician like Prince - whose catalogue arguably spanned pop, R&B, and soul - we give the same genre designation to all his songs. Moreover, genres are inherently subjective. And often they bleed together as a new genre emerges and is influenced by another. Generally, we think the results are consistent with our expectations. But if you see any song/artist that seems grossly misclassified, please send us an email.
