Music and Societal Trends (Lyrics)
- isabossav
- Jan 20, 2022
- 6 min read
Updated: Feb 1, 2022
Welcome to part 3! In the previous posts of the series (part 1 and part 2), I explored the evolution of music attributes over time and the interrelatedness of music and societal developments. It turns out music is now louder, shorter, less positive and acoustic and more danceable, we're undergoing a period of "80's nostalgia", listeners seem to prefer more relatable artists, and guitar and piano (and human psychology) have exerted a big influence on the way popular songs sound. In this part, I take a deep dive into song lyrics and their evolution and themes.
Human language can be messy, repetitive, imprecise, inconsistent, and thus hard to analyze. Imagine what it takes for a machine that runs on binary (it can be difficult to conceptualize but everything we do on machines gets reduced to ones and zeroes) to understand grammar, intent, tone, sarcasm, and many of the elements that make up the human speech. This is all made more complicated by the fact that music lyrics contain so much slang, and I had to go through the songs to make sure I did a proper "clean up", switching "wanna" into "want to", "going'" into "going", "ain't" into "not", "'em" into "them" and so on and so forth so the text was consistent.
Regardless, lyrics are one of my favorite components of popular music and I am fascinated by Natural Language Processing (basically the computational analysis of "natural" or human language), so I hope you are as excited as I am for this part of the analysis.
Song Lyrics
After the clean up and pre-processing parts - making slang into "formal" language, lower-casing, tokenizing, removing stop words (i.e. articles, prepositions, etc. that don't act much information to the text like "the" or "of"), stemming (for example, bringing "walk", "walking" and "walks" into their stem "walk"), and so on, I started the study with a frequency distribution to find the most commonly used words.
Frequency Distribution
The frequency distribution analysis left me with three main conclusions:
"Love" is by far the most popular word.
The conjunction "oh" (defined by the Oxford dictionary as "used to express a range of emotions "), in third place, reflects the informal and emotional tone of popular music.
"Babi" (the stem of "baby"), used in most of the lyrics to mean an endearing form of address (as opposed to an actual baby), is also vastly popular.
I got rid of the conjunctions ("oh", "yeah", and the like) and calculated the frequency distributions for every decade. I found no significant changes over time and discovered that "love" was the most popular word in every decade except the 90s (where it was surpassed by "go").
Word Cloud
I decided to focus primarily on nouns as they are the best indicators of sentence subjects and objects, and I visualized them using one of my favorite visualizations - the word cloud.
I configured the word cloud so that the size of each word was relative to its incidence on the overall text. The overwhelming popularity of love-related words ("baby", "heart", "love") is evident.
When analyzing the different word clouds for each decade, I again found no major differences. However, some time periods, like the 1950s had unique and socially relevant results:
"Rock", both the word and the music genre, were highly popular in the 50s but have declined over time.
"Suede" and "shoes" were so unexpected that they jumped up to me almost right away. I looked them up and their background story epitomizes the very interrelatedness I am trying to study: blue suede shoes were very popular in the early 1950s, prompting an incident that later inspired Carl Perkins to write Blue Suede Shoes; when Elvis Presley (incidentally the "King of Rock and Roll") released a cover of the song, it became such a hit that it inspired the fashion revival of blue suede shoes.
"Sue" appeared only on the 50s word cloud, precisely coinciding with the name's peak in popularity among young girls.
TF-IDF
Term Frequency - Inverse Document Frequency (more commonly known as TF-IDF) is a technique used to calculate the importance of a word in a document within a collection or corpus. Basically, the higher the score, the higher the importance.
The five most important nouns for every decade were:
1950s: "love", "lama", "baby", "heart" and "day".
1960s: "baby", "love", "jude", "lie" and "heart".
1970s: "love", "baby", "chance", "time" and "way".
1980s: "love", "time", "heart", "baby" and "day".
1990s: "baby", "life", "way", "love", and "time".
2000s: "love", "baby", "time", "way" and "heart".
2010s: "baby", "love", "life", "way" and "heart".
It appears that, at least for popular music, love is really all you need: the most important word for every decade was either "love" or "baby", with "heart" joining the top 5 almost every time. Additionally, some decades' results were biased because of hit songs, for example Hey Jude by The Beatles in the 1960s and Take a Chance by ABBA in the 1970s.
Sentiment Analysis
Sentiment analysis, as its name suggests, is used to determine the "sentiment" (positivity, neutrality or negativity) of textual data.
I analyzed the "happy", "angry", "surprise" and "fear" scores of song lyrics over time and found that music is getting sadder, more fearful, angrier and more full of surprise. These results are consistent with the decrease in valence (from part 1) and the increase in minor modes (from part 2) and reveal a recurring theme: music is indeed becoming less positive. Unfortunately, I still have no idea why.
Song Lyrics From the 1950s to Now
Despite temporary fluctuations, song lyrics haven't changed that much: they remain emotional, love-centered and highly repetitive.
All You Need is Love
The most popular lyrical theme is love. In fact, in almost every decade about 75% of the songs contained the words love, heart and/or baby. This popularity is attributed to two factors: relatability and mere exposure effect.
Humans like relatable stuff (and, as we saw in part 2, social media has sparked a growing appreciation for relatability). Love is something every single one of us has experienced, in one way or another, and therefore love topics are as relatable as it gets.
The mere exposure effect, sometimes called the familiarity effect, is a psychological phenomenon by which humans tend to develop a preference for things or people that are more familiar to them. This tendency comes naturally to us and pairing it with our penchant for relatability can explain why musicians and listeners alike are drawn towards love songs.
Repeat On and On
Popular Western music is very repetitive. We see it in its tonality (songs reusing the same scales over and over) and we see it in its lyrics (the most popular words used haven't really changed much over time). Moreover, there is also word repetition within songs, and it is increasing: in the 2010s, on average, only 40% of the nouns within a track were unique, compared to 48% in the 1950s:
The explanation behind this lies in human psychology. Researchers at the Music Cognition Lab at the University of Arkansas found that humans actually prefer music pieces with repetition, and another study by the University of Southern California discovered a strong correlation between the number of times a song repeated its chorus and the popularity of that song. Elizabeth Margullis, director of the Music Cognition Lab at Princeton University, argues that musical repetition is compelling because it allows us to anticipate what is coming, thus making us feel like active participants rather than mere passive listeners.
The End
To conclude,
Popular music is getting louder, shorter, less positive, less acoustic and more danceable.
The most popular modes are the more upbeat major modes.
C major and D major dominate the tonality center.
Fans have developed a growing appreciation towards artists' reliability and authenticity.
Lyrics have not changed much over the past decades, with love and repetition being the primary themes.
Technology and human psychology are the primary driving factors behind these developments, with the former fundamentally altering track features and the communication channels between artists and the public, and the latter having a big impact on lyrics, song popularity and tonality center.
On the other hand, the music industry has not been a mere passive observer of social transformations. Instead, the entire industry - including songs, composers, singers, and the apparatus behind them, has influenced societal and cultural trends, at times reinforcing them at at others propelling change. Impacting everything from race to fashion to religion to feminism, music has exerted a tremendous power on the development of our society.
This marks the end of everything I wanted to share about my analysis. If you've made it all the way here, thank you for reading this particularly long series!
Comments