Hardwired..for tidy text (Last updated: 2025-04-19)

Posted on rstats, tidytext, ggplot2, music, genius, metallica, sentiment-analysis

using the genius and tidytext packages to perform sentiment and text analysis of Metallica’s song lyrics over time

Song lyric and sentiment analysis for all

A while back I did a tidy text analysis on Faith No More lyrics. I had thought about doing this with Metallica album lyrics, as they have had a long career, spanning their late teens/twenties to their 50’s.

However, I found the process of obtaining lyrics and getting them into shape for analysis too painful, so I chose a band with slightly less output.

Good news though - things have changed with the release of the genius package from Josiah Parry (@JosiahParry).

This makes getting song / album lyrics a piece of cake.

With my FNM analysis, I obtained individual tracks, organised them into folders by album, and then went through a lot of manual processing ( the site I obtained the lyrics from concatenated each line into a single string).

This package does away with all that.

Want to get all the lyrics for an album? Use the genius_album function

MOP <- genius_album(artist = "Metallica",                           album = "master of puppets",                            nested = FALSE) 

This returns a 4 column dataframe with the tracknumber, title and lyrics. Each line of the song lyric is one one row of the dataframe, in the perfect format for manipulating further with the tidytext package.

(You can of course just obtain lyrics for individual tracks, using the genius_lyrics function).

I want to say thanks to Josiah for being super-helpful in sorting out a glitch I chanced upon, and for being patient with me when I forgot to reinstall after he fixed it!

Anyway - I soon had 10 albums worth of lyrics. Most of the code is very similar to what I did for my FNM analysis, so I don’t want to get into that too much. I know that tidytext has been updated recently and I may go back and do some further work on this, but for now, I’m just going to stick up a few images and some notes.

Some of my earlier draft pics used the viridis palette, but as much as I love that (especially “Plasma”), it didn’t look right considering I was dealing with the guys who went for black text on a black cover for their 5th and most career changing album (“Metallica”).

Josiah pointed me in the direction of the ggthemr package. Armed with that, and some online colour palette sites, I cooked up a few custom palettes based on the band’s 10 studio album covers.

Where possible, I used the relevant palette, and I also created a “Combo” palettte for use where I needed more colours:

colours_Reload <- c("#080606","#484848","#9d9c9c","#f0c43e","#fba104", "#e43609","#AD0215") colours_Hardwired <- c("#C8C8C8","#61a28b","#e7cd9b","#7E4611","#a93119", "#2c2625","#000000") Combo <- define_palette( swatch = c(colours_Reload,colours_Hardwired),                          gradient = c(lower = colours_Reload[1L], upper = colours_Hardwired[7L]))  

So - following the pattern of my FNM analysis, here are a few plots & thoughts:

A brief history of Metallica

A Danish junior pro tennis player gets into heavy metal, follows Motorhead around the UK, ends up moving to the US and places an add to start a band. In the same week, a self-confessed loner, who happens to play guitar, also places an add to start a band. They hook up. The Danish kid is Lars Ulrich, drummer and mastermind behind the band. The loner is James Hetfield, who it turns out is a Riff God in waiting.

There have been others involved (notably Kirk Hammett on lead guitar and Cliff Burton (RIP), Jason Newstead and now Rob Trujillo on bass ) but Lars & James are the main 2, involved in all the songwriting.

Albums - the classic first 4

• Kill ‘Em All : The album that kickstarted a new musical movement - thrash metal was born in San Francisco. I wouldn’t choose to listen to this now, but while ‘Seek and Destroy’ is a fan favourite, I always liked ‘Hit the Lights’ as a really good opener.

• Ride The Lightning : 1 year later, they kick things up a notch, hitting on a classic formula. Fast opener. Title Track. Power Ballad at the end of side one. Instrumental. Fast closer. Some of these tunes are still getting played night after night. No band should be capable of releasing a second album this good.

• Master of Puppets : They did it again. The acoustic intro to “Battery” lulls the unwary listener into a false sense of tranquility. I like “Leper Messiah” and “Orion” the most, but I’m awkward. The title track still absolutely stuns people hearing it for the first time. The leap in musical ability, in such a short time, was ridiculous. Listening to this album for the first time is an experience.

• And Justice For All : A bit left field. Ferocious opening and closing tracks, and also “One”. Drums on 11. Bass on 1, negating Jason’s debut, following Cliff’s tragic death. For me, this hasn’t aged as well as some of the others - I wouldn’t choose to listen to it, but ‘Blackened’ and ‘Harvester of Sorrow’ are definitely among my all time favourites. Other fans will say I’m crazy, and point to ‘One’, ‘Shortest Straw’ and ‘Dyer’s Eve’, along with the title track, and say this is Metallica’s strongest outing. For me, the production is a bit grey, sterile, and I hate the horribly clicky bass drums. I loved Lars as a drummer, but his bass drum sound was poor by comparison to Slayer’s Dave Lombardo, who set a standard that is rarely matched.

The 90’s rock mega-stars

• Metallica : “Enter Sandman”, “Sad But True”, “Nothing Else Matters” - someone you know owns this album. No quibbles about the drum sound this time. They sound amazing, along with everything else. Possibly the greatest sounding rock album of all time.

• Load : Woah, they’re wearing make-up. Still, “Outlaw Torn”, “Bleeding Me” and “Wasting My Hate” are all great tracks. “Until It Sleeps” is the standout for me - it might just be the best track they ever record. I can’t ever only listen to it once. I love everything about it. Only now are some fans realising just how good this album was.

• ReLoad : A bunch of stuff that didn’t make it on to Load. Still, “Fuel” is catchy, as is ‘The Memory Remains’. My guilty pleasure is ‘Bad Seed’, which would be the pinnacle of most other rock bands careers. That riff is just filthy. ‘Fixxer’ is this album’s ‘Bleeding Me’, which is a good thing.

The difficult 7th album

• St.Anger : Wooly hats, rage, and a dodgy drum sound. And that documentary. Is there anything good about this album? Apart from “Frantic”? And the title track? Edit - yes - ‘Some Kind of Monster’ is itself, a great track. Many years later, the internet has decreed that if you’re in a deep hole, this album is the soundtrack to your life. I can’t disagree, given that the band had reached rock bottom and this was their reaction.

Back on track

• Death Magnetic : Rick Rubin on production duty so it should have sounded great. It didn’t quite get there. I was hoping for Metallica with the production sound of Slayer’s ‘South of Heaven’ or The Cult’s ‘Electric’, but it’s pretty kludgy.

My original impression was that there were some great riffs, but not necessarily great songs. I was wrong. Seeing the video for “All Nightmare Long” changed my mind and made me listen to this properly. A good album, not great, but certainly a step forward.

• Hardwired to Self Destruct : A return to form. Some great tracks (“Here Comes Revenge” my favourite). “Now That We’re Dead” is Enter Sandman’s older, cooler cousin. The riff at 3:10 in “Atlas Rise” is guaranteed to have you tapping your feet / air guitaring. During “Halo on Fire” they turn into The Cult. This is a Good Thing, honest, although most of their fans would be horrified to hear the comparison, I’m sure.

OK - all ready? Let’s “Hit The Lights”:

I’ve only heard 2 tracks from St.Anger (lucky me). Produced at their lowest point, with an “acquired taste” drum sound ( Lars Ulrich forgot to switch the snares on for one recording, and liked the sound, so kept it for the entire album) and alongside the title track , “Frantic” is the other. It’s got a pretty repetitive chorus, which is why “Tick” is the most common word overall, though I did get a surprise when I first saw this. Altogether now - “ tick tick tick tick tick tick tick tock” No? It’s actually annoyingly catchy.

Can I get a “Yeah”?

Let’s dig in to this at album level:

The first thing that I want to highlight here is I couldn’t for the life of me get the bars to go into the correct descending order, despite copying the code precisely as I have used it several times before. In the end, I had to publish and be damned, but - I KNOW! - it’s not 100% correct.

The other point to note is that “Yeah” features heavily throughout the 90’s albums (Metallica, Load & Reload). I pretty much stopped listening to the band sometime between “Metallica” and “Load” - and have to confess hadn’t picked up on this. (I did buy “Load” years later out of curiosity (and it was on sale!). It’s a good album. There’s some good stuff on there).

Term Document Frequency - finding most commonly used words, and the inverse term document frequency, finding words that are rarely used but also important:

Looking at the sentiment of the band’s output, and guess what? They’re not exactly shiny happy people,at least, not on paper:

I produced a couple of plots for the sentiment by track:

The more traditional bar plot, and also this one using geom_point instead:

I quite like this one to be honest, although I could do with a bigger monitor to fit everything in.

I also produced more of a timeline strip plot by release date.

Here’s the positive sentiment score by track over time:

Judging by this - “Master of Puppets” has the most positive net sentiment.

“St.Anger” is the most negative, followed by “All Within My Hands”. (For this plot, I multiplied the negative sentiment by -1 to force it below the axis):

“St.Anger” also scores highly for positive sentiment, leaving “All Within My Hands” as the most miserable net sentiment score.

Topic Models

As with last time, I’m going to take the easy way out and refer you to the tidytext website to explain topic models. Here they are for each album, with custom , cover specific colours.

That’s right, I found several shades of black for you.

Almost done…

Some words have “not” or other negating words immediately in front of them, so this can change the meaning / sentiment. Which words does this most commonly affect?

I looked at correlations of individual words, but it was not very informative, however, looking at the words used by album, there was quite a nice pattern emerging:

Although the correlations are weak - we can see Metallica, Load and Reload being very closely linked. At the other end, Death Magnetic, Hardwired and Kill ‘Em All are fairly closely linked (30 + years apart) and in the middle are the classics - Ride The Lightning, Master of Puppets, and also And Justice For All.

I’d expected there to be a clear difference between the newer stuff and the old stuff. I certainly didn’t expect to see “Kill ‘Em All” and “Hardwired..” so close together. However- if you watch the “making of” videos on YouTube, you can see the bands writes the guitar and drums first, and then James adds lyrics later - and for the latest album, the rest of the band were also pitching in. I suspect that the lyrics aren’t perhaps as important as the music - certainly not compared to the And Justice For All period where everything seemed to tie together more.

To wrap up, this has been a much smoother task second time round, (I wasn’t planning to do this again) and if I ever do it again, going to have to look into purrr to make things even easier.

Thanks to Josiah - cool guy, cool R package!