Audio Signals in Python

Up to now I’ve mostly analysed meta data about music, and when I have looked at the track content I’ve focused on the lyrics. Now I want to look at analysing the sound itself. In this post I will demonstrate how to extract some useful information from an audio file using Python.

Continue reading

Integrating D3.js into R Shiny

Spotify Interactive Discography Shiny App

In the past I’ve built apps with R Shiny, and I’ve also developed a few data visualisations with d3.js. Given that R Shiny is an R based Back End Server that renders a Front End in Java Script, it seemed like it would be possible to integrate a d3.js visualisation into an R Shiny App. After some quick research, it turns out that it is possible, this blog explains how to do it, and here is an example (please note this is hosted on and sometimes runs out of free hours each month)

Continue reading

The Evolution of Pop Lyrics and a tale of two LDA’s

word cloud d3

Inspired by this amazing Paper, that used audio signalling processing to analyse 30 second clips, from around 17K pop songs, to understand the evolution of Pop music over the last 50 years. I thought it would be interesting to see if something similar could be done with Pop lyrics.

The Evolution of Pop paper explains how Latent Dirichlet Allocation (LDA) was used to describe musical topics in each song. These topics were based on the chord progressions, timbre, and harmonics in the song, as derived using audio signal processing techniques. The songs were then clustered together, according to these topics, to give 13 major genres of pop music. LDA is traditionally used in text analysis. I was interested to see if I could classify the same songs into the 13 major genres according to the lyrics in the songs.

Continue reading

Spotify Related Artists App

Untitled drawing
A while back I created an R package to pull data out of the Spotify API and turn it into a d3.js visualisation. Here is the blogpost. I’ve started to teach myself Python and I’ve now re-built this process with it. The exciting part is, as it’s in Python I can use the Google App Engine to create an app that hosts the code online. That means anyone can generate a related artists visualisation. Hurray! Have a go yourself by following this link

To find out more about how its done read on…
Continue reading

The life of the 7+ song


Following on from this blog post on how to access the million song dataset I decided to do some analysis of the data. I focussed in particular on the length of songs. I performed the analysis using R and Hive and Hadoop through AWS. This first gist uses R to construct some hadoop code to import all the data into the HDFS. Once the data was in the HDFS I ran the following Hive code to mine the data. Then finally, more R code to analyse and visualise the results. For this I mainly used the ggplot2 package which is great for producing good looking graphs.

Continue reading

Plotting BPI data with Plotly

plotly image 2

Plotly is a new tool I’ve recently come across for sharing interactive graphs. Its effectively an API that allows you to pump graphs to the web using various coding languages; currently R, Python and Matlab. It automatically adds an element of interactivity to the graphs using java script. Its then easy to share those graphs with other people. Its business model works in a similar way to github and bitbucket. It’s free if you are happy to share your output with the world but you have to pay for privacy. So for bloggers and open source enthusiasts its a useful free tool.

There are plenty of instructions on the site to guide you through using it, but here’s a quick overview of using it in R from me.

Continue reading

Spotify Artists Discography

shuggie discography

Following on from my last post I’ve added a few more functions to my SpotifyAPI package.

It now includes getArtistsAlbums which takes the output from a getArtists search and finds the albums by that artist and outputs a data.frame. This can be followed up by a getAlbumsTracks which will find all the tracks from those albums and create a data.frame. Finally I’ve added visDiscography. This uses both those functions along with the get artist function to create an interactive visualisation of an artists discography.

Continue reading

Spotify Related Artists Visualisation

Otis Shuggie Related Artists

This is my first post, so I needed some data to play with. I’ve been wanting to learn more about APIs so tackling the Spotify API seemed like a great place to start. I soon came across the related artists function in the API and that gave me a great idea. What if you could map out and visualise how your favourite artists relate to each other according to Spotify. It could be a useful way to discover new similar artists. A visual recommendation engine.

Continue reading