Sign in to follow this  
Followers 0
BINREV SPYD3R

HPR - HPR1204: My Magnatune Downloader

1 post in this topic

The Problem

I'm a fan of Magnatune (http://magnatune.com/) and have been buying music

from them for 7 or 8 years. The Magnatune website itself is good for exploring

and downloading, and interfaces for browsing and purchasing are available in

a number of players on Linux. I have direct experience of:

Amarok: allows you to browse, purchase, examine artist information and album

details.

Rhythmbox: the plugin, which used to allow browsing and purchasing, is

currently unavailable, but is apparently due to return soon.

Gnome Music Player Client: (a front-end to the Music Player Daemon, mpd)

offers a Magnatune browser plugin

Magnatune Web 2.0 player: a web-based tool which will browse, play and

download Magnatune music.

Magnatune Android player: a fairly basic browser and player for Android 2.0

and up.

The Magnatune Web 2.0 player is the best of the bunch as far as I am

concerned, particularly since it allows me to explore the music collection

whilst listening to streamed music at the same time. However, none of these

interfaces provide me with exactly what I want in terms of the download

process, so I decided to write my own.

The Plan

I currently host my music on my HP Proliant microserver, share it across the

home network, and play it with the Music Player Daemon

(http://sourceforge.net/projects/musicpd/) on my desktop system. I normally

keep the album cover image, artwork and related material in the same directory

as the album itself, and I want to be able to save all files in their

appropriate places automatically.

Magnatune provides an API which is documented at

http://download.magnatune.com/info/api, though this information is only available

to members. Data is available in several formats: XML, SQlite and MySQL.

Design

I didn't want to launch into building a full-blown application, especially

since I only needed a downloader, so I decided to create a collection of

scripts.

I decided to use the XML data organised by album. This is updated on about

a weekly or two weekly basis, and there is a signalling mechanism through

a downloadable file containing a checksum. When this changes the large data

file has changed and can be downloaded. At the time of writing I simply run

this by hand when I receive an email alert from Magnatune.

Magnatune uses an unique key made from the artist and album names which it

refers to as the SKU (Stock Keeping Unit) or albumsku. They use this

as an URL component and in XML tags. I use it to identify the stuff I download

and to keep a simple inventory.

I decided to write some basic scripts:

To download the catalogue

To extract information from the catalogue

To download an album

To unpack the downloaded items into the target directory

I wanted to learn more about manipulating XML data, so I decided to use

XSL, the Extensible Stylesheet Language. This lets you define

stylsheets for XML data, including ways of identifying XML components with

XPath and of transforming XML with XSLT.

I have included a number of links to the resources I used in the shownotes.

Repository

I have placed all of the scripts, their associated files, and HTML and PDF

README files (extended shownotes) in a gitorious repository. This can be

browsed at https://gitorious.org/magnatune-downloader or, if a copy is

required it can be obtained with the command:

git clone https://git.gitorious.org/magnatune-downloader/magnatune-downloader.git

This makes a local git repository containing a copy of all of the files in

the current directory.

Scripts

update_albums: a Bash script to download a new version of the album

catalogue, as a bzipped XML file, if it is different from the current

version. It generates a summary of the catalogue for simple searching using

XSLT.

report_albumsku: a Bash script to take a SKU code and look up the

album details in the XML file.

get_album: a Bash script to download an album, cover images and artwork.

It takes the SKU as an argument and uses it to make an URL for an XML

file which points at all of the components, and this is downloaded (with

authentication). The script then parses this file to get the necessary URLs

for downloading. I only use the OGG format but it could easily collect any

or all formats available from Magnatune. The script records the fact that

this particular SKU code has been downloaded so that it isn't

collected again in error. All downloaded files are given names beginning

with the SKU code and are stored for the installation phase.

install_download: a Perl script which unpacks the downloaded zip file to

its final destination then adds the cover images and artwork to the same

place. I used Perl because it allowed me to query the zip file to determine

the name of the directory that was going to be created.

Further Developments

I have added further scripts to this system since I created it. I have one that

synchronises the music files from my workstation to the server, and two that

give me a simple wish-list or queue functionality.

Since I have a 200GB download limit per month on my broadband contract I try

not to download music too often and avoid contention with the rest of the

family. My queueing system is used to keep a list of stuff I'd like to buy

from Magnatune, and I simply feed the top element from the queue into my

download script every week or so.

In the future I expect to be refining all of these scripts and making them

less vulnerable to errors. For example, I have found a few cases where

Magnatune's XML is not valid and this causes the xsltproc tool to fail.

I'd like to be able to recover from such errors more elegantly than I'm doing

now.

At some point I may well be tempted to consolidate all of the current

functions into a single Perl script.

Disclaimer

I have no connections to Magnatune other than being a contented customer.

Links

Magnatune: http://magnatune.com/

Magnatune API: http://magnatune.com/info/api

AsciiDoc: http://www.methods.co.nz/asciidoc/index.html

XSLT information:

XSLT Tutorial: http://linux.dd.com.au/wiki/XSLT_Tutorial

XSL Transformations: http://www.cafeconleche.org/books/bible3/chapters/ch15.html

XSLT tutorial: http://www.w3schools.com/xsl/default.asp

Go to this episode

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0