Sample harvesting for robots (aka how to download all audio content from a website )

Some random pic on farm automation

I was feeling a bit hacky tonight and wanted to try and automate downloading all the audio samples I wanted from a particular website. Basically I couldn’t be arsed to go though all the pages and download everything from links.

Turns out it’s pretty easy on OSX, with one handy command line tool. But first things first, make sure you have homebrew installed.

One you have homebrew, open the terminal and install wget:

brew install wget

Next, point it to the website you want to pillage. If the website happens to store all the audio in one directory, then all the quicker. Otherwise, no biggie, as this command should recursively search through sub-directories too.

wget -r -np -k -A wav,WAV,aiff

The main flag we’re interested here is -A mp3,wav,aiff as this filters the filetypes we are downloading. So you could replace these for jpg or mov etc if you are after other filetypes.

Set it off, sit back and the files will be downloaded into a directory in your home folder.

Happy farming!