Convert MP3s to iPod Audio Book format (M4B)#

I had the need to convert a group of mp3 files into a format that was suitable for playing on my iPod. Of course the mp3s could be played directly on the iPod without any trouble. This is great for songs, but an audio book is significantly longer. In my case I have a 40 minute commute each way and most audio books are too long to listen to during a commute. The iPod supports m4b files which are audio book files and they remember where they were stopped so you can resume listening to it after putting the iPod to sleep or listening to your music collection. The audio book format also supports changing the play back speed so it will be read to you much faster.

Mp3 based audio books usually come in mp3 chunks (about 10MiB or so). They can be converted into an audio book manually using the following steps:

  • vbrfix - Vbrfix reads the mp3 structure and rebuilds the file including a new Xing VBR header. This is applied to all the mp3s that comprise the audio book.

  • mp3wrap - Takes a list of mp3s and wraps them into one big one. The only thing to note is that the mp3s have to have a naming convention that allows them to be sorted properly at the command line. Otherwise mp3s could be placed in the wrong position.

  • madplay streaming into faac_ - madplay is used to convert the output of mp3wrap into a wav file which is streamed into faac which creates the m4b file.

  • faac

  • aacgain - Takes the m4b file and applies a gain to it in an attempt to make it louder.

These steps can be performed manually, but it is tedious and error prone. I have written a python script that puts all of these together in an automated fashion.

The script takes a configuration file which:

  • Points to the directory containing the mp3 chunks

  • Points to a jpg or png file that represents the cover

  • Specifies an output name

  • Tag information

    • Artist

    • Year

    • Genre

    • Comment

A sample configuration file (typically named with the .cfg extension):

    [mp3]
    path=/mnt/media/iPod/unconverted/call_of_the_wild_64kb_mp3
    coverart=/mnt/media/iPod/unconverted/call_of_the_wild_64kb_mp3/cover.jpg
    outputfile=Jack London-Call of the Wild
    artist=Jack London
    title=Call of the Wild
    year=1903
    genre=AudioBook
    comment=The Call of the Wild is a novel by American  writer Jack London. The plot concerns a previously domesticated  dog named Buck, whose primordial instincts return after a series of events leads to his serving as a sled dog in the Yukon during the 19th-century Klondike Gold Rush, in which sled dogs were bought at generous prices. Published in 1903, The Call of the Wild is London's most-read book, and it is generally considered his best, the masterpiece of his so-called "early period". Because the protagonist is a dog, it is sometimes classified as a juvenile novel, suitable for children, but it is dark in tone and contains numerous scenes of cruelty and violence. London followed the book in 1906 with White Fang, a companion novel with many similar plot elements and themes as Call of the Wild, although following a mirror image plot in which a wild wolf becomes civilized by a mining expert from San Francisco named Weedon Scott.The Yeehat, a group of Alaska Natives portrayed in the novel, are a fiction of London's.

Note: Wikipedia is an excellent source of biographical material

Typically, a number of configuration files are created so audio books can be created unattended in a batch operation.

The script features:

  • Logging capabilities - successes and failures are logged. If a failure occurs in a conversion during a batch operation it is easy to track it down

  • Checks to see if all required components are available to the script. If not it prompts for the required components. It even provides an apt-get string for Ubuntu that can be used to install the required components

  • Fixes an vbr inconsistencies

  • Wraps the mp3s into one large mp3 - beware that the mp3s need to be properly named i.e. they need to be named so that when they are sorted by the operating system they are in the correct order

  • Tags the resulting m4b file with artist, comment, genre, year and cover art. Tagging the cover art is particularly nice as it shows up in the iPod

mp3tom4b.py:


    #!/usr/bin/env python
    #-*- coding:utf-8 -*-

    """
    This script will take a folder and attempt to convert the mp3s within it to m4b
    files (iPod audiobook format).

    1) The mp3s are processed using vbrfix
    2) The mp3s are joined using the mp3wrap
    2) It will encode the newly joined mp3 to m4b
    3) The wrapped mp3 will be removed

    The output file will be placed in a sub folder of the mp3 folder.

    Note: all of the mp3s to be joined as part of the conversion must be in the same
    folder and they must have a number or identifier that allows them to be sorted
    properly i.e. a proper string sort.

    Documentation:

    References:

    Dependencies:
        vbrfix - https://gna.org/projects/vbrfix
        mp3wrap - http://mp3wrap.sourceforge.net/
        madplay - http://www.underbit.com/products/mad/ - This is a decoder used to
        convert the mp3 to wave
        faac - http://www.audiocoding.com/ - convert wav file to m4b format
        aacgain - http://altosdesign.com/aacgain/


    TODO:

    License:

    The MIT License
    Copyright (c) 2010 Troy Williams

    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in
    all copies or substantial portions of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
    THE SOFTWARE.
    """

    import sys
    import os
    import subprocess
    import ConfigParser
    import logging

    #Constants

    __uuid__ = '62a5aa15-2f1f-40e8-8a01-2a5cc74f6fb6'
    __version__ = '0.6'
    __author__ = 'Troy Williams'
    __email__ = 'troy.williams@bluebill.net'
    __copyright__ = 'Copyright (c) 2010, Troy Williams'
    __date__ = '2010-04-05'
    __license__ = 'MIT'
    __maintainer__ = 'Troy Williams'
    __status__ = 'Development'

    #script Level Variables
    mainLogger = None

    def initialize_log_options():
        """
        Creates a dictionary with the proper values to pass to the logging object

        Dictionary keys:
        level - the debug level to display in the log file
        name - the name of the logger
        quiet - whether to display log messages to the screen - Default=False
        clean - deletes the log file if it exists - Default=True
        log file - the log file to use
        """

        options = {'level' : 'info',
                   'name' : 'Log Tester',
                   'quiet' : False,
                   'clean' : False,
                   'log file' : None}
        return options


    def initialize_logging(options):
        """
        Log information based upon users options

        options is a dictionary that contains the various log options - see
        initialize_log_options for details

        StackOverflow.com Attribution:
        http://stackoverflow.com/questions/616645/how-do-i-duplicate-sys-stdout-to-a-log-file-in-python/648322#648322
            User Profile: http://stackoverflow.com/users/48658/atlas1j

        Note: The initialize_logging function is only used and it has been modified
              to use a dictionary instead of optparse options class.

        Levels:
        Logger.debug()
        Logger.info()
        Logger.warning()
        Logger.error()
        Logger.exception() <- same as error except provides a stack trace
        Logger.critical()
        """

        if not options:
            raise Exception, 'No logging options set...'

        logger = logging.getLogger(options['name'])
        formatter = logging.Formatter('%(asctime)s %(levelname)s\t%(message)s')
        level = logging.__dict__.get(options['level'].upper(), logging.DEBUG)
        logger.setLevel(level)

        # Output logging information to screen
        if not options['quiet']:
            hdlr = logging.StreamHandler(sys.stderr)
            hdlr.setFormatter(formatter)
            logger.addHandler(hdlr)

        # Output logging information to file
        logfile = options['log file']
        if options['clean'] and os.path.isfile(logfile):
            os.remove(logfile)
        hdlr2 = logging.FileHandler(logfile)
        hdlr2.setFormatter(formatter)
        logger.addHandler(hdlr2)

        return logger


    def which(program):
        """
        Takes a binary file name as an argument and searches the path(s) for it. If
        found, the full path is returned. Else None is returned

        StackOverflow.com Attribution::
        http://stackoverflow.com/questions/377017/test-if-executable-exists-in-python/377028#377028
            User Profile: http://stackoverflow.com/users/20840/jay
        """

        def is_exe(fpath):
            return os.path.exists(fpath) and os.access(fpath, os.X_OK)

        fpath, fname = os.path.split(program)
        if fpath:
            if is_exe(program):
                return program
        else:
            for path in os.environ['PATH'].split(os.pathsep):
                exe_file = os.path.join(path, program)
                if is_exe(exe_file):
                    return exe_file
        return None


    def BuildAptGet(programs):
        """
        Takes the list of programs, a tupple of two values - program name and url,
        and builds an apt get string.

        returns a sudo apt-get string that a user could use to install the required
        components on Ubuntu Linux
        """
        install = []
        if programs:
            for p in programs:
                install.append(p[0])

            return 'sudo apt-get install ', ' '.join(install)


    def CheckDependencies():
        """
        Checks the current operation system to see if the dependencies are available
        and installed. An error is raised if the program doesn't exist
        """

        programs = []
        #mp3wrap - http://mp3wrap.sourceforge.net/
        programs.append(('mp3wrap', 'http://mp3wrap.sourceforge.net/'))

        #faac - http://www.audiocoding.com/ - convert wav file to m4b format
        programs.append(('faac', 'http://www.audiocoding.com/'))

        #madplay - http://www.underbit.com/products/mad/ - This is a decoder used to
        #convert the mp3 to wave
        programs.append(('madplay','http://www.underbit.com/products/mad/'))

        #vbrfix - https://gna.org/projects/vbrfix
        programs.append(('vbrfix','http://gna.org/projects/vbrfix'))

        #aacgain - http://altosdesign.com/aacgain/
        programs.append(('aacgain','http://altosdesign.com/aacgain/'))

        #loop through the programs and see if they exist. If they do not, then
        #add them to the missing list
        missing = []
        for p in programs:
            if not which(p[0]):
                missing.append(p)

        #If there are any missing programs, create a printable list
        #and raise an exception
        if missing:
            messages = []
            for p in missing:
                messages.append('%s not found! Please install see %s for details'
                                % p)
            print 'Missing files:'
            print messages
            #Build the aptget string suitable for Ubuntu
            aptGet = BuildAptGet(missing)
            print 'If using Ubuntu you can execute this line to install missing programs:'
            print aptGet

            raise Exception, 'Missing critical programs...'


    def makeDirectory(dir_path):
        """
        Takes the passed directory path and attempts to create it including all
        directories or sub-directories that do not exist on the path.
        """

        try:
            os.makedirs(dir_path)
        except OSError:
            #Check to see if the directory already exists
            if os.path.exists(dir_path):
                #It exists so ignore the exception
                pass
            else:
                #There was some other error
                raise


    def process_command_line():
        """
        From the Docs: http://docs.python.org/library/optparse.html
        """
        from optparse import OptionParser

        usage = """
                usage: %prog [options] file

                This script will take a series of mp3 files and combine them to form
                an iPod audio book (.m4b) file. It will join the mp3's using
                mp3wrap. It will then run vbrfix to correct any issues. After that
                mp3gain will be used to increase the volume of the mp3 file. Finally
                faac will  be used to convert the mp3 to m4b and tag it with the
                appropriate information.

                file - the name of the configuration file that holds the information
                about the mp3's to be converted to an audiobook. It should look
                somthing like this:
                #-------------------------------
                [mp3]
                path=/path/to/mp3s
                coverart=/path/to/mp3s/cover.jpg
                outputfile=output-audiobook
                artist=Author
                title=book title
                year=2010
                genre=AudioBook
                comment=Some comments about the book
                #-------------------------------

                where:
                path - the absolute path to the mp3s that comprise the audio book
                outputfile - the name of the final output file
                artist - the author of the book
                title - the title of the book
                year - the year the book was published
                genre - should be set to AudioBook or some appropriate genre
                coverart - the absolute path to the image used as the book cover
                """
        parser = OptionParser(usage=usage, version='%prog v' + __version__)

        options, args = parser.parse_args(args=None, values=None)

        if len(args) != 1:
            parser.error('Only one configuration file is required')
            parser.print_help()

        return options, args


    def RunCommand(command, useshell=False):
        """
        Takes the list and attempts to run it in the command shell.

        Note: all bits of the command and paramter must be a separate entry in the
        list.
        """
        if not command:
            raise Exception, 'Valid command required - fill the list please!'

        p = subprocess.Popen(command, shell=useshell)
        retval = p.wait()
        return retval


    def loadConfigParameters(path):
        """
        Takes a path to a configuration file and reads in the values stored there.

        Returns: dictionary
        """

        if not os.path.exists(path):
            raise Exception, '%s does not exist!' % path

        #Set the defaults
        configParams = {}
        configParams['path'] = None
        configParams['outputfile'] = None
        configParams['artist'] = None
        configParams['title'] = None
        configParams['album'] = None
        configParams['year'] = None
        configParams['comment'] = None
        configParams['genre'] = None
        configParams['track'] = None
        configParams['coverart'] = None

        config = ConfigParser.RawConfigParser()
        config.read(path)

        #loop through all the items in the section and assign the values to the the
        #configParams dictionary... We don't assign it as the default dictionary
        #because, the options we are interested in are defined above... This
        #appears to be case sensitive therefore we make the keys lower case
        for name, value in config.items('mp3'):
            configParams[name.lower()] = value

        return configParams


    def find_mp3s(path):
        """
        Takes the folder and returns a list of mp3s in that folder.

        Returns a sorted list of files with the full path name.
        """
        files = []
        for i in os.listdir(path):
            filename = os.path.join(path, i)
            if os.path.isfile(filename):
                basename, ext = os.path.splitext(filename)
                if ext.lower() == '.mp3':
                    files.append(filename)

        files.sort()
        return files


    def fixMP3Bitrate(mp3Path, outputdirName):
        """
        mp3Path - the path to the directory contain the mp3s that will be adjusted
        by vbrFix

        outputdirName - the name of the directory to store the fixed mp3s - will be
        a subdirectory
        """
        if not os.path.exists(mp3Path):
            raise Exception, '%s does not exist!' % mp3Path

        outputPath = os.path.join(mp3Path, outputdirName)

        #make the output directory
        makeDirectory(outputPath)

        #fix the bit rate on each and every mp3 that comprises the audio book -
        #copying the modified files to the output directory
        mp3files = find_mp3s(mp3Path)

        if not mp3files:
            raise Exception, '%s does not contain mp3s!' % mp3Path

        command = []
        for mp3 in mp3files:
            (dirName, fileName) = os.path.split(mp3)
            newpath = os.path.join(outputPath, fileName)
            command = ['vbrfix', '-allways']
            command.append('%s' % mp3)
            command.append('%s' % newpath)
            RunCommand(command)


    def pathExists(path):
        """
        takes a tupple that contains a folder path and file name and attempts
        to determine if it exists
        """

        filepath, filename = path
        fullpath = os.path.join(filepath, filename)

        return os.path.exists(fullpath)


    def wrapMP3(path):
        """
        Takes the path to a directory containing mp3s to wrap into one mp3

        returns a tupple containing the path and filename of the wrapped mp3
        """

        if not os.path.exists(path):
            raise Exception('Path does not exist!')

        filename = 'wrap'
        output = os.path.join(path, '%s.mp3' % filename)

        command = ['mp3wrap', '-v', '%s' % output]

        files = find_mp3s(path)

        if files:
            #append the files to the command list
            command = command + files
        else:
            raise Exception, 'No mp3 files to wrap!'

        RunCommand(command)

        return (path,'%s_MP3WRAP.mp3' % filename)


    def adjust_aac_gain(path):
        """
        Takes a tupple of file path and file name to an aac to adjust the gain
        using aacgain
        """

        filepath, filename = path
        fullpath = os.path.join(filepath, filename)

        if not os.path.exists(fullpath):
            raise Exception, 'Path does not exist!'

        command = ['aacgain']
        command.append('-r')
        command.append('-k')
        command.append('%s' % fullpath)

        RunCommand(command)

        return path


    def convert_m4b(path, configParams = None):
        """
        Takes a tupple representing a file path and file name of an mp3
        and attempts to convert it to an m4b file.

        It returns a tupple containing the file path and filename of the results
        """

        filepath, filename = path
        fullpath = os.path.join(filepath, filename)
        mainLogger.debug('Path to mp3 to convert to m4b = %s' % fullpath)

        if not os.path.exists(fullpath):
            raise Exception, 'Path does not exist!'

        output = 'converted.m4b'

        commandMadPlay = ['nice', '-10']
        commandMadPlay.append('madplay')
        commandMadPlay.append('-q')
        commandMadPlay.append('-o')
        commandMadPlay.append('wave:-')
        commandMadPlay.append('%s' % fullpath)

        commandfaac = ['nice', '-10']
        commandfaac.append('faac')
        commandfaac.append('-w')

        if configParams:
            if configParams['artist']:
                commandfaac.append('--artist')
                commandfaac.append('%s' % configParams['artist'])

            if configParams['title']:
                commandfaac.append('--title')
                commandfaac.append('%s' % configParams['title'])

            if configParams['album']:
                commandfaac.append('--album')
                commandfaac.append('%s' % configParams['album'])

            if configParams['year']:
                commandfaac.append('--year')
                commandfaac.append('%s' % configParams['year'])

            if configParams['comment']:
                commandfaac.append('--comment')
                commandfaac.append('%s' % configParams['comment'])

            if configParams['genre']:
                commandfaac.append('--genre')
                commandfaac.append('%s' % configParams['genre'])

            if configParams['track']:
                commandfaac.append('--track')
                commandfaac.append('%s' % configParams['track'])

            if configParams['coverart']:
                commandfaac.append('--cover-art')
                commandfaac.append('%s' % configParams['coverart'])

        commandfaac.append('-q')
        commandfaac.append('80')
        commandfaac.append('-o')
        commandfaac.append('%s' % os.path.join(filepath, output))
        commandfaac.append('-')

        mainLogger.debug('madplay cmd line = %s' % subprocess.list2cmdline(commandMadPlay))
        mainLogger.debug('faac cmd line = %s' % subprocess.list2cmdline(commandfaac))

        madplayProcess = subprocess.Popen(commandMadPlay, shell=False,
                                                          stdout=subprocess.PIPE)
        faacProcess = subprocess.Popen(commandfaac, shell=False,
                                stdin=madplayProcess.stdout, stdout=subprocess.PIPE)
        retval = faacProcess.wait()

        return (filepath, output)


    def main():
        """
        Take a number of mp3 bits that comprise an audiobook and convert it to
        an m4b file - an iPod audiobook file format
        """

        global mainLogger #make sure that other methods can use the log

        logoptions = initialize_log_options()
        #NOTE: the options can be pulled from the command line arguments
        logoptions['log file'] = os.path.join(sys.path[0], sys.argv[0] + '.log')
        #options['clean]' = True

        # Setup logger format and output locations
        mainLogger = initialize_logging(logoptions)

        #grab the command line arguments
        options, args = process_command_line()
        mainLogger.debug('len(args) = %s' % len(args))

        mainLogger.info('Loading Configuration Parameters...')
        configParams = loadConfigParameters(args[0])

        #The working folder under the mp3 path
        outputdir = 'output'

        try:
            mainLogger.info('Checking Dependencies...')
            CheckDependencies()

            mainLogger.info('Working on %s' % configParams['path'])
            mainLogger.info('Validating Configuration Parameters...')
            if not os.path.exists(configParams['path']):
                raise Exception, '%s does not exist!' % configParams['path']

            mainLogger.info('Fixing mp3 bitrate...')
            fixMP3Bitrate(configParams['path'], outputdir)

            path = os.path.join(configParams['path'], outputdir)
            mainLogger.debug('Output folder = %s' % path)
            mainLogger.info('Combining mp3s into one big one...')
            output = wrapMP3(path)

            if not pathExists(output):
                raise Exception, 'The wrapped mp3 does not exist!'

            #convert the mp3 to m4b
            mainLogger.info('Converting to audiobook...')
            output = convert_m4b(output, configParams)
            mainLogger.debug('m4b = %s/%s' % output)

            if not pathExists(output):
                raise Exception, 'conversion result does not exist!'

            #rename the output file
            source =  os.path.join(output[0], output[1])
            dest = os.path.join(output[0], '%s.m4b' % configParams['outputfile'])

            mainLogger.info('Renaming the audio book...')
            mainLogger.debug('rename %s to %s' % (source, dest))

            os.rename(source, dest)
            output = (output[0], '%s.m4b' % configParams['outputfile'])

            #adjust the gain of the audiobook
            mainLogger.info('Adjusting the gain...')
            output = adjust_aac_gain(output)

            mainLogger.info('completed %s/%s' % output)
        except Exception as inst:
            mainLogger.error(inst, ' Occured while processing ', configParams['path'])
            mainLogger.exception(inst, configParams)
            return 1

        finally:
            #Clean up the files by deleting everything in the output folder except
            #for the .m4b file
            searchFolder = os.path.join(configParams['path'], outputdir)
            files = []
            if os.path.exists(searchFolder):
                for i in os.listdir(searchFolder):
                    f = os.path.join(searchFolder, i)
                    if os.path.isfile(f):
                        ext = os.path.splitext(f)[1]
                        if ext.lower() != '.m4b':
                            files.append(f)
                [os.remove(f) for f in files]

        return 0


    if __name__ == '__main__':
        status = main()
        sys.exit(status)

Here is an example of a shell script that can be created to call the conversion script:

#!/bin/sh
#A simple shell script to call the mp3 to m4b conversion script on various cfg files
./convertMP3toM4b.py cfgs/callofthewild.cfg