Dencoder is my distributed encoder project. It is simply a collection of existing tools combined with some python scripts for control. This page documents how to setup, configure and use the software. Dencoder is currently focused on Mac OS X but porting the clients to Linux and Windows is possible. While Erlang and RabbitMQ are stable software, the scripts I’m using to tie the whole operation are not. This is alpha software. Please review the Known Issues section below.
To use dencoder you need:
- a home network, preferably wired
- Intel Macs, and at least one (though it’s pointless overhead with just one)
- a shared file system using apple file protocol or SMB/CIFS
You will also need the following software:
- Erlang – Erlang Installer
- RabbitMQ – RabbitMQ Installer
- Dencoder Client – Dencoder Client
Erlang and RabbitMQ have been configured to fit the needs of this project, they are otherwise unchanged and are available from (erlang.org) and (rabbitmq.com) respectively.
Setup
To cut down on the amount of configuration needed create a folder somewhere on the master server and name it Encode. Inside of it create two folders, one called Pending and another called Finished. Share this folder using OS X’s File Sharing option located in System Preferences.
On each client computer connect to this new file share and verify that it is now available via Finder.
On the master, install the Erlang, RabbitMQ and Dencoder Client packages. When finished, reboot the system. This will ensure the RabbitMQ service starts properly.
On each client computer, install the Dencoder Client package. This will install the dencoder scripts, OS X services and required python modules. At the time of this writing these modules are installed: pika, simplejson and pybonjour.
Configuration
At this time, the dencoder client requires some configuration in order to find the master server. A future release will bonjour to find the master server.
To configure dencoder start terminal using spotlight to search for terminal. Once loaded, issue setDencoderHost.py <hostname> where <hostname> is the name or IP address of the master server. Repeat this step on all clients and the master. If you don’t know the IP address of your master server start terminal by searching for terminal in spotlight. Enter ifconfig and record the IP address for en0 if you use wired networking or en1 if you use wireless.
On the master only, issue setDencoderBasePath.py <path to encode folder> where <path to encode folder> is the path for the encode folder. This step is required because the master doesn’t use file sharing to find the files, they are local. The easiest thing to do is to type setDencoderBasePath.py and drag and drop the Encode folder to the terminal and press enter.
Using Dencoder
At this point Dencoder should ready for use. To encode a file drop the file into the Encode/Pending folder, right click it and pick the Encode using <preset> preset where <preset> is the preset you want to encode with. At this time the valid presets are AppleTV, AppleTV 2 and HighProfile. I’ll provide directions on how to customize this in the future. When an encode is finished it’ll be placed in the Encode/Finished directory.
If you like to hack on things
The config files for dencoder are located at /usr/local/etc/dencoder. If you haven’t used setDencoderHost.py or setDencoderBasePath.py then it is documented. The scripts themselves are at /usr/local/bin/dencoder.py and /usr/local/bin/send2dencoder.py. The launchd files are in /Library/LaunchDaemons/com.dustinrue*
It is possible to take the dencoder.py file and config file and move it to a Linux system with virtually no changes. You must manually install pika, simplejson and pybonjour using easy_install
The OS X services are created using Automator. Open /System/Library/Services/Encode* to edit the service.
On the clients you can monitor HandBrake’s progress by issuing tail -f /tmp/hb.log in a terminal window. The error log is at /tmp/hb.err
You can download the source here
Known Issues:
- Clients aren’t aware if the master goes away
- Clients aren’t aware if the shared file system isn’t mounted or is mounted and later goes away
- There is no way to view what is in the queue
- Clients immediately dump files into the finished directory, even when they’re not yet finished. The file is finished when the date stamp stops updating
- Finished encodes are not removed from Pending
- The installers always overwrite any config file modifications you’ve made
- When in doubt, restart the master and then each of the clients, ensure the shared file system is available on each client