Wanted to quickly share some thoughts and links of software I’ve found recently and what I’ve been up to.

If you’re into home automation at all you have to check out Home Assistant. There is a bit of a learning curve initially but once you get an understanding of how to configure it you’ll find there is a lot of potential with it. I recently replaced my HomeBridge installation on my Raspberry Pi 3 with the prebuilt RPi3 image.

If you manage servers big or small take a look at Ansible. It isn’t new technology but it is something I’ve grown quite fond of recently. It’s easy to install on Linux, Mac and even Windows 10 if you have that oddly named Linux add-on. Even if you don’t use Ansible to manage servers you should use something.

If you enjoy Destiny or Destiny 2, checkout Guardian Theater. This is a project spearheaded by a friend of mine after collaborating with me on a Xbox GameDVR clip site (https://xboxrecord.us) and deciding it’d be way cooler if you could look up clips related to yours. Guardian Theater promises to show game clips recorded by other guardians while in the same activity as you. Lots of fun!

Found myself with an odd situation when running Jenkins using Docker. The time displayed was correct but claimed it was UTC which lead to some inconsistent behavior. The best way to resolve this is to force the Docker container to use the correct timezone from the host system.

To do so, add the following to your run command:

-v /etc/timezone:/etc/timezone -v /etc/localtime:/etc/localtime

screen-shot-2016-10-24-at-11-22-05-pm

Not too long ago a Raspberry Pi 3 found its way into my home and after looking around on the Internet for Raspberry Pi based projects I decided to turn it into a HomeKit hub using Homebridge. While installing Homebridge isn’t terribly difficult I decided I could make it even easier by utilizing puppet.

Using puppet to install Homebridge as well as any additional modules you want is beneficial because you can backup the puppet files and if you need to rebuild your Raspberry Pi for any reason you can quickly get back up and running by applying your existing puppet config. If you’re feeling adventurous you can manage all of your software and configs using puppet.

This post details how to get Homebridge installed on a Raspberry Pi running Raspbian Jessie and assumes you have already Raspbian installed, have gained access to it using ssh or are using the terminal application in PIXELS.

To install Homebridge, do the following (prefix all commands with sudo if you are not already logged in as root):

  1. apt-get update
  2. apt-get install puppet
  3. puppet module install puppetlabs-apt
  4. puppet module install puppet-nodejs
  5. puppet module install dustinrue-homebridgepi

With that out of the way you can create the puppet file that defines what software is installed on your system. To get started, all you need is the following:

include homebridgepi

Into a file called homebridge.pp using your favorite editor. You can now tell puppet to apply this to your system.

puppet apply homebridge.pp

This will install Homebridge and set it to start on system boot using systemd. Homebridge will be installed to run as root but you can change this behavior by editing /etc/systemd/system/homebridge.service and changing the User value. I choose to run it as root as some modules require root privileges to run helper programs. Know that if you do so you’ll need to manually move your .homebridge directory from /root to whatever user you choose.

You can view the logs of Homebridge by issuing:

journalctl -uf homebridge

You will find your config.json file located at /root/.homebridge/config.json. In it will be the default config that you can expand on. You can configure any additional modules you want to install (described below) in this file as well.

From here you can define additional Homebridge modules you want to install by editing your homebridge.pp file. As an example, here are some of the modules I install on my system to support what I’m using Homebridge for:

 

include homebridgepi

package {
  'vim':
    ensure => 'installed',
    provider => 'apt';

  'homebridge-wol':
    ensure => 'latest',
    install_options => '--unsafe-perms',
    provider => 'npm';

  'homebridge-xbox-one':
    ensure => 'latest',
    source => 'https://github.com/dustinrue/homebridge-xbox-one.git',
    install_options => '--unsafe-perms',
    provider => 'npm';
}

 

My multicore Solr on Ubuntu 10.04 has proven to be one of my most popular posts yet.  Seeing the success of that post I decided it was time to show how to get the latest version of Solr up and running on Ubuntu 10.04.  As of this writing the latest version of Solr is 3.4.0.

Before we get started you should read and follow my previous post because I borrow all of the config settings from Ubuntu’s Solr 1.4 packages.  The default config settings from the Ubuntu maintainers is still a decent starting point with Solr 3.4.  Once finished you can safely remove the old Solr 1.4 package if you want to.

With a working Solr 1.4 installation in place, we can get started on getting Solr 3.4 running.  You can change some of the following paths if you want, just remember to change them in all of the appropriate places.  Everything you’re about to see should be done as the root user.

Create some required paths

mkdir /usr/local/share/solr3
mkdir /usr/local/etc/solr3
mkdir -p /usr/local/lib/solr3/data

Next, re-own the data dir to the proper user

chown -R tomcat6.tomcat6 /usr/local/lib/solr3/data

Download the latest version of Solr

You can get the latest version of Solr from http://lucene.apache.org/solr/ and extract the files into root’s home directory.

wget http://mirrors.axint.net/apache//lucene/solr/<version>/apache-solr-<version>.tgz
tar zxvf apache-solr-<version>tgz

Extract the war Solr war file

Extract the Solr war file into a location.  You may need to install the unzip utility with apt-get install unzip.

cd /usr/local/share/solr3 
unzip /root/apache-solr-<version>/dist/apache-solr-<version>.war

Install additional libs

There are a few other libs included with the Solr distribution.  You can install anything else you need, I specifically need to have the dataimporthandler add ons.

cp /root/apache-solr-3.4.0/dist/apache-solr-dataimporthandler-* WEB-INF/lib/

Configure Multicore

If you want to have multicore enabled you’ll need to perform the following actions.  The rest of this post assumes you have copied this file and will require you to make some changes to support multicore.  I’ve marked steps that can be skipped if you also wish to skip the multicore functionality.

Copy in the multicore config file:

cp /root/apache-solr-3.4.0/example/multicore/solr.xml .

You should now edit the solr.xml file at this point, doing the following:

  • Set persistent to true
  • Remove entries for core0 and core1

Next, change the ownership and permissions so that tomcat is able to modify this file when needed

chown tomcat6.tomcat6 /usr/local/share/solr3
chown tomcat6.tomcat6 /usr/local/share/solr3/solr.xml

Copy existing config files

This is where we’re going to borrow some files from Ubuntu’s Solr package maintainer.

cd /usr/local/etc/solr3
cp -av /etc/solr/* .

Because we simply copied the config files we need to modify them to fit our new environment.  Change the following in the solr-tomcat.xml file:

  • Change docBase to /usr/local/share/solr3
  • Change Environment value to /usr/local/share/solr3

Also edit tomcat.policy file changing:

  • Modify all entries referencing solr to point to appropriate /usr/local location

Change the following in conf/solrconfig.xml:

  • Change <dataDir> to /usr/local/lib/solr3/data

If you are using multicore and you followed the Solr 1.4 multicore post you’ll have a conftemplate directory and you’ll need make changes to conftemplate/solrconfig.xml

  • Change <dataDir> to /usr/local/lib/solr3/data/CORENAME

Create symlinks

Here we’ll create some symlinks to support the way Ubuntu packages Solr.  This is necessary because we copied Ubuntu’s config files and those files reference a few locations.  Creating the symlinks also allows us to continue using the scripts created in the previous post with minimal modifications.

  • cd /usr/local/share/solr3
  • ln -s /usr/local/etc/solr3/conf
  • ln -s /usr/local/etc/solr3/ /etc/solr3
  • ln -s /usr/local/lib/solr3 /var/lib/solr3

Enable/Start the new Solr instance

We can now enable our new Solr 3.4 instance in tomcat by doing the following:

cd /etc/tomcat6/Catalina/localhost
ln -s /usr/local/etc/solr3/solr-tomcat.xml solr3.xml

Note that the name of the symlink is important as it will define where we find this instance (/solr vs /solr3).  At this point you can create a new core.  I’ve provided the updated scripts here.

 

I’ve been putting a lot of time into this little project. Nobody uses it (yet?) and truth be told I barely use it in the house but it’s been such a great way to learn a number of different things including python, mDNS (bonjour), creating installer files for debian and OS X systems and even git that I can’t stop working on it.

I’m now releasing version 0.3.0. This version brings a few changes but most notably the Linux client is now ready. The next release will be coming shortly and will focus on making the client the more robust about how it deals with network disconnects.

You can read more about the 0.3.0 release at https://github.com/dustinrue/Dencoder/wiki

I just spent that last couple of hours trying to figure out why I couldn’t create a new software RAID set on my Ubuntu 10.04 system. Long story short, it turned out to be device mapper grabbing hold of the drives at boot. No amount of lsof would show that the devices were busy. The key was running dmsetup table and seeing that the drives in question were indeed “locked” by the device mapper.

This thread was the key I needed to get it all figured out – http://www.mail-archive.com/[email protected]/msg10661.html

After issuing dmsetup remove followed by the device name shown in dmsetup table I was off and running.

There are two things I’ve learned about Ubuntu 10.04 if you’re running Squid. Upstart doesn’t properly handle squid’s shutdown process properly. You need to issue squid -k shutdown *twice* if you want it to shutdown.

Two, the /etc/default/squid.conf SQUID_MAXFD is not listened to either, you must set this in /etc/squid/squid.conf instead.

https://bugs.launchpad.net/ubuntu/+source/squid/+bug/580590

Well not quite. Turns out people have been getting confused on the pricing grid Oracle has on their site for the various products they provide. The confusion comes from the Embedded version of MySQL not supporting InnoDB and that the community edition isn’t listed as part of the grid.

The community edition still has InnoDB built in as an available storage engine but you can’t buy support from Oracle.

http://www.mysql.com/products/
http://palominodb.com/blog/2010/11/04/oracle-not-removing-innodb

In my previous post I talk about needing a TFTP server in order to serve some files to a hardware device. This post describes how I used expect to automate the process of logging into the hardware device and issue commands that copy in a config file, commit it to the device, upgrade the firmware and finally tell the device to reset to factory defaults and reboot.

Expect is a way to programmatically work with a normally interactive process. Using expect you can write a script that telnets into a system and then issues commands based on what it “sees.” Here is the script I used, with some important values removed, to automate the process of updating a number of devices.

#!/usr/bin/expect
set timeout 300
spawn telnet 192.168.1.1
expect "login: "
send "root\n"
expect "Password: "
send "tehmagicphrase\n"
expect "# "
send "cd /tmp \n"
expect "# "
send "tftp -g -r config.ini 192.168.1.159\n"
expect "# "
send "config.sh import config.ini\n"
expect "# "
send "tftp -g -r firmware.img 192.168.1.159\n"
expect "# "
send "firmware_upgrade /tmp/firmware.img 1\n"
expect EOF

The above script was saved into a file called pushConfig.expect and set as executable using ‘chmod +x pushConfig.expect’. To run the script, I powered on the device and waited for it to be ready, once ready I issued ./pushConfig.expect to start the update process.

Using expect is fairly straightforward. The most difficult part is ensuring you correctly tell expect what to look for before sending the next command. In the script above I do the following:

set timeout 300

This tells expect to wait at least 5 minutes for matching text before continuing to the next send command. What this means, is if I tell it to send some data it’ll wait up to 5 minutes to see what is in the expect line after the send. In the case of my script the firmware upgrade could take quite a bit of time and I didn’t want it to timeout so I set the value fairly high.

The next line tells expect to start a telnet session to a remote machine and then to wait until it sees:

login: 

Once it sees that it sends the username. The script continues like this until it sees EOF. At this point expect knows that the process is now complete and it exits.

By using an expect script I was able to simply power on the hardware device and wait for it to boot. Once booted I ran the script. This saved me and a co-worker a lot of time while pushing custom configurations and upgrading the firmware on a number of devices.

Expect is capable of a lot more than I used in my example and can react differently based on what it receives back from the interactive process or even loop over a series of commands. To learn more about expect try ‘man expect’ or search your favorite search engine.