A Ghost blog (not a blog about ghosts).

Denise Horrell: 1937-2017

Denise Horrell: 1937-2017


Creating a Custom FreeBSD 10 ISO with Automated Installation

Last month I was working on a FreeBSD 10.1 KVM image for SmartOS, SmartDataCenter and the Joyent Public Cloud. The first version of the image was released a few weeks ago and I'd like to share how I went about building the image. More specifically, I'd like to provide an overview of how I built a custom FreeBSD 10.1 ISO that I use as part of my build process.

A Little Background

If you're not familiar with how KVM images are created for SmartOS/SmartDataCenter the general workflow goes like this:

  1. Create a blank KVM VM
  2. Boot it with an installation ISO (in this case a FreeBSD ISO)
  3. Install the OS via the ISO
  4. Install the guest tools for the OS and customize the installation
  5. Shut the VM down (poweroff)
  6. Take a ZFS snapshot of the VM
  7. Save the ZFS snapshot as a file using the zfs send command
  8. Finally, create an image manifest for the image

For more detail about all of the steps above, you can read more in the SDC7 doc under How to Create a KVM Image.

For this post, I'm going to focus on steps 3 to 5 which are all handled via a custom ISO I built and helps to speed up the whole process tremendously.

Building a Custom FreeBSD 10.1 ISO

Before I begin, you can find all my scripts and files I used to build a custom FreeBSD ISO on GitHub here: mi-freebsd-10 . Pretty much everything I'm going to talk about can be found in the build_freebsd_iso script, so if you want take a look at that first, got for it.

Building a custom FreeBSD ISO is fairly easy but there are a few catches (more on that later). Once you have the source ISO (e.g., FreeBSD-10.1-RELEASE-amd64-disc1.iso) you mount it, make the modifications you need, then create a new custom ISO. If you're doing this under FreeBSD, it would look something like this:

# Mount the ISO using /mnt/freebsd-iso as the mount point
mount -t cd9660 /dev/$(mdconfig -f FreeBSD-10.1-RELEASE-amd64-disc1.iso`)/mnt/freebsd-iso

# Copy the ISO contents to /mnt/custom-freebsd-iso
rsync -aq  /mnt/freebsd-iso/ /mnt/custom-freebsd-iso

# After making modifications to /mnt/custom-freebsd-iso,
# create a new ISO
CUSTOM_ISO_TITLE=$(isoinfo -d -i ${ISO_DIR}/${ISO} | grep "Volume id" | awk '{print $3}')
mkisofs -J -R -no-emul-boot \
-p "Joyent" -b boot/cdboot \
-o freebsd-10-custom.iso /mnt/custom-freebsd-iso

One thing you'll notice about the above code is that I'm getting the ISO label using isoinfo. Starting with the FreeBSD 10.1 release, each ISO label (or "Volume ID") is unique and is based on the release information (version, architecture, that kind if thing). If you don't use the correct label in your custom ISO, the FreeBSD installation will fail spectacularly! I learned this the hard way...

As for actually customizing the ISO itself, there are a couple of tricks to that. First, modifications you make to the ISO won't actually be included in your FreeBSD installation. So for example, if you make changes to /mnt/custom-freebsd-iso/etc/rc.conf it won't be included in the installation when you start the install process. Typically you'd want to do things like add networking info to the rc.conf file so the installer has network access (ifconfig_vtnet0="DHCP") or shorten the boot delay by adding autoboot_delay="5" to /mnt/custom-freebsd-iso/boot/loader.conf (the default is 10 seconds). So what if you want to have an installation that has modifications to the rc.conf file? And how do you add new files to the installation? Well, it turns out that FreeBSD allows you to script the installation process!

Automating the Installation

FreeBSD 9.0 introduced a new installer called bsdinstall. When you boot a FreeBSD 9.0 or newer ISO the installer is automatically started once the boot timer runs out, which kicks off the bsdinstall process.

You can automate the installation by including a file called installerconfig in the /etc/ folder of the ISO. When the FreeBSD installation starts it looks for that file. The file allows you to set some configuration defaults in a "preamble" section using environment variables. It's similar to a Kickstart file you'd use in Fedora or CentOS. If you take a look at the installerconfig file in the mi-freebsd-10 repos you'll see that I have it partition vtbd0 using the typical defaults:


You'll also notice I'm telling the installer to include the following distributions:

DISTRIBUTIONS="kernel.txz base.txz joyent.txz"

If you want a minimal FreeBSD install, you'd typically just have kernel.txz base.txz here. So what's the joyent.txz distribution file and where did it come from? Glad you asked! That's how I install the various guest tools I need for our KVM image to run under SmartOS and SmartDataCenter. I'll get back to that in a bit, but there's more stuff in the installerconfig script I want to cover.

After the preamble section theres the "Setup Script" which starts with a shebang:


Here is where you do all your customization occurs. This section is run after the FreeBSD installation completes. Looking at the installerconfig file on GitHub you can see this is where I do things like modify the rc.conf file for the installation, create a custom /etc/motd file and install packages via pkg install -y. But what about adding new files?

Adding a Custom Distribution File

The installerconfig file lets you do a lot of useful things. One thing that you can't do is copy files into the installation. This is because at this point the installation is all happening within a chroot. By default that's the /mnt directory (which you can change via the BSDINSTALL_CHROOT environment variable, so the installation is run in that directory with no visibility to anything outside of it. That means you can't copy files over using the scripted part of installerconfig. This is where the joyent.txz distribution comes in.

To install files to the installation, you just save them all as a distribution file, the add that file to DISTRIBUTIONS= in the preamble of installerconfig. The bsdinstall process will take you distribution file and unpack it to where it needs to go.

Here's the basic steps for creating the distribution file called custom. Let's say you have a custom rc.conf file that you want to include. Create a directory called custom_files.

mkdir custom_files

The rc.conf file likes in /etc, so you need to create an /custom_files/etc folder to match:

mkdir custom_files/etc

Now, add your file to custom_files/etc and create the distribution using the tar command:

cp rc.conf custom_files/etc/
cd custom_files
tar -cvJpf ../custom.txz ./etc

And that's it, you're custom rc.conf file is now in a distribution file called "custom.txz". If you include in in the preamble part of your installerconfig file (e.g., DISTRIBUTIONS="kernel.txz base.txz custom.txz"), bsdinstall will unpack and copy everything in your custom distribution over to their respective directories.

One final thing to note about the installerconfig: at the end of the file I'm issuing the poweroff command. Normally the bsdinstall process will run what's after the shebang and then automatically reboot. Since I'm creating a SmartOS/SmartDataCenter image, I don't want that to happen because I need to do a snapshot. Issuing a full shutdown at the end prevents the automatic reboot from happening.

So that's pretty much it. The above information should hopefully be useful to anyone who wants to create a custom FreeBSD ISO with an automated/unattended installation.

And special thanks to Daniel Malon who provide a lot of help and introduced me to the bsdinstall man page :)

Switching over from Textpattern to Ghost

I recently switched Horrell.ca from Textpattern to Ghost. Chances are there are a bunch of broken links to files and such. I'm still tweaking and fixing things and reworking the site design, so expect more changes in the coming months :)

For anyone else who wants to attempt this, a word of warning: there's no easy way to export content from Textpattern to import into Ghost. Ideally, a Textpattern plugin for this would help tremendously, but I wasn't able to find one. I was going to attempt to create one myself, but it's been so long since I've worked in PHP or created a Textpattern plugin that it didn't seem to be worth my time. Instead I (mostly) followed the instructions I found here. The short version of what you have to do: you need to temporarily setup a WordPress blog, import your Textpattern posts into that, then export your posts via a Ghost WordPress plugin, then import into Ghost. Yeah. If you use Textpattern and are thinking about moving to Ghost, it might be best to wait for a bit until bettor options come up.

Despite all that, at least for me, the switch has been worthwhile. It's a great platform for writting and since it runs on Node.js, it's very light on resources. I was able to resize my current Joyent SmartOS instance down significantly since removing Apache and MySQL from the equation.

Speaking of SmartOS instances, I created a handy script for getting Ghost up and running on Joyent called smarty-ghost . It works fairly well and takes care of setting up the SMF manifest for you. I'll be improving that script over time and the install process should become much simpler when Ghost becomes an published npm package.

Oh, and while I was moving things around on my site I put all the Textpattern plugins I created over the years on GitHub. I probably won't be using Textpattern much or contributiing to the ecosystem anymore, but my old code is out there for anyone who might find it usefull. Here's a list of links to each plugin repo:

I could watch this all day



Installing mod_cloudflare on a Joyent SmartMachine

CloudFlare is a great service and I’ve been using for my horrell.ca site for some time. It speeds everything up, acts as a firewall protecting your site from bots and jerks, and also provides IPv6 support. One side affect of using CloudFlare is the Apache logs for your site will show IP addresses from the CloudFlare network for any site visitors rather than the true visitors IP address. You can fix that by installing the mod_cloudflare Apache module.

Here’s how to install it on a new Joyent SmartMachine

  1. First, download the mod_cloudflare.c source file from CloudFlare’s Github repos (the -L flag makes sure curl follows any redirects):

    curl -OL https://raw.github.com/cloudflare/CloudFlare-Tools/master/mod_cloudflare.c
  2. Install the gcc compiler:

    pkgin install gcc47
  3. Now compile and install the module:

    apxs -c -i mod_cloudflare.c

    If you have a 64 bit machine, you’ll need to use something like this instead:

    apxs -Wc,-m64 -Wl,-m64 -c -i mod_cloudflare.c
  4. Set the permissions:

    chmod 755 /opt/local/lib/httpd/mod_cloudflare.so
  5. Add this to /opt/local/etc/httpd/includes/dso.conf (this will load the module when Apache starts):

    LoadModule cloudflare_module lib/httpd/mod_cloudflare.so
  6. Now refresh Apache to load the module:

    svcadm refresh apache

And that’s it! If you check your Apache logs (/var/log/httpd/access.log or /home/NAMEOFSITE/logs/access.log) you should start seeing accurate IP addresses.

You can check if the installation went smoothly with:

svcs -vx apache

If Apache is in maintenance, chances are something went wrong with loading the module. The /var/svc/log/network-apache:default.log service log file will give an idea of what you need to fix.

Periodically, CloudFlare adds new IP addresses to their network, so you’ll need to reinstall the module. You don’t need to do anything special for that, just repeat the above steps using the updated mod_cloudflare.c. source file. I follow the git repos for changes (which doesn’t happen that often).