Ubuntu 10.04 on EC2

Not the most thrilling of titles, to be sure, but if you’re looking to dive into the world of Amazon Web Services by starting up some EC2 instances from the Canonical AMIs then my working notes may be of interest. I encountered a couple of problems on the way, but little that did more than expose my ignorance!

The scenario: I want to spool up servers which will run Ubuntu 10.04 LTS configured by puppet so that I need have minimal interaction with the instances. To do this I need to build a custom AMI which, for convenience, I’d like to build from a known base AMI in EC2.

Note that I am using the Python boto package to manage my instances

Step 1: reach the start line

You will need to setup an AWS account and avail yourself of the following:

• A key-pair, created in the EC2 management console, that will allow you to SSH into new instances
• An X509 certificate and private key file, created through the account management pages
• Access key ID and Secret Access Key, also created through the account management pages

Next you’ll need to pick a starting point - an AMI from which you are going to build your own machine image. Canonical release a number of AMIs of base systems and it is one of these that I shall use. For 10.04 there are two AMIs configured for use without EBS, one 32bit and one 64bit:

• 64 bit - ami-631f2b17
• 32 bit - ami-a11e2ad5

Setup your machine

Fire up the Canonical AMI that you have chosen. You can login with SSH as follows, assuming your SSH private key is in ~/master.pem:

$ ssh -i ~/master.pem -l ubuntu <your ami public dns name>

Now add software as you see fit - update the apt-repository, install tools… everything you’d want to have every time you spin up a host based on this AMI.

In my case I wanted to update the system:

$ apt-get update
$ apt-get upgrade

And install puppet. I also put an entry in /etc/hosts for my puppet master, mapping the IP address to the name ‘puppet’. Finally I edit /etc/default/puppet and set START=yes so that, on boot, puppetd will start up.

Create your AMI

Now you’ve got a machine made you need to use the EC2 AMI tools to bundle it up and push to S3. Unfortunately, when using the AMIs above, the tools in the default ec2-ami-tools package (available in multiverse) do not create a bootable machine. This is because the root filesystem in fstab is referenced by the label uec-rootfs, but /dev/disk/by-label/uec-rootfs does not get copied into the image so on boot, there’s no root to be found. 

There are a couple of ways around this, but a nice one is to use a patched version of the tools built by Scott Moser and available here. Install as follows:

$ add-apt-repository ppa:smoser/bundle-fix-sru-test
$ apt-get update
$ apt-get install ec2-ami-tools


Now copy your x509 certificate and private key file to /mnt on this machine.

You will now bundle the running machine, storing its state in a set of files to be made into an AMI that will allow you to spin up many instances that look just like this machine.

$ ec2-bundle-vol  -k /mnt/pk-YOURPKID.pem -c /mnt/keys/cert-YOURCERTID.pem -s ROOT_PARTITION_SIZE -u YOUR_USER_ID

• substitute the path to your private key file for -k
• substitute the path to your x509 certificate file for -c
• substitute your Amazon user ID for -u
• insert the desired size of your root partition in Mb for -s. The maximum is 10240 (10Gb). This will determine the size of your AMI and the price you pay for storage in S3

Now create an S3 bucket to store your AMI - you can do this through the console or using command line tools or scripting libraries like Boto. Your call.

Upload your image to S3:

$ ec2-upload-bundle -b BUCKET_NAME -m /tmp/image.manifest.xml -a ACCESS_KEY -s SECRET_KEY

• the bucket name is that which you just created
• Access key and Secret key must be substituted for in -a and -s

Register your AMI

Via the EC2 console go to AMIs, click ‘Register new AMI’ and give the path to the S3 bucket you just uploaded the image to.

Now you can startup the AMI and try it. Once running SSH in exactly as you did to the Canonical image you started to begin with.

Note that if you added puppet in the manner described, your puppet master will now have a certificate waiting to be signed from the new machine. The name seen by puppet is the private DNS name for the host.