The Cult of Gary

And it was the one my blog was running on.

It was the damnest thing too. I was able to reboot it from the API and look at the console output. As far as I can tell, the network adapter wasn’t able to DHCP an IP address:


Welcome to CentOS release 5 (Final)
Press 'I' to enter interactive startup.
Setting clock : Mon Dec 22 20:06:28 EST 2008 [ OK ]
Starting udev: [ OK ]
Setting hostname localhost.localdomain: [ OK ]
No devices found
Setting up Logical Volume Management: No volume groups found
[ OK ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/sda1
/dev/sda1: clean, 96119/1313280 files, 616984/2621440 blocks
[/sbin/fsck.ext3 (1) -- /mnt] fsck.ext3 -a /dev/sda2
/dev/sda2: clean, 5853/19546112 files, 892921/39092224 blocks
[ OK ]
Remounting root filesystem in read-write mode: [ OK ]
Mounting local filesystems: [ OK ]
Enabling local filesystem quotas: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
INIT: Entering runlevel: 4
Entering non-interactive startup
Starting background readahead: [ OK ]
Checking for hardware changes [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0... failed.
[FAILED]
Starting auditd: [FAILED]
curl: (7) Failed to connect to 169.254.169.254: Network is unreachable
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Starting syslog-ng: [ OK ]
Starting irqbalance: [ OK ]
Starting system message bus: [ OK ]
Mounting other filesystems: [ OK ]
Starting sshd: [ OK ]
Starting cups: [ OK ]
Starting MySQL: [ OK ]
Starting postfix: [ OK ]
curl: (7) Failed to connect to 169.254.169.254: Network is unreachable
Starting httpd: Warning: DocumentRoot [/dev/null] does not exist
Warning: DocumentRoot [/dev/null] does not exist
Warning: DocumentRoot [/dev/null] does not exist
httpd: Could not reliably determine the server's fully qualified domain name, using localhost.localdomain for ServerName
[ OK ]
Starting crond: [ OK ]
Starting process accounting: [ OK ]
Starting atd: [ OK ]
Starting jexec: Starting jexec services[ OK ]
Starting HAL daemon: [ OK ]
+ Updating ec2-ami-tools
curl: (6) Couldn't resolve host 's3.amazonaws.com'
c
CentOS release 5 (Final)
Kernel 2.6.16-xenU on an i686
localhost l

Luckily I had backups, so I booted a new instance and restored it it. It looks like I posted one article since my last backup. I reckon I’ll be able to get that back from a google cache.

It took me about 30 minutes to start the instance, restore the backup and test everything out. This time around, I set up an elastic IP. If this happens again, I won’t have to update DNS.