Tag Archives: migration

Migrating email to GMail

I recently moved extricate.org to sit on Amazon EC2. This meant I had to decide what to do with my email. Should I just leave it on Dreamhost and access it via IMAP?

In the end, I decided to enable Google Apps for my domain. Free for 10 users or less so that will do nicely. GMail has a fantastic interface nowadays and after using it for a while I was happy that I would use it instead of IMAP, although IMAP is on offer should it ever be needed.

Getting up and running was straightforward, with the main element being to point my DNS to the new servers. Google walk you through all this via their setup wizard, so well played to them there. One immediate issue was that my new email address already existed as a ‘personal’ Google account, and that conflict needed to be resolved. In the end I renamed that account, started a ‘blank’ new one, and migrated the data across.

I transferred most of my email from Dreamhost by using Thunderbird. Having enabled IMAP on GMail, the folders could be consolidated and dragged between accounts. This wasn’t particularly fast as it was reliant on my home broadband connection. When it came to larger folders, I needed a better way!

Take it to the cloud…

It made sense to perform the copying via my Amazon EC2 server, as it wouldn’t be constrained by my home network connection. It did mean finding some suitable Linux command-line software. I decided upon imapcopy. I downloaded it straight from the home site.

However, this software is quite old and does not support secure connections, as required by GMail’s IMAP servers! This is where stunnel came in, which may be installed from the Amazon Linux AMI repositories:

sudo yum install stunnel

Configuration guide: stunnel and imapcopy.

I then just needed to configure imapcopy (ImapCopy.cfg). Naturally, the DestServer had to be sent to ‘localhost:1143’ so that the routing occurred through the new tunnel. I also explicitly stated what I wanted to happen:

copyfolder INBOX.Sent
DstRootFolder “Migrated”

It was then just a case of running imapcopy and away it went! It worked flawlessly which meant that all my email was now accessible within GMail.

A successful migration!

Moving extricate.org to Amazon EC2

I have been continuing work with Amazon EC2 recently. I am a big cloud fan (evangelist?) and, as a result of that, it was time to eat my own dog food and move my domain over.

My host for a couple of years has been Dreamhost. They provide an excellent hosting service, including a very comprehensive control panel and secure shell access. Nothing wrong there at all: I just wanted to go down the Amazon route.

Another factor is that the Dreamhost servers are based in the US. This meant that the site always seemed slightly sluggish to me, so I was hopeful that moving hosting to Ireland (The ‘EU-West’ region in EC2 terms) would speed things up.

Instance and OS selection

It made sense to embrace the AWS Free Usage Tier, which meant I elected to go with a ‘Micro’ instance:

613 MB memory
Up to 2 EC2 Compute Units (for short periodic bursts)
EBS storage only
32-bit or 64-bit platform
I/O Performance: Low
API name: t1.micro

64-bit was a natural choice. There is no ephemeral storage with this instance size. That was fine by me, as I wanted to ensure that everything was secure in the event of the instance terminating for whatever reason. Hurrah for EBS!

I tried out two different Linux images: Amazon’s own flavour, and Ubuntu Cloud. There was not a great deal in it, and although I am more familiar with Ubuntu on a day-to-day basis, I went with Amazon Linux. It is optimised by them for their own platform and has a CentOS pedigree.

Getting up and running

The instance was very quickly provisioned. I followed the standard WordPress Installation Guide, which included getting Apache and MySQL up and running on the box. Both of those are in the Amazon repositories. WordPress itself I did directly from the source, copying themes, plugins and other content data from my old host via scp.

I needed to bring the old MySQL database across as well. A simple mysqldump got the required SQL which was trivial to import. I used phpMyAdmin to help out with user creation and permissions, as it is a lot friendlier than tapping things out on the command line.

Memory considerations

During testing, MySQL was terminated due to the instance running out of memory. By default, there is no swap space provided. Swap space is easy to provision if required, be that via a swap file or swap partition (allocating a new EBS volume).

However, it should be noted that using swap space will naturally increase I/O, along with charges!

I decided to keep things streamlined and not enable swap space. Instead, I toned down Apache’s memory usage, as by default it would spin up 10 servers to handle requests. I went with the following settings:

StartServers 2
MinSpareServers 1
MaxSpareServers 3
ServerLimit 8
MaxClients 8
MaxRequestsPerChild 1000

Everything has been stable since then.

CloudFront

WordPress a great cache plugin: WP Super Cache. I soon got this up and running, and took advantage of its CDN support. This allows it to rewrite wp-content URLs in order to be served up by the Content Delivery Network of your choice.

Here’s a good tutorial on this: ‘Setting Up Amazon CloudFront CDN in WordPress is Really Easy!’.

This handily takes further load off the micro instance, and therefore performance is improved.

Route 53 DNS

That just left DNS over at Dreamhost. Amazon offer this service as well, in the form of Route 53. I set up a Hosted Zone for extricate.org and it was very quick to set up the required entries (pretty much just copying over from Dreamhost). The zone was instantly provisioned and worked perfectly once I instructed my registrar to use the new servers. Very easy!

The results…

I’m happy to report that everything just works! Performance is snappier as well, although I am sure that the server now sitting more local to me is a big boost here. I also love having full ‘root’ access over the system now.

That did leave migrating my email, and that is a future article!

Migrating to a new Xbox 360

I have had an Xbox 360 for a very long time now. I started out with the 20 GB model which suffered the almost obligatory Red Ring Of Death and I took full advantage of the extended warranty which had been issued due to that exact problem!

Over time, the 20 GB storage has proved to not be enough. Various improvements to the 360 offering have made this happen:

  1. LIVE ARCADE SIZE RESTRICTIONS: The size limit for downloaded games was 50 MB per game on launch. It now sits at 350 MB! Plus the number of titles being released, including ‘homebrew’ has escalated.
  2. MORE MULTIMEDIA: Video rental is now in full force on the 360. They take up a lot of space, particularly if in HD format.
  3. INSTALL TO HARD DRIVE: The ability to install games from DVD to the hard drive did not exist when the console was originally launched. It is now there! A game could take up to 3-4 GB of space and this is a significant chunk of the original 360’s 20 GB hard drive!

It is perhaps also worth remembering that one of the SKU bundles for the 360 did not even have a hard drive!

Good fortune has led me to be in the possession of a shiny new black Xbox 360! Now, deep down, it is doing exactly the same thing as my original console but there are some very noticable differences:

  1. LOTS OF SPACE: The obvious one really, isn’t it?
  2. QUIET: This one was not hard to notice given that the original 360 sounded like a jet turbine. Now it is possible to hear the DVD drive whirring around from across the room.
  3. THE EJECT BUTTON IS CUTE: Wait, what?! This caught me by surprise. When you touch the eject button on the console it makes a little chiming noise. It’s great!

I purchased the Hard Drive Transfer Kit from Amazon UK. This works as simply as you can imagine. Connect one end to the new console and the other to your old hard drive. The dashboard then does the rest (although it took about 15 seconds to realise the cable was there) and migrates the items of your choosing.

One caveat here. I made the mistake of ‘recovering’ my gamer profile on the new 360 before performing the transfer. This proved to be a waste of time because my profile was then overwritten by the one on the old hard drive. Plus, because I had performed the recovery process, it was then marked as out of date so I had to do it again! I didn’t lose any data but it did waste time as everything needed to be downloaded from Microsoft’s servers again.

I’m loving the new console so far. It now matches my other kit (funny how things change: most of my AV stuff used to be white), doesn’t damage hearing and no more worries (yet) about hard drive space!