We are hoping over time to build up more and more functionality to help people find the AWS images they need quickly and effectively and also allowing them to track the evolution of images (both paid and free).  Also we have already just added support for EU images!

I’m sure I’ll have more to blog on it in the future – right now its back to cooking up business ideas.

So when I saw on Justin Kestelyn twitter than Oracle now has a Cloud Computing Center I had to give it a go.  After only playing for a very short while I have to say I really liked the experience from the AMI’s,   I thought the easiest way to tell you about it was just to give some basic instructions on getting going.  First I’m going to assume you have an AWS account and that you have also set-up the EC2 command line tools to get you going (if you are having trouble with the command line tools I find the tools at RightScale just rock for managing the servers).

So to get started I kicked up a terminal on my trusty mac and listed down the available images.

pdodds:/usr/local/ec2/bin$ ./ec2-describe-images -a | grep oracle-corporation
IMAGE	ami-02cb2f6b	/oracle-corporation/database-ami/64-bit/oracle_11106_SE_SE1_64Bit-image.manifest.xml	725966715235	available	public		x86_64	machine	aki-0d9f7b64	ari-369f7b5f
IMAGE	ami-7acb2f13	oracle-corporation/database-ami/32-bit/oracle_10g_XE_32Bit-image.manifest.xml	725966715235	available	public		i386	machine	aki-c5e703ac	ari-c2e703ab
IMAGE	ami-cecb2fa7	oracle-corporation/database-ami/32-bit/oracle_11106_EE_32Bit-image.manifest.xml	725966715235	available	public		i386	machine	aki-c5e703ac	ari-c2e703ab
IMAGE	ami-7ecb2f17	oracle-corporation/database-ami/64-bit/oracle_11106_EE_64Bit-image.manifest.xml	725966715235	available	public		x86_64	machine	aki-0d9f7b64	ari-369f7b5f
IMAGE	aki-c5e703ac	oracle-corporation/oracle_linux_kernels/32_bit/2.6.18-53.1.13.9.2.el5xen/vmlinuz-2.6.18-53.1.13.9.2.el5xen.manifest.xml	725966715235	available	public		i386	kernel
IMAGE	ari-c2e703ab	oracle-corporation/oracle_linux_ramdisks/32_bit/2.6.18-53.1.13.9.2.el5xen/initrd-2.6.18-53.1.13.9.2.el5xen.img.manifest.xml	725966715235	available	public	i386	ramdisk

Since I just wanted a little instance to play with I picked up the /oracle_11106_EE_32Bit, and started that, note that I had already created a default security group (with port 22 access) and a default keypair which I had on my machine.

pdodds:/usr/local/ec2/bin$ ./ec2-run-instances -k primary-keypair -g default ami-cecb2fa7
RESERVATION	r-2ed30047	207015290718	default
INSTANCE	i-0b842462	ami-cecb2fa7			pending	primary-keypair	0	m1.small	2008-09-22T21:14:17+0000	us-east-1a	aki-c5e703ac	ari-c2e703ab
pdodds:/usr/local/ec2/bin$

Next up I wanted to stick the database on an EBS volume to I decided to create one of those while I’m here (and waiting for that server to start). Note that since I was a bit lazy on setting the availability zone I wanted for the server I’ll be a little more disciplined on the volume (note that volumes need to be in the same availability zone as the server).

pdodds:/usr/local/ec2/bin$ ./ec2-create-volume -s 5 -z us-east-1a
VOLUME	vol-1104e178	5		us-east-1a	creating	2008-09-22T21:17:14+0000
pdodds:/usr/local/ec2/bin$

Now if all is well I should be able to check back on my EC2 instance to see if it is available.

pdodds:/usr/local/ec2/bin$ ./ec2-describe-instances i-0b842462
RESERVATION	r-2ed30047	207015290718	default
INSTANCE	i-0b842462	ami-cecb2fa7	ec2-75-101-230-110.compute-1.amazonaws.com	domU-12-31-39-00-E0-07.compute-1.internal	running	primary-keypair	0		m1.small	2008-09-22T21:14:17+0000	us-east-1a	aki-c5e703ac	ari-c2e703ab
pdodds:/usr/local/ec2/bin$

Once it is available, as above then we can attach the EBS volume that we created.

pdodds:/usr/local/ec2/bin$ ./ec2-attach-volume vol-1104e178 -i i-0b842462 -d /dev/sdh
ATTACHMENT	vol-1104e178	i-0b842462	/dev/sdh	attaching	2008-09-22T21:24:02+0000
pdodds:/usr/local/ec2/bin$

While that is getting hooked up, we can now log into the new server – with simple ssh -i primary-keypair root@ec2-75-101-230-110.compute-1.amazonaws.com.

At this point we accept the license terms and then set the new password, though I would recommend switching off password based connections to the server (for anyone). When we get to the “Would you like to create a database now [y|n]:” lets say no first time. Since we still need to get that EBS drive ready for use. This will drop you back to the root login, though it is easy to get back to the script.

$ mkfs /dev/sdh
mke2fs 1.39 (29-May-2006)
/dev/sdh is entire device, not just one partition!
Proceed anyway? (y,n) y

I’m probably far to lazy, but this will just get me a drive ready for use, and then its back to the Oracle script, this time we select our new device for our database to be stored, and also the flash recovery area.

/home/oracle/scripts/run_dbca.sh
The Oracle Database Software (ORACLE_HOME) has been installed at
/u01/app/oracle/product/11.1.0/db_1.
Before you can use the Oracle Software, you will need to create a database.

Would you like to create a database now [y|n]:  y

You chose to create a database now.  Before we begin, please provide the following details: 

Select a disk device to store your datafiles and redo logs on:

ID  DEVICE_NAME
--- -----------
[1] /dev/sda2
[2] /dev/sdh 

Type in the ID from the list above [DEFAULT: 1]:
2
Enter the mount point you would like this device mounted as [DEFAULT:  /u02]:

You have selected to mount /dev/sdh on to /u02, is this correct [y|n] [DEFAULT:  n]:
y

Select a disk device to store your flash recovery area files on:

ID  DEVICE_NAME
--- -----------
[1] /dev/sda2
[2] /dev/sdh 

Type in the ID from the list above [DEFAULT: 1]:
2
Using mount point /u02

You have selected to mount /dev/sdh on to /u02, is this correct [y|n] [DEFAULT:  n]:
y

Checking disk devices and mount points for Datafiles and Logfiles
Checking if directory /u02 exists...

Creating directory /u02
Trying to mount /dev/sdh on /u02
DBFILES:  /u02/oradata
FRA:  /u02/flash_recovery_area

Please enter the name for your Oracle Database.
This name will be used as your
ORACLE SID (System Identifier):

From here on in its a simple case of providing a name for the database and all the passwords for the Oracle users, and away it goes. After just a few minutes you will have your database ready to roll, the files stored on EBS and running against the Oracle AMI. It will have started the Enterprise Manager web server etc (though to be honest I kinda wish that was optional since I don’t tend to use it a whole lot).

All in all I have to say the experience was smooth and quick, making Oracle suddenly a lot more welcoming in a cloud environment and I’m sure with Oracle World in full swing we’ll be hearing a lot more about it.

And as always once you are done its nice and easy to clean up…

pdodds:/usr/local/ec2/bin$ ./ec2-terminate-instances i-0b842462
INSTANCE	i-0b842462	running	shutting-down
pdodds:/usr/local/ec2/bin$ ./ec2-detach-volume vol-1104e178
ATTACHMENT	vol-1104e178	i-0b842462	/dev/sdh	detaching	2008-09-22T21:24:02+0000
pdodds:/usr/local/ec2/bin$ ./ec2-delete-volume vol-1104e178
VOLUME	vol-1104e178
pdodds:/usr/local/ec2/bin$

In order to get things started I simply set-up the new Amazon EC2 gem.

# ONE TIME ONLY
# Execute this on each machine where you install gems to add github as a gem source
gem sources -a http://gems.github.com/

# ONE TIME ONLY
# Only if you had a previous version of the gem installed from RubyForge before we moved to GitHub
# Say 'Y' to remove all previous versions and all binaries when prompted.
sudo gem uninstall amazon-ec2

# Finally install the gem
sudo gem install grempe-amazon-ec2

Once installed we need to add a couple of environment variables  to our .bash_profile, this is useful to have around if you want to use the EC2 CLI anyway. Here’s what I put in my .bash_profile (on Leopard):

export RUBYOPT="rubygems"

# For amazon-ec2 and amazon s3 ruby gems
export AMAZON_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID"
export AMAZON_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY_ID"

Before going too far we will need to create a key pair that we can use, this is easy enough – lets start up the ec2sh and create a new keypair. We store this in a local file.

>> File.open("pdodds-keypair","w") { |file| file.write(@ec2.create_keypair(:key_name=>"pdodds-keypair").keyMaterial); file.close }
>> File.chmod 0700,"pdodds-keypair"

First up I thought I would take a look around the images that are available so that I can pick up something that I could use, since I’m bedding slowly into the Ruby world I thought I would use ec2sh and get back some information on what is available.

>> @ec2.describe_images(:owner=>'amazon').imagesSet.item.each { |image| puts "#{image.imageId} #{image.imageLocation}" } == true

Beside the scruffy ==true on the end basically this prints out the ID’s and locations of the Amazon owned images. For those new to EC2 these are the images of operating systems that we can use to boot our instance. EC2 machines currently only support persistent filesystems for those lucky enough to get on the beta (anyone going to invite me?). These means that when you boot up a machine the image is all you have, and when you power down anything you have written to disk is lost. In a while we’ll touch upon bundles – this is a way to allow you to build AMIs (Amazon Machine Image) and store them away.

The output of our command should give us a nice list, we can do a little to filter down the results (since their is a fair few to choose from).

>> @ec2.describe_images(:owner=>'amazon').imagesSet.item.each { |image| puts "#{image.imageId} #{image.imageLocation}" if image.imageLocation =~ /rails/} == true

This gave me a few to choose from

ami-0cf61365 ec2-on-rails-0.9.2/ec2-on-rails.manifest.xml
ami-540aef3d ec2onrails/ec2onrails-v0_9_7-x86_64.manifest.xml
ami-5c0aef35 ec2onrails/ec2onrails-v0_9_7-i386.manifest.xml
ami-6bcd2802 rails1-51193934437/rails1-51193934437.img.manifest.xml
ami-86c025ef ec2onrails/ec2onrails-v0_9_5-i386.manifest.xml
ami-99c025f0 ec2onrails/ec2onrails-v0_9_5-x86_64.manifest.xml
ami-99f712f0 ec2-on-rails-0.9.1/ec2-on-rails.manifest.xml
ami-a38b6eca amis.winelibrary.com/rails_svn.manifest.xml
ami-a3f91cca ec2onrails/ec2onrails-v0_9_3.manifest.xml
ami-a93adfc0 webficient/ec2_images/fc8-rails2-nginx-mysql/image.manifest.xml
ami-e620c58f ec2onrails/ec2onrails-v0_9_6-i386.manifest.xml
ami-ecc12485 ec2onrails/ec2onrails-v0_9_4-i386.manifest.xml
ami-efc12486 ec2onrails/ec2onrails-v0_9_4-x86_64.manifest.xml
ami-f920c590 ec2onrails/ec2onrails-v0_9_6-x86_64.manifest.xml

So lets get down to business – time to provision one of those servers for our use, I have to admit this was the bit that I always thought was cool, we can just use a simple command from ec2sh to create an instance of a server. Back to ec2sh.

>> @ec2.run_instances :image_id=>"ami-a93adfc0", :key_name=>"pdodds-keypair"

This will dump out a bunch of XML, basically the response from the web service. You could probably have done something to pretty up the response, however for now we can look up our instances to see if it is running

>> @ec2.describe_instances.reservationSet.item.each{ |reservation| puts "Reservation: #{reservation.reservationId}"; reservation.instancesSet.item.each {|item| puts "Instance: #{item.instanceId} #{item.instanceType} #{item.launchTime} #{item.dnsName}" }} == true

This will output the happy information, as long as the server has had a few minutes to provision and start.

i-9e5a97f7 m1.small 2008-05-03T09:19:12.000Z ec2-75-101-212-159.compute-1.amazonaws.com

Next we need to authorize the access to the server

>> @ec2.authorize_security_group_ingress :group_name=>"default",:from_port=>22,:ip_protocol=>"tcp", :to_port=>22, :cidr_ip=>"0.0.0.0/24"

Finally now from terminal we can SSH into the server, note that we do need to include the key that we created.

ssh -i pdodds-keypair root@ec2-75-101-212-159.compute-1.amazonaws.com

There we go – we have an instance of that Unix box, up and running, we can play around with the instance and when it’s time to say goodbye its a simple as going back to our ec2sh session and running:

>> @ec2.terminate_instances :instance_id=>"i-9e5a97f7"

I’ll follow up this blog post with how you can modify and then store images, later however I hope this shows how you can use the Ruby EC2 API to play around and create servers :) happy provisioning in the cloud….