Thursday, April 30, 2009

Create Image from existing instance and reuse to create a new instance

Often we need to create one production instance and another test instance for testing. Environment is similar for both instances. So if you are using Amazon ec2 then you can configure one instance and then create an image from that instance to reuse it to create another instance just like the first one.

Create volume from your instance

Copy pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem and cert-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem file into /mnt folder in your instance.

Note: You can download these files from your amazon account.

/mnt is not included in the image so your certification files will be secured and not stored in the image.

login to your instance and run in command line,

ec2-bundle-vol -d /mnt -k /mnt/pk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem -c /mnt/cert-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.pem -u xxxxxxxxxxxx -r i386 -s 1536
Here,
–d => is the directory where the images will be generated
-k => privatekey PATH
-c => cert PATH
-u => your AWS account ID as your username (not your AWS Access Key ID)
-r => Specify target architecture. [i386, x86_64]
-s => The size, in MB (1024 * 1024 bytes), of the image file to create. The maximum size is 10240 MB
upload bundle into your s3 bucket

Then run,

ec2-upload-bundle -b 'your-s3-bucket' -m /mnt/image.manifest.xml -a 'aws-access-key-id' -s 'aws-secret-access-key' 
Register image

run

ec2-register 'your-s3-bucket'/image.manifest.xml

You should see some thing like this in the screen.

IMAGE   ami-xxxxxxxx

Note: You might not get this command from instance. For this you will need ec2 api tool in you instance or in your local machine.

You can download it from here.

Running Instances from your newly created image

run

ec2-run-instances ami-xxxxxxxx

You should see some thing like this in the screen.

RESERVATION     r-632db90a      106811649921    default
INSTANCE        i-xxxxxxxx      ami-xxxxxxxx                    pending
0               m1.small        2009-04-30T05:26:38+0000        us-east-1c

resources

http://docs.amazonwebservices.com/AmazonEC2/gsg/2006-06-26/creating-an-image.html

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351

http://wiki.apache.org/hadoop/AmazonEC2[For publishing Image as public]

Monday, April 27, 2009

Using webrat for integration testing

What is webrat

Webrat lets you quickly write expressive and robust acceptance tests for a rails application.

How Webrat is different from default rails integration test

1. Rails default integration testing is nothing more than a series of functional tests. But in webrat you can actually do actions in pages. Here is a scenario for login.

visit('/')
fill_in('email', :with => user.email)
fill_in('password', :with => '1234')
click_button('login')

2. With webrat you can test user experience by filling in the text boxes, clicking the buttons etc.

3. Use webrat API for Browser Simulator and real Selenium tests using Webrat::Selenium when necessary (eg. for testing AJAX interactions).

4. Supports popular test frameworks: RSpec, Cucumber, Test::Unit and Shoulda

Installing webrat

gem install nokogiri
gem install webrat

In test_helper.rb add,

require "webrat"

Webrat.configure do |config|
 config.mode = :rails
end

For more information please visit http://github.com/brynary/webrat/tree/master

Some Tips

You can check something like, if your desired dropdown select option is currently selected or not using assert_select.

Let me show you the expected html to test

<select name="user[user_type_id]" id="user_user_type_id">
 <option selected="selected" value="1">Shipper</option>
 <option value="2">Transporter</option>
</select>

Here we want to test of Shipper is selected or not.

So the test code can be

test 'signup form with shipper' do
 visit('/')
 click_link('Signup as Shipper')
 assert_selected_option('user_user_type_id', 'Shipper')
end

def assert_selected_option(select_id, selected_text)
 assert_select("##{select_id} option[selected=selected]", selected_text)
end

This is how you can test contents of html.

Resources

http://github.com/brynary/webrat/tree/master

http://www.slideshare.net/brynary/webrat-rails-acceptance-testing-evolved

http://cheat.errtheblog.com/s/webrat/

Sunday, April 26, 2009

Rails route sequence

For a large RESTful rails project routes can become very complicated and a single controller can be accessed under different resources.

In rails 2.3.2 route sequence is important. Let me give a scenario,

map.resources :shippers, :shallow => true do |shipper|
 shipper.resources :jobs, :member => {:publish => :put} do |job|
 end
end

map.resources :jobs, :collection => {:open_jobs => :get}

This will generate routes as follows(by running rake routes)

job             GET    /jobs/:id(.:format)          {:action=>"show", :controller=>"jobs"}
open_jobs_jobs  GET    /jobs/open_jobs(.:format)    {:action=>"open_jobs", :controller=>"jobs"}

As in route.rb file /job/:id route path is defined first, when you will hit /jobs/open_jobs you will be mapped to {:action=>"show", :controller=>"jobs",:id=>"open_jobs"}

Then when I moved the open_job route definition up before the job resource under shipper, and it worked.

And now /jobs/open_jobs was mapped to {:action=>"open_jobs", :controller=>"jobs"}. So the resource definition is now like,

map.resources :jobs, :collection => {:open_jobs => :get}

map.resources :shippers, :shallow => true do |shipper|
 shipper.resources :jobs, :member => {:publish => :put} do |job|
 end
end

Reordering the sequence resolved that routing issue.

Friday, April 24, 2009

Use Migration Helpers to ease your migration script

One of the purpose of rails migration script was to wrap SQL with simple ruby code. But still we need some SQL for creating foreign key or inserting some default data or updating some existing data. There is a rails plugin to solve this problem.migration_helpers provides some basic helpers for writing migration scripts.

It wraps

  1. Add/remove foreign key
  2. Insert statements
  3. update statements

Here is a sample migration that uses insert and update statement.

class CreatePosts < ActiveRecord::Migration
    def self.up
      create_table :posts do |t|
        t.string :title
        t.text :body
        t.boolean :published
    
        t.timestamps
      end
    
      insert_row('posts', :title => 'Dummy Title', :body => 'A sample post', :published => false)
      update_row('posts', :set => { :published => true }, :where => { :title => 'Dummy Title' })
    end
    
    def self.down
      drop_table :posts
    end
end
And here is a sample migration that uses foreign key reference.
class CreateComments < ActiveRecord::Migration<br />    def self.up<br />      create_table :comments do |t|<br />        t.text :body<br />        t.boolean :published<br />        t.references :post<br />    <br />        t.timestamps<br />      end<br />    <br />      add_foreign_key('comments', 'post_id', 'posts')<br />    end<br />    <br />    def self.down<br />      remove_foreign_key('comments', 'posts')<br />      drop_table :comments<br />    end<br />end

Caution:

This is really a nice plugin but I needed to modify some code to work remove_foreign_key.

In plugins/migration_helpers/lib/migration_helpers_tasks.rb file replace the remove_foreign_key with the following code

def remove_foreign_key(table, target_table, constraint_name="#{table.to_s}_#{target_table.to_s}_fkey")<br />    execute "ALTER TABLE #{table.to_s} DROP FOREIGN KEY #{constraint_name};"<br />    execute "ALTER TABLE #{table.to_s} DROP KEY #{constraint_name};"<br />end
Reference:

http://github.com/patientslikeme/migration_helpers/tree/master



Thursday, April 23, 2009

Transaction in rails 2.3.2

In rails when we want some series of tasks to be atomic we use transaction. But how should we write our code to support transaction?

Let me explain how transaction in rails works. If any code inside a transaction raises an exception

  1. the transaction block rescues it
  2. rolls back the database and
  3. re throws the exception

So we need to make sure that statements that are not complete, raise exception. Here are some scenarios with explanation.

For all the scenarios the Comment model is like,

class Comment < ActiveRecord::Base
  belongs_to :post
  validates_presence_of :body
end

As model.collection << child_model does not raises exception the post is saved, as the transaction is unaware of the data save failure.

test 'transaction should not rollback transaction with collection create method' do<br />  is_comment_saved = false<br />  Post.transaction do<br />    post = Post.new({:title=> 'Test title', :body => 'A sample post', :published => true})<br />    post.save!<br />    is_comment_saved = post.comments << Comment.new({:published => true})<br />    #Comment with out body should be not saved for validation<br />    #but this does not raise exception so the transaction is not rolled back<br />    #I know that rails convention is that you first assign the comment and then save post<br />    #But there might be situation when you might want to save model which is unrelated to post in the same transaction<br />  end<br /><br />  assert_equal(1, Post.count)<br />  assert_equal(0, Comment.count)<br />  assert_equal(false, is_comment_saved)<br />end

To support transaction we need to write some thing like this,

test 'transaction should rollback transaction with throwing exception manually' do<br />  is_comment_saved = false<br />  begin<br />    Post.transaction do<br />      post = Post.new({:title=> 'Test title', :body => 'A sample post', :published => true})<br />      post.save!<br />      #Comment with out body should be not saved for validation<br />      raise Exception unless post.comments << Comment.new({:published => true})<br />    end<br />  rescue Exception<br />  end<br /><br />  assert_equal(0, Post.count)<br />  assert_equal(0, Comment.count)<br />  assert_equal(false, is_comment_saved)<br />end

model.collection << returns false the operation fails. So raising an exception while it returns false causes the transaction to rollback.

Let me give you a scenario with nested transaction.

test 'default Nested transaction should rollback with raising Exception' do<br />  begin<br />    Post.transaction do<br />      post = Post.new({:title=> 'Test title', :body => 'A sample post', :published => true})<br />      post.save!<br />      Comment.transaction(:requires_new => true) do<br />        post.comments.create({:body => 'A sample comment', :published => true})<br />        raise Exception<br />      end<br />    end<br />  rescue Exception => error<br />  end<br /><br />  assert_equal(0, Post.count)<br />  assert_equal(0, Comment.count)<br />end

Note: For nested transaction in rails 2.3.2 by default any transaction inherits the transaction of it’s parent transaction. To make a new child transaction you need to pass “:requires_new => true” as the parameter of transaction method.

Identical to single transaction nested transaction behaves same if you throw an exception. So if you really mean to use a child transaction, you will need to handle exception inside or around the child transaction.

The above example behaves same even if we do not use “:requires_new => true” for child transaction. now If you want to break the child transaction and not to interfere the parent transaction, you can use “ActiveRecord::Rollback”.

Transaction relays all the exception except ActiveRecord::Rollback. If you raise ActiveRecord::Rollback then the steps are

  1. the transaction block rescues it
  2. rolls back the database and
  3. DOES NOT re throw the exception

Here is an example,

test 'nested transaction should not rollback parent transaction that throws ActiveRecord Rollback in child transaction' do<br />  begin<br />    Post.transaction do<br />      post = Post.new({:title=> 'Test title', :body => 'A sample post', :published => true})<br />      post.save!<br />      Comment.transaction(:requires_new => true) do<br />        comment = Comment.new({:body => 'Test', :published => true, :post_id => post.id})<br />        comment.save!<br />        raise ActiveRecord::Rollback<br />      end<br />    end<br />  rescue Exception<br />  end<br />  assert_equal(1, Post.count)<br />  assert_equal(0, Comment.count)<br />end

In rails 2.3.2 ActiveRecord::Rollback does not work for child transaction that is not defined as :requires_new => true.

Here is another scenario for that test,

test 'default nested transaction should not rollback even child trasnsaction that throws exception in child transaction' do<br />  Post.transaction do<br />    post = Post.new({:title=> 'Test title', :body => 'A sample post', :published => true})<br />    post.save!<br />    Comment.transaction do<br />      comment = Comment.new({:body => 'Test', :published => true, :post_id => post.id})<br />      comment.save!<br />      raise ActiveRecord::Rollback<br />    end<br />  end<br />  assert_equal(1, Post.count)<br />  #Here comment is saved too<br />  assert_equal(1, Comment.count)<br />end

As you can see raising ActiveRecord::Rollback does not impact the result.

These are some of the scenario explained, which should help you to choose appropriate way to use transaction for your application.

You can download the test project from here.



Add custom rake task in your rails project

It is very easy to create a custom rake task in rails. We can often use this feature to ease our daily work. Recently I needed to run a cron job for a rails project, which is the perfect place for a custom rake task. You can follow these process to create a task.

1. Create a rake file as your_tasks.rake and put it in project_path\lib\tasks\

rails detects task from project_path\lib\tasks\

2. A sample rake task

task :start do
  #do some magic and start the cron
end
task :stop do
  #do some more magic and stop IT
end

3. Now run in command prompt from the project folder as

rake start

To make your tasks more meaningful you might want to try giving namespaces and a little description which explains the task a bit.

Here the simple modification you need

Rake task

namespace :cron do
  desc "Starts cron job"
  task :start do
    #do some magic and start the cron
  end
  desc "Stops cron job"
  task :stop do
    #do some more magic and stop IT
  end
end

Calling tasks

rake cron:start

Rake gives us the flexibility of defining dependent tasks so that the dependent tasks can be executed first.

Often we may need our rails environment to load before we do any thing. You can do it by

namespace :cron do
  desc "Starts cron job"
  task(:start => :environment)do
    #do some magic and start the cron
  end
  desc "Stops cron job"
  task(:stop => :environment)do
    #do some more magic and stop IT
  end
end

Now when you will run the task, you will find that your environment is now loaded.

Resource:

http://www.railsenvy.com/2007/6/11/ruby-on-rails-rake-tutorial

http://rake.rubyforge.org/

http://martinfowler.com/articles/rake.html

Monday, April 13, 2009

DateTime with zone support in rails fixture

To get time with zone support we usually call, Time.now.utc

But if we use Time.now.utc in fixture like,

fixture1:
deadline: <%= Time.now.utc %>

It gives an error:

ActiveRecord::StatementInvalid: Mysql::Error: Incorrect datetime value: 'Sun Apr 12 11:40:05 UTC 2009' for column 'deadline' at row 1: INSERT INTO job_assignments (job_id, updated_at, id, deadline, transporter_fee, transporter_id, created_at, state) VALUES (245383586, '2009-04-12 09:40:07', 809788707, 'Sun Apr 12 11:40:05 UTC 2009', 75, 468814034, '2009-04-12 09:40:07', 'assigned')

Because fixtures calls a default to_s function for each fields to create the insert script. Fixtures does not save records through Models. Time.now.utc.to_s generates a string which is not in the format to save date time in mysql. To generate such format we need to call Time.now.utc.to_s(:db). So the fixture turned out like,

fixture1:
deadline: <%= Time.now.utc.to_s(:db) %>