Sunday, July 26, 2009

Give partial test release and get early feedback

In scrumpad project we give our test rollout on Tuesday and production rollout on Wednesday(if every thing goes well :) ). But soon we realized that one day after test deployment is not enough to provide  a smooth production delivery, as we need to work on client feedback and issues after test rollout. This becomes a bigger issue if the client is in the opposite time zone, which gives us less time to communicate.

To resolve this issue we introduced partial test rollout with in the sprint, which helped us a big time. This way we get early feedback from client, which makes us more efficient in responding.

Benefits

  • Early feedback reduces risk on successful production delivery
  • Reduces the scope of communication gap between client and developers

Things to keep an eye on

  • With in the sprint the code base can be unstable. To resolve this we predefine a date for partial test deployment, so that people are prepared and commit only stable code on that day. Before test rollout we run all the test codes and always keep a sharp eye on the continuous integration report so that our code base is kept stable.
  • Be brave to give partial test rollout. Some functionality might break(so let the client be aware of this) but the early feedback will surely make your life easy :) . So iterate test deploy with in the iteration.

Thursday, July 23, 2009

Looking into rails named_scope

named scope?

Nick Kallen’s has_finder plugin has found it’s way into rails 2.x as named_scope. Named scope is all about scoping or narrowing the database query.

Let me go through an example to make things easy.

class Post < ActiveRecord::Base
  belongs_to :author,   :class_name => 'User'
  has_many   :comments, :dependent  => :destroy

  named_scope :published,   :conditions => { :is_published => true }
  named_scope :draft,       :conditions => { :is_published => false }
  named_scope :has_comment, :conditions => [ 'comments_count > 0' ]

  named_scope :recent,    lambda { |*date| {:conditions => { :created_at => date.first || 1.weeks.ago } } }
  named_scope :before,    lambda{ |date| { :conditions => ['created_at > ?',  date] } }
end
You can scope your query and make it more readable.
Post.published #equivalant to Post.all.published

You can also scope on finder methods like,

Post.find(:all, :conditions => { :created_at => 1.weeks.ago }).published

You can use finder on scope like,

Post.published.find(:all, :conditions => { :created_at => 1.weeks.ago })

Even scope on the relations.

class Comment < ActiveRecord::Base
  belongs_to :post
  belongs_to :author, :class_name => 'User'

  named_scope :unread, :conditions => {:is_unread => true}
  named_scope :read, :conditions => {:is_unread => false}
end

So now you can write like,

Post.find(id).comments.unread

Chaining

Now would not it be wonderful if you could write like “Get all posts which are recently published” and it would return you what it means.

Using named_scope that dream will come true ;). You can finally write code like this,

Post.recent.published

You can chain up to your hearts content ;). You can do all sort of combination like,

Post.recent.published.has_comment

Which will return you posts which are recently published and has comments. See how much readable the code has become.

Passing Arguments

You can pass arguments using lambda.

Post.published.before(1.days.ago)

Named Scope Extensions

Extend named scope and add methods to it.

class Post < ActiveRecord::Base
  named_scope :draft,       :conditions => { :is_published => false } do
    def publish
      each { |post| post.update_attribute(:is_published, true) }
    end
  end
end
Post.recent.draft.publish

This will find all recent draft posts and publish them.

Anonymous Scopes

Creating named scopes on the fly.

# Store named scopes
published = Post.scoped(:conditions => { :is_published => true })
recent    = Post.scoped(:conditions => { :created_at => 1.weeks.ago })

# Which can be combined
recent_published = recent.published

Why use named scope?

  • Increases readability
  • Reduces number of queries as it can be chained
  • Very helpful for dynamic queries(like using pagination or dynamic filter)

A common issue

Named scope is a class methods. And you should always be very careful for class methods. Here is an example where most people does it wrong.

class Post < ActiveRecord::Base
  belongs_to :author,   :class_name => 'User'
  has_many   :comments, :dependent  => :destroy
  
  named_scope :recent,  :conditions => { :created_at => 1.weeks.ago } 
end

Now I can call Post.recent and expect to get all Posts within one week. But in times you will find that this query is starting to give wrong results.

Why???!!!

Named scope is a class method and so 1.weeks.ago is evaluated when the class was loaded. Now it does not matter when you call the named scope it will always run the query with the same created_at date. We also made this mistake :( and learned from it :)

A lambda block is not evaluated when the class is loaded. So there is a work around it.

class Post < ActiveRecord::Base
  belongs_to :author,   :class_name => 'User'
  has_many   :comments, :dependent  => :destroy

  named_scope :recent,    lambda { |*date| {:conditions => { :created_at => date.first || 1.weeks.ago } } }
end

Taking the lambda input like this will give you flexibility of not providing the parameter at all. For now ruby does not provide facility to take default parameter for lambda.

Deep inside named scope

The implementation of Named scope is a beauty of rails. Although the result that named scope produces seems like an Array, but actually it is not an Array but a simulation of an Array. Calling a named scope over a finder object actually returns a ActiveRecord:NamedScope:Scope object. When we chain scopes on that object, appends the query logic in the scope object. The query is not evaluated until an Array method(like each, first, last) etc is called. And when these Array methods are called the scope object then evaluates the query and delegates the method which was called as an array to the real array that has been evaluated. Now you can not chain the scope any more, as it is already been evaluated. That is how the named scope magic works in ruby on rails.

Reference

http://ryandaigle.com/articles/2008/3/24/what-s-new-in-edge-rails-has-finder-functionality

http://railscasts.com/episodes/108-named-scope

http://apidock.com/rails/ActiveRecord/NamedScope/ClassMethods/named_scope

Friday, July 17, 2009

Some Css and Javascripts optimization tips

There are so many aspect for optimizing the performance of a web application, that some times people tends to forget about Javascript and Css.

Yahoo developers experienced that, 80% of the end-user response time is spent on the front-end. Most of the time browser is busy downloading images, stylesheets, scripts, Flash, etc. Here you can find the details.

But there are improvement opportunities in these areas to improve the over all performance of the application.

Mostly we concentrate on some basic criteria

  1. Caching
  2. Minifying Javascript and Css
  3. Reducing number of files

Caching

Caching is great and browser gives you for free :). But should all our css and Javascripts should be cached??

Actually no!! Because some times browsers cache too much(specially Internet Explorer). You want the client to get the latest files if the file is changed.

So we need to identify the files that changes and files that do not change(library files,  like jquery, prototype etc). We expect library files to get cached and custom files that changes a lot, not to be cached. We can skip caching by passing a random query string like,

<script type="text/javascript" src="scripts/default.js?rand=1"></script>
That random number can be generated from the server side while serving the page (which is a built in facility for rails).
Now for common libraries(jQuery, protopye) we should use Google API for 2 reasons
  1. It grabs the library from the closest location it can get. So it loads faster.
  2. It frees your server from serving the library file, so that your server can serve more files instead.
This is how you can use Google API to load popular Javascript libraries.
<script type="text/javascript" src="http://www.google.com/jsapi"></script><br /><script type="text/javascript"><br />  google.load("jquery", "1.3.2");<br /></script>

Minifying Javascript and Css

According to Yahoo!'s Exceptional Performance Team, 40% to 60% of Yahoo!'s users have an empty cache experience and about 20% of all page views are done with an empty cache (see this article by Tenni Theurer on the YUIBlog for more information on browser cache usage).

So surely you want to minify all your javascripts and css on your production server. There are some nice online Javascript and css packer which actually does the trick.

Javascript

http://jscompress.com/

http://dean.edwards.name/packer/

Css

http://www.cssoptimiser.com/

http://flumpcakes.co.uk/css/optimiser/

I know what you are thinking. You can’t go to web and pack all your css and Javascripts all the time you deploy, specially if you are following Agile and deploying your code after every 2 weeks.

There are platform specific implementations to resolve this problem. For rails we can use this to minify and pack javascripts and reduce number of Javascripts.

You can use yahoo compressor written in Java, so you can run that in any platform.

http://developer.yahoo.com/yui/compressor/#work

There is another implementation by Douglas Crockford,

http://crockford.com/javascript/jsmin

We can add necessary commands to our deployment script, so that after or before deployment the compressor is run.

Reducing number of files

Web servers can not serve a lot of files consecutively. There are limitation for browsers as well. If you watch your site with firebug net inspection, you will see that a lot of file calls which are not called parallely. So reduction in number of files served can improve the performance of the web application.

How can we reduce number of files?

There are several ways we can reduce number of files.

1. We can merged css and javascript the way we minified it.

2. Most of the cases we don’t care about the images. As a result the number of images grow rapidly.

There are some nice ways to merge images into one using css background-position property. People call this css sprite.

And every one use it to reduce file size as well as number of files.

If you merge some images into one, then size is actually reduced. Because other then the picture information every image maintains a color template. If some images are merged then a super set of color template is generated which reduces the total size of the image then the sum of the size of individuals.

Here are some nice examples

http://www.smashingmagazine.com/2009/04/27/the-mystery-of-css-sprites-techniques-tools-and-tutorials/

http://www.alistapart.com/articles/sprites

http://css-tricks.com/css-sprites/

Here is a site that generates css sprite.



Thursday, July 9, 2009

Rails logger configuration and log level in production environment

In production environment, it not feasible to use one log file forever. So people use log that rolls over, depending on file size or time etc.

If we want to roll over the log file, we can define our custom logger using the following code in the configuration(in config/environments/production.rb).

config.logger = Logger.new(config.log_path, 50, 5242880)
Here is the doc for instantiating Logger. The params are as follows.
Logger.new(name, shift_age = 7, shift_size = 1048576)<br />Logger.new(name, shift_age = 'weekly')
Another thing that we need to set for the production is the log level. We do not need to log the query generated by active record or logs that are in debug level. So we can define the log level as follows.
config.logger = Logger.new(config.log_path, 50, 5242880)<br />config.logger.level = Logger::INFO
But if you are using the default log which rails offers by default, you can set the log level as,
config.log_level = :info
You can find the rails environment configuration cheat sheet here.

Wednesday, July 8, 2009

Some of the useful way of using fixture better

1. Using new fixture facilities in rails 2.(example)

    Which will remove the relational fixtures for has_many :through and has_and_belongs_to_many relations.

    It will remove the ids and will give a better way of relating between tables.

2. Fixtures are read only:    Never modify any existing fixture, unless your database schema has changed.    Which also indicates that we should be very careful when are adding a new fixture.

3. Another way to identify if the fixture is well written is that they all should be meaningful.    And none of them should look similar.

   If a different fixture is needed for only 1 or 2 test, then update existing before doing the test.    For an example,

should 'assign responsible person to unassigned task' do
    task = tasks(:task_not_started)
    task.users.clear #clear the existing fixture(not creating a new one for that fixture) 

    assign_and_assert_responsible_person(task, users(:login_user))
end
This will reduce the number of fixture set and keep them clean.

4. Often we write test codes that involves a lot of fixture.    Some times when we get test fail, we run through the fixtures, trying to find out which went wrong.    But instead if we comment our code such a way that the fixture is understood from the test code,    then that would be even better. Like,

context 'in progress task' do
  setup do
    @task = tasks(:task_in_progress)
    # task -> time_estimates
    #         [ user: login_user, hours_spent: 2, hours_remaining: 8, entry_date: 1.days.ago ]
    #         [ user: developer , hours_spent: 4, hours_remaining: 3, entry_date: Date.today ]
  end

  should 'return hours spent upto' do
    assert_equal(2, @task.hours_spent_upto(1.days.ago))
    assert_equal(6, @task.hours_spent_upto(Date.today))
  end
end
5. Inherit fields, that are less likely to be different for each fixture
backlog_item: &common_backlog_fields
  project_id: scrumpad
  type: Story

bug:
  <<: *common_backlog_fields
  type: Bug
  title: Test Bug
  content: Bug content
  created_at: <%= 10.days.ago.to_s(:db) %>
  updated_at: <%= 3.days.ago.to_s(:db) %>

story:
  <<: *common_backlog_fields
  title: Sample story
  state: <%= WorkItem::COMPLETED %>
  created_at: <%= 10.days.ago.to_s(:db) %>
  updated_at: <%= 2.days.ago.to_s(:db) %>
Here we can define the common required fields in backlog_item and use that definition for other fixtures. The fields can be overwritten by redefining in the fixtures as we did overwritten type for bug.