Signed and Permanent cookies in Rails 3 7

Posted by pratik
on Friday, February 05

David added a very cool feature to Rails recently – Signed cookies and permanent cookies This lets you set permanent and/or signed cookies very easily.

Before this, you’d have to write :

1
2
3
4
cookies[:user_preference] = {
  :value => @current_user.preferences,
  :expires => 20.years.from_now.utc
}

Now just becomes :


cookies.permanent[:user_preference] = @current_user.preferences

In case you happen to have seen my Railssummit presentation I had talked about using ActiveSupport::MessageVerifier for implementing “Remember me” functionality. The above commit makes that a whole lot easier.

In your model User.rb :

1
2
3
4
5
# User.rb
def self.authenticated_with_token(id, stored_salt)
  u = find_by_id(user_id)
  u && u.salt == stored_salt ? u : nil
end

And when the user checks “Remember me” box, make sure the following gets run :


cookies.permanent.signed[:remember_me] = [current_user.id, current_user.salt]

This will set a permanent and signed cookie using the secret specified in ActionController::Base.cookie_verifier_secret. If you don’t have the cookie_verifier_secret defined, you might want to do that in one of the initializers.

Now when you want to login using the cookie :


user = User.authenticated_with_token(*cookies.signed[:remember_me])

In this specific case, it’s very important to use the salt in the cookie value. That makes sure the cookie gets invalidated if the user changes his password.

Now sponsorsed by Ruby Row 1

Posted by pratik
on Saturday, January 30

I’m happy to announce that this blog is now sponsored by an exclusive ad network run by James Avery called Ruby Row. While I could have just slapped google adsense all over the blog, Ruby Row provides much more value to the visitors as the ads are primarily targeted at Ruby/Rails developers.

I still have a spot open for per post text advertising. So if you’re interested, drop me an email on pratiknaik gmail

Active Record Query Interface 3.0 109

Posted by pratik
on Friday, January 22

I’ve been working on revamping the Active Record query interface for the last few weeks ( while taking some time off in India from consulting work, before joining 37signals ), building on top of Emilio’s GSOC project of integrating ARel and ActiveRecord. So here’s an overview of how things are going to work in Rails 3.

What’s going to be deprecated in Rails 3.1 ?

These deprecations will be effective in Rails’ 3.1 release ( NOT Rails 3 ) and will be fully removed in Rails 3.2, though there will be an official plugin to continue supporting them. Consider this an advance warning as it involves changing a lot of code.

In short, passing options hash containing :conditions, :include, :joins, :limit, :offset, :order, :select, :readonly, :group, :having, :from, :lock to any of the ActiveRecord provided class methods, is now deprecated.

Going into details, currently ActiveRecord provides the following finder methods :

  • find(id_or_array_of_ids, options)
  • find(:first, options)
  • find(:all, options)
  • first(options)
  • all(options)
  • update_all(updates, conditions, options)

And the following calculation methods :

  • count(column, options)
  • average(column, options)
  • minimum(column, options)
  • maximum(column, options)
  • sum(column, options)
  • calculate(operation, column, options)

Starting with Rails 3, supplying any option to the methods above will be deprecated. Support for supplying options will be removed from Rails 3.2. Moreover, find(:first) and find(:all) ( without any options ) are also being deprecated in favour of first and all. A tiny little exception here is that count() will still accept a :distinct option.

The following shows a few example of the deprecated usages :

1
2
3
4
5
User.find(:all, :limit => 1)
User.find(:all)
User.find(:first)
User.first(:conditions => {:name => 'lifo'})
User.all(:joins => :items)

But the following is NOT deprecated :

1
2
3
User.find(1)
User.find(1,2,3)
User.find_by_name('lifo')

Additionally, supplying options hash to named_scope is also deprecated :

1
2
named_scope :red, :conditions => { :colour => 'red' }
named_scope :red, lambda {|colour| {:conditions => { :colour => colour }} }

Supplying options hash to with_scope, with_exclusive_scope and default_scope has also been deprecated :

1
2
3
with_scope(:find => {:conditions => {:name => 'lifo'}) { ... }
with_exclusive_scope(:find => {:limit =>1}) { ... }
default_scope :order => "id DESC"

Dynamic scoped_by_ are also going to be deprecated :

1
2
red_items = Item.scoped_by_colour('red')
red_old_items = Item.scoped_by_colour_and_age('red', 2)

New API

ActiveRecord in Rails 3 will have the following new finder methods.

  • where (:conditions)
  • having (:conditions)
  • select
  • group
  • order
  • limit
  • offset
  • joins
  • includes (:include)
  • lock
  • readonly
  • from

1 Value in the bracket ( if different ) indicates the previous equivalent finder option.

Chainability

All of the above methods returns a Relation. Conceptually, a relation is very similar to an anonymous named scope. All these methods are defined on the Relation object as well, making it possible to chain them.

1
2
lifo = User.where(:name => 'lifo')
new_users = User.order('users.id DESC').limit(20).includes(:items)

You could also apply more finders to the existing relations :

1
2
cars = Car.where(:colour => 'black')
rich_ppls_cars = cars.order('cars.price DESC').limit(10)

Quacks like a Model

A relation quacks just like a model when it comes to the primary CRUD methods. You could call any of the following methods on a relation :

  • new(attributes)
  • create(attributes)
  • create!(attributes)
  • find(id_or_array)
  • destroy(id_or_array)
  • destroy_all
  • delete(id_or_array)
  • delete_all
  • update(ids, updates)
  • update_all(updates)
  • exists?

So the following code examples work as expected :

1
2
3
4
5
6
7
8
red_items = Item.where(:colour => 'red')
red_items.find(1)
item = red_items.new
item.colour #=> 'red'

red_items.exists? #=> true
red_items.update_all :colour => 'black'
red_items.exists? #=> false

Note that calling any of the update or delete/destroy methods would reset the relation, i.e delete the cached records used for optimizing methods like relation.size.

Lazy Loading

As it might be clear from the examples above, relations are loaded lazily – i.e you call an enumerable method on them. This is very similar to how associations and named_scopes already work.

1
2
cars = Car.where(:colour => 'black') # No Query
cars.each {|c| puts c.name } # Fires "select * from cars where ..."

This is very useful along side fragment caching. So in your controller action, you could just do :

1
2
3
def index
  @recent_items = Item.limit(10).order('created_at DESC')
end

And in your view :

1
2
3
4
5
<% cache('recent_items') do %>
  <% @recent_items.each do |item| %>
    ...
  <% end %>
<% end %>

In the above example, @recent_items are loaded on @recent_items.each call from the view. As the controller doesn’t actually fire any query, fragment caching becomes more effective without requiring any special work arounds.

Force loading – all, first & last

For the times you don’t need lazy loading, you could just call all on the relation :


cars = Car.where(:colour => 'black').all

It’s important to note that all returns an Array and not a Relation. This is similar to how things work in Rails 2.3 with named_scopes and associations.

Similarly, first and last will always return an ActiveRecord object ( or nil ).

1
2
3
cars = Car.order('created_at ASC')
oldest_car = cars.first
newest_car = cars.last

named_scope -> scopes

Using the method named_scope is deprecated in Rails 3.0. But the only change you’ll need to make is to remove the “named_” part. Supplying finder options hash will be deprecated in Rails 3.1.

named_scope have now been renamed to just scope.

So a definition like :

1
2
3
4
class Item
  named_scope :red, :conditions => { :colour => 'red' }
  named_scope :since, lambda {|time| {:conditions => ["created_at > ?", time] }}
end

Now becomes :

1
2
3
4
class Item
  scope :red, :conditions => { :colour => 'red' }
  scope :since, lambda {|time| {:conditions => ["created_at > ?", time] }}
end

However, as using options hash is going to be deprecated in 3.1, you should write it using the new finder methods :

1
2
3
4
class Item
  scope :red, where(:colour => 'red')
  scope :since, lambda {|time| where("created_at > ?", time) }
end

Internally, named scopes are built on top of Relation, making it very easy to mix and match them with the finder methods :

1
2
3
red_items = Item.red
available_red_items = red_items.where("quantity > ?", 0)
old_red_items = Item.red.since(10.days.ago)

Model.scoped

If you want to build a complex relation/query, starting with a blank relation, Model.scoped is what you would use.

1
2
3
cars = Car.scoped
rich_ppls_cars = cars.order('cars.price DESC').limit(10)
white_cars = cars.where(:colour => 'red')

Speaking of internals, ActiveRecord::Base has the following delegations :

1
2
3
delegate :find, :first, :last, :all, :destroy, :destroy_all, :exists?, :delete, :delete_all, :update, :update_all, :to => :scoped
delegate :select, :group, :order, :limit, :joins, :where, :preload, :eager_load, :includes, :from, :lock, :readonly, :having, :to => :scoped
delegate :count, :average, :minimum, :maximum, :sum, :calculate, :to => :scoped

The above might give you a better insight on how ActiveRecord is doing things internally. Additionally, dynamic finder methods find_by_name, find_all_by_name_and_colour etc. are also delegated to Relation.

with_scope and with_exclusive_scope

with_scope and with_exclusive_scope are now implemented on top of Relation as well. Making it possible to use any relation with them :

1
2
3
with_scope(where(:name => 'lifo')) do
  ...
end

Or even use a named scope :

1
2
3
with_exclusive_scope(Item.red) do
  ...
end

That’s all. Please open a lighthouse ticket if you find a bug or have a patch for an improvement!

UPDATE 1 : Added information about deprecating scoped_by_ dynamic methods.

UPDATE 2 : Added information about deprecating default_scope with finder options.

Websockets made easy with Cramp 8

Posted by pratik
on Friday, January 15

If you aren’t aware already, HTML5 has Websockets API – enabling bidirectional communication between client and server. You should check Ilya’s post Ruby & WebSockets: TCP for the Browser for a better explanation.

Starting with version 0.9, Cramp has in built support for Websockets. Unlike other solutions, Cramp extends the underlying webserver ( thin or Rainbows! ) to add the websockets superpower. As websockets protocol is an extension of the HTTP protocol, hopefully the webservers will be supporting it out of the box once websockets are well established.

For working with websockets, Cramp::Controller provides a class called Cramp::Controller::Websocket. Cramp::Controller::Websocket is very similar to Cramp::Controller::Action. If you don’t know much about Cramp::Controller::Action, you could read my introductory post on cramp.

As Cramp extends the underlying webserver to support websockets, you must tell it which webserver you are using :


Cramp::Controller::Websocket.backend = :thin # or :rainbows

Cramp::Controller::Websocket adds a new callback called on_data in addition to the functionalities available to Cramp::Controller::Action. Whenever the client sends any data over the websocket, Cramp will call the specified on_data callbacks, passing client supplied data as the lone argument. To send any data to the client over the websocket, you could use render() as usual.

Here’s a very basic example using Cramp’s websockets, which echoes everything received over the socket :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
require 'rubygems'
require 'cramp/controller'

Cramp::Controller::Websocket.backend = :thin

class WelcomeController < Cramp::Controller::Websocket
  on_data :received_data

  def received_data(data)
    render "Got your #{data}"
  end
end


Rack::Handler::Thin.run WelcomeController, :Port => 3000

Demo

I have ported Jetty’s Websocket example to Cramp. You can find the code at http://gist.github.com/273794. It’s currently running at http://cramp.in:8080, however I wouldn’t bring it up once it dies of natural causes :) Make sure you open the demo in Google Chrome or Webkit. On a related note, Firefox should have websockets support soonish.

But what about the fucking Internet Explorer ?!

As you might have guessed, no such luck as websockets support for IE. But not all is lost. You can very easily fallback to flash sockets if the browser doesn’t have websockets support. Check out web-socket-js library by Hiroshi Ichikawa which does exactly that.

Fabio’s Demo Application

Fabio Akita has written an excellent demo application for Cramp – http://github.com/akitaonrails/cramp_chat_demo. The chat application has two versions : long polling and websockets. The websockets version even falls back to flash socket if the browser doesn’t provide the support. It’s an excellent example of how you should write cramp applications, and rack applications in general.

Introducing Cramp 17

Posted by pratik
on Thursday, January 07

Cramp is the latest entry on the ruby web frameworks list. However, unlike all the others, Cramp is an asynchronous framework, always running inside EventMachine reactor loop. Cramp isn’t a good fit for most of the web applications out there. However, Cramp is good at holding and working with a large number of open connections. Hence it’ll work great for things like comet, long polling, streaming API or even when your application needs to handle thousands of concurrent connections.

This article assumes that you’re aware with the evented programming model. If you are not, things below this point might not make much of a sense. If you’re interested in learning, you could start by reading about EventMachine and Twisted.

Install

As Cramp requires 0.2.0-pre Arel and 3.0-pre versions of ActiveSupport and ActiveModel. So you’ll have to install them first. This step will be irrelevant after Rails 3 gets released. But for now, the following should install them :

gem install arel --pre
gem install activemodel --pre

And then,

gem install cramp

That’ll install Cramp gem along with all the needed dependencies. Please note that Cramp depends on ActiveSupport 3.0-pre gem, which isn’t backwards compatible and this may affect any other gems requiring ActiveSupport without specifying the version number.

Two faces of Cramp

Cramp comes with two layers : Controller & Model.

1) Cramp::Controller

Cramp::Controller is an asynchronous controller layer, that tries being rack compliant as much as possible. Currently, you must use thin or Rainbows! in order to run it. If you’re using Rainbows!, make sure you use EventMachine concurrency model.

Here’s the “hello world” of Cramp::Controller:

1
2
3
4
5
6
7
8
9
10
11
12
13
require 'rubygems'
require 'cramp/controller'

class WelcomeAction < Cramp::Controller::Action
  on_start :send_hello_world

  def send_hello_world
    render "Hello World"
    finish
  end
end

Rack::Handler::Thin.run WelcomeAction, :Port => 3000

Now run it directly :

ruby welcome_action.rb

Here WelcomeAction is the rack endpoint, which you can use with config.ru or what have you.

Now let’s dig dipper to understand the code snippet above.

When serving a request, a Cramp::Controller::Action object goes through the following four stages :

  • Initialization – This is the initial stage of an action when a request is received. During this stage, it’s possible to abort the request along with whatever headers/body you wish to send. Hence, it is typically useful for checking permissions and validating the request etc.
  • Response Initialization – If the request doesn’t get aborted during the initialization stage, Cramp::Controller::Action enters the second stage, where response headers and a deferrable body are sent to the client. It’s important to note that as the headers are already sent here, following stages will only be able to send body and not able to change the headers.
  • Starting – This is where the actual work happens. From this stage, you can send body to the clients as many times as you wish to or finish the request.
  • FinishingCramp::Controller::Action enters this stage if the request is marked as finished during the Starting stage or if the client closes the connection. This stage is useful for cleanup activities or anything else you may wish to do upon request completion.

To hook into each of these states, Cramp::Controller::Action provides the following callbacks/methods :

before_start

before_start callback provides hook into the initialization stage. Each before_start callback accepts a block and must call yield ( or block.call – whichever is appropriate ) or halt. Here’s what a typical usage looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
before_start :verify_id, :find_user

def verify_id
  if params[:id] !~ /\A\d+\z/
    halt 500, {'Content-Type' => 'text/plain'}, "Invalid ID"
  else
    yield
  end
end

def find_user
  User.where(User[:id].eq(params[:id])).first do |user|
    if @user = user
      yield
    else
      halt 404, {}, "User not found"
    end
  end
end

As you can see above, halt takes status, headers and body as parameters and sends them to the client. It would also halt the callback chain and the request itself. Please note that on_finish callbacks will not get run when you halt. Calling yield will continue the filter chain or enter the next stage if no filters are left to run.

respond_with

After the before_start stage, Cramp::Controller::Action enters the next stage of Response Initialization. It’ll call the method respond_with, which must return an array of [status, headers]. If this method is not defined, [200, {‘Content-Type’ => ‘text/html’}] status and headers will be used by default.

Here’s an example of using respond_with to send custom status and headers :

1
2
3
4
def respond_with
  content_type = params[:format] == 'xml' ? 'application/xml' : 'application/json'
  [200, {'Content-Type' => content_type}]
end

A deferrable body is also sent out along with these headers.

on_start

After all the verifications and sending out headers, the real work starts. on_start callbacks provide multiple entry points into the Starting stage. These callbacks can send any body to the client using render() method or finish the request by invoking finish(). Note that the render() method can be called any number of times.

Here’s a full example imitating running two long running sql queries from on_start callbacks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
require 'rubygems'
require 'cramp/controller'
require 'cramp/model'

Cramp::Model.init(:username => 'root', :database => 'arel_development')

class SleepingAction < Cramp::Controller::Action
  on_start :start_sleeping

  def start_sleeping
    Cramp::Model.select 'select sleep(1)', method(:on_first_sleep)
  end

  def on_first_sleep
    render "First Sleep Complete...\n"
    render "Going to sleep one more time now..\n"

    Cramp::Model.select 'select sleep(1)', method(:on_second_sleep)
  end

  def on_second_sleep
    render "Second Sleep Complete. Time to finish!\n"
    finish
  end
end

Rack::Handler::Thin.run SleepingAction, :Port => 3000

Now if you hit this using curl:

[lifo@null ~]$ curl http://0.0.0.0:3000/
First Sleep Complete...
Going to sleep one more time now..
Second Sleep Complete. Time to finish!
[lifo@null ~]$

on_finish

on_finish provides hook into the last stage : Finishing. These callbacks are run when you call finish() from an on_start callback or when the client closes the connection. These callbacks are the perfect place for any cleaning up activities.

Here’s an example of a Cramp::Controller::Action using periodical timer and an on_finish callback.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
require 'rubygems'
require 'cramp/controller'
require 'cramp/model'

class PollUserAction < Cramp::Controller::Action
  on_start :start_polling
  on_finish :stop_polling

  def start_polling
    # Call find_user every 10 seconds
    @timer = EventMachine::PeriodicTimer.new(2) { find_user }
  end

  def find_user
    User.first {|u| render "#{u.inspect}\n" }
  end

  def stop_polling
    puts "Cancelling the timer.."
    @timer.cancel
  end
end

Rack::Handler::Thin.run PollUserAction, :Port => 3000

In the case above, if you don’t call @timer.cancel, it’ll keep running even after the client closes the connection. The following section will cover the helper method provided by Cramp::Controller for the above pattern called periodic_timer.

Helper Methods

Periodic Timers

The on_finish callback example above has a very common pattern : Starting periodic timers using on_start and cleaning them up using on_finish. Cramp::Controller provides a better alternative : periodic_timer

Using periodic_timer to rewrite the example above :

1
2
3
4
5
6
7
class PollUserAction < Cramp::Controller::Action
  periodic_timer :poll_user, :every => 2

  def poll_user
    User.first {|u| render "#{u.inspect}\n" }
  end
end

And then Cramp::Controller will take care of starting and stopping EventMachine::PeriodicTimer from the appropriate stages.

You can use more than one periodic timers as well :

1
2
3
4
5
6
7
8
9
10
11
12
13
class PollUserAction < Cramp::Controller::Action
  periodic_timer :poll_user, :every => 2
  periodic_timer :check_limit_exceed, :every => 10

  def poll_user
    ..
  end

  def check_limit_exceed
    finish if request_limit_exceeded
  end

end

In the example above, check_limit_exceed() calls finish() if the request limit is exceeded, which in turn will terminate the connection and stop all the timers too.

Keeping Connection Alive

If you’re using Cramp for streaming or long polling, you’d want to make sure the client doesn’t close the connection prematurely. Cramp::Controller has a handy helper method to make sure that doesn’t happen – keep_connection_alive

keep_connection_alive sends the client an empty string (” “) every 15 seconds by default.

1
2
3
4
class PollUserAction < Cramp::Controller::Action
  periodic_timer :poll_user, :every => 2
  keep_connection_alive
end

Or you can change the period by supplying :every option :

1
2
3
4
class PollUserAction < Cramp::Controller::Action
  periodic_timer :poll_user, :every => 2
  keep_connection_alive, :every => 30
end

Routing, Request and Parameters

Cramp::Controller is capable of using a Rack middlewares for routing request and populating params. Here’s an example of a Cramp::Controller using Usher for routing :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
require 'rubygems'
require 'cramp/controller'

class HomeController < Cramp::Controller::Action
  on_start :send_hello_world

  def send_hello_world
    render "Hello World with #{params[:id]}"
    finish
  end
end

routes = Usher::Interface.for(:rack) do
  add('/:id').to(HomeController)
end

Rack::Handler::Thin.run routes, :Port => 3000

Cramp::Controller::Action implements the params method like:

1
2
3
def params
  @params ||= @env['usher.params']
end

If you’re using another rack based router like rack-mount, you might want to override that with the appropriate definition.

As Cramp::Controller tries being a Rack based framework, you should be able to use Rails middlewares for cookies and sessions stores.

Views

Cramp::Controller does not have any built in mechanism for rendering files. However, using something like tilt should be very straight forward.

Testing

Cramp::Controller comes with a few helper methods to write integration tests for the application. However, that’s not within the scope of this article. You should have a read through http://github.com/lifo/cramp/tree/master/test/controller to know more.

Benchmarks

I haven’t really benchmarked Hello World for Cramp::Controller. However it should be very fast. I was able to have a single Cramp::Controller instance stream to 2000+ concurrent connections just fine. And this is where Cramp::Controller really shines, and not some silly Hello World masturbations.

Also, note that OS X isn’t the best environment for testing concurrent connections, you should be using Linux with a tuned Kernel for this. Check this article for details on tuning the Kernel.

Gotchas

Even though Cramp::Controller tries being as much Rack compliant as possible, it’s not a 100% Rack compliant framework. Rack specs are designed primarily for synchronous frameworks. As Cramp::Controller uses deferrable body, any middlewares operating on the response body will not work without modifications.

2) Cramp::Model

Cramp::Model is an asynchronous ORM ( only MySQL supported at the moment ) built on top of ARel and ActiveModel. It’s currently in a very primitive state and provides the following features :

  • Validations using ActiveModel
  • CRUD operations
  • AR/ARel like Finder methods

Here’s what a Cramp model looks like :

1
2
3
4
5
6
7
8
9
10
11
require 'rubygems'
require 'cramp/model'

Cramp::Model.init(:username => 'root', :database => 'cramp_development')

class User < Cramp::Model::Base
  attribute :id, :type => Integer, :primary_key => true
  attribute :name

  validates_presence_of :name
end

You are required to declare all the columns you wish to use as Attributes. If type is not supplied, String type is assumed.

Currently Cramp::Model provides the following finder method :

  • all
  • first
  • each

It also provides the following methods for specifying options on the find :

  • where
  • select
  • group
  • order
  • limit
  • offset

The above methods are delegated to Arel and can be chained. As Cramp::Model is an asynchronous ORM, you must supply a callback method for processing the result records. You could do that by calling all, first or each at the end of the chain and supply the callback as a block or a method.

Here are some example usages :

1
2
3
4
5
6
7
8
9
10
11
EM.run do
  User.select(:id, :name).limit(10).offset(20).first {|u| .. }
  User.where(User[:name].eq('Lush')).limit(2).all {|users| ... }
  User.each {|user| .. }

  def lush_users_found(users)
    ...
  end

  User.where(User[:name].eq('Lush'), User[:id].gt(5)).all method(:lush_users_found)
end

For the basic CRUD operations, Cramp::Model provides methods similar to Model#save, which accepts an optional callback. If a callback is provided, it’ll be called after the completion with a Status object, containing metadata about the success or failure of the save operation. Status object has just two methods defined on it : record & success?.

Here’s how you would typically want to save a record to the database :

1
2
3
4
5
6
7
8
9
10
11
12
13
EM.run do
  def user_saved(status)
    if status.success?
      ...
    else
      user = status.record
      puts "Oops. Found errors : #{status.record.errors.inspect}"
    end
  end

  user = User.new
  user.save method(:user_saved)
end

Contributing to Cramp

Cramp is hosted on Github http://github.com/lifo/cramp. Please do fork and use the issues if you encounter a bug or need any kind of help.

UPDATE 1 : Add instructions to install arel 0.2.0-pre

UPDATE 2 : Rainbows! work with cramp!

Railssummit Slides 3

Posted by pratik
on Sunday, October 18

Here are the slides of my presentation at Railssummit 2009. Huge thanks to Locaweb and Fabio Akita for organizing the conference and having me there.

My talk was about Rails focused tips/tricks.

Or you can download the pdf from here.

Make your shoulda tests faster with fast_context 24

Posted by pratik
on Sunday, September 20

I’ve been using Shoulda for a while. And for my current project, I decided to go fixtureless with Shoulda + Factory Girl. All good, except one problem. Slow as fuck tests. So here’s fast_context as a solution for it. fast_context compiles all the ‘should’s within a context into a single test.

For example :

1
2
3
4
5
6
7
8
9
10
11
class UsersControllerTest < ActionController::TestCase
  fast_context "#new" do
    setup do
      @user = Factory(:user)
      get :new
    end

    should_render_template :new
    should_render_with_layout :users
  end
end

This will be compiled into something like :

1
2
3
4
5
6
7
8
class UsersControllerTest < ActionController::TestCase
  test "test: #new should run_fast"
    @user = Factory(:user)
    get :new
    should_render_template :new
    should_render_with_layout :users
  end
end

As your setup method just once per fast_context, the tests run much faster.

To install :


script/plugin install git://github.com/lifo/fast_context.git

This was a result of a short time of hacking. So there may be bugs. Please comment here if you run into any issues.

save! > save 12

Posted by pratik
on Friday, August 07

Thoughtbot folks have a great article on not expecting exceptions – save bang your head, active record will drive you mad. I’ll admit, just like the poster, I used to use save! in controllers to DRY my code. And have a global rescue_from in application.rb. But over the time, I changed the camp and now I’m fully in that “Don’t expect expectations” camp. Some things are more important that DRYing 3 lines of code.

But I’d want to take this a step further. When you’re not expecting something to fail, always use the methods that raise exceptions on failure.

So I strongly disagree with the poster on this :

I think ActiveRecord::Base#save! and ActiveRecord::Base.update_attributes! should be pulled from the public API

I would advocate just the opposite for certain cases. In many of the code reviews we’ve done via ActionRails, the following pattern was seen in many of the models :

1
2
3
4
5
6
7
8
def do_something
  self.foo = 'bar'
  save
end

def create_items
  names.each {|n| self.items.create :name => n }
end

In the snippets above, it’s not checking for cases where the save fails. And for good reasons that they’re not likely to fail as code is changing some very minor. But in these scenarios, a failure would be an unexpected situation. Hence you should always use save! or create!.

There could be easily be any unexpected reasons the above save could fail. And using save! protects you from those situations and help catch those minor programming mistakes early, which otherwise could prove to be very costly in terms of time/efforts. So the above code should really be :

1
2
3
4
5
6
7
8
def do_something
  self.foo = 'bar'
  save!
end

def create_items
  names.each {|n| self.items.create! :name => n }
end

However, if you’re using exceptions for flow control, this practise won’t always help you :

1
2
3
4
5
6
7
def create
  @user = User.create! params[:user]
  redirect_to @user
rescue ActiveRecord::RecordNotSaved
  flash[:notice] = 'Unable to create user'
  render :new
end

As this catches the exception ActiveRecord::RecordNotSaved, unexpected save! failures from your model methods/callbacks will get caught too. And hide the real error.

Moral of the story :

  • Don’t expect exceptions
  • Use methods throwing exceptions when you’re not expecting a failure. For example, everywhere you’re not checking if save or create fails when working with Active Record objects, always use save! and create! instead.

USE INDEX with Active Record finders 6

Posted by pratik
on Thursday, August 06

MySQL doesn’t always pick the right index for your queries. Hence, sometimes you must tell it which index to use. Consider the example :


Activity.all(:conditions => ['created_at >= ? AND country_id = ?', 10.days.ago, 79])

Running EXPLAIN on the above query :

EXPLAIN SELECT * FROM `activities` WHERE (created_at >= '2009-07-27 12:58:44' AND country_id = 79);

Possible keys : index_activities_on_created_at,index_activities_on_created_at_and_country_id
Using the key : index_activities_on_created_at

As you can see, even though the table has index on both the fields involved in the query – index_activities_on_created_at_and_country_id, MySQL still uses index_activities_on_created_at. You can explicitly ask MySQL to use the index you want by supplying USE INDEX

1
2
SELECT * FROM `activities` USE INDEX(index_activities_on_created_at_and_country_id) 
  WHERE (created_at >= '2009-07-27 12:58:44' AND country_id = 79);

Active Record does not have any finder option to specify the index hint. Hence the solution is to exploit the :from option :

1
2
3
from = "#{quoted_table_name} USE INDEX(index_activities_on_created_at_and_country_id)"
Activity.all(:from => from, 
             :conditions => ['created_at >= ? AND country_id = ?', 10.days.ago, 79])

Ruby I don't like #3 - Object#freeze 15

Posted by pratik
on Wednesday, August 05

Object#freeze annoys me. Not a lot, but enough to bitch blog about it. So, freeze lets you make sure no one else modifies your precious little object :

1
2
3
4
5
6
>> a = "hello"
>> a.freeze
>> a << "wtf"
TypeError: can't modify frozen string
  from (irb):23:in `<<'
  from (irb):23

However, freeze does not protect the variable. It only protects the value.

1
2
3
4
>> a = "hello"
>> a.freeze
>> a += "wtf"
=> "hellowtf"

The weird behaviour is even more visible when you’re dealing with arrays :

1
2
3
4
5
6
7
8
9
>> x = ["hello", "freedom"]
>> x.freeze
>> x << "world"
TypeError: can't modify frozen array
  from (irb):6:in `<<'
  from (irb):6
>> x[0] << "wtf"
>> x
=> ["hellowtf", "freedom"]

It’s even weird with hashes :

1
2
3
4
5
6
>> a = {:x => 1}
>> a.freeze
>> a[:x] = 2
TypeError: can't modify frozen hash
  from (irb):11:in `[]='
  from (irb):11

However, above are not really the primary reasons I don’t like freeze. It’s the fact that you cannot unfreeze an object without using something like evil.rb. And this goes against a lot of things Ruby stands for in my book. Ruby is never about defensive programming. Even where it tries to save you from yourself, there are always proper ways you can overcome the restriction. For example, private methods and send. If you want to restrict programmers, Ruby is not for you. Use Java/Python/whatever. Not Ruby. Ruby is not meant for preventing idiots from shooting their leg.

Taking a real example :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
module ActiveSupport
  module Testing
    module Performance
      DEFAULTS =
        if benchmark = ARGV.include?('--benchmark')  # HAX for rake test
          { :benchmark => true,
            :runs => 4,
            :metrics => [:process_time, :memory, :objects, :gc_runs, :gc_time],
            :output => 'tmp/performance' }
        else
          { :benchmark => false,
            :runs => 1,
            :min_percent => 0.01,
            :metrics => [:process_time, :memory, :objects],
            :formats => [:flat, :graph_html, :call_tree],
            :output => 'tmp/performance' }
        end.freeze

Here’s a code from Rails performance tests. As you can see, it defines a hash with config variables based on benchmark or profile mode. And then freezes the hash. Assuming you’re benchmarking, DEFAULTS[:runs] determines how many times Rails should run the test in a loop :


DEFAULTS[:runs].times { run_test_with_benchmarking "whatever test" }

Now many times when I’m benchmarking and want to increase the number of times a test is ran, I just want to do something like :

1
2
3
4
5
6
7
class SpeedTest < ActionController::PerformanceTest
  DEFAULTS[:runs] = 1000

  def test_some_method
    Model.some_method
  end
end

However, that’s not possible thanks to the freeze. I do know that changing DEFAULTS[:runs] is not a public API yada yada yada. But it’s Ruby and I’ll change whatever the fuck I want to. I can understand certain cases where people use freeze to prevent silly errors. That’s probably OK to a certain extent. But remember, if you design your software for idiots, only idiots will use it.

Ruby I don't like #2 - catch(:wtf) { throw :wtf } 8

Posted by pratik
on Tuesday, August 04

The 1960s and 1970s saw computer scientists move away from GOTO statements in favor of the structured programming programming paradigm. Some programming style coding standards prohibit use of GOTO statements. – Wikipedia

Ruby takes the whole GOTO nonsense to an entirely new heights. Ruby’s version of GOTO/LABEL is called throw/catch. The lunacy goes further as Ruby’s throw is equivalent to GOTO with a return value.

1
2
3
4
5
6
def hello
  throw :done, "wtf"
end

catch(:done) { hello }
=> "wtf"

Not only it makes the flow control hard to follow, it also shows your lack of fundamental programming skills. I’d love to see a case where you use throw/catch because there’s no other way. Only place I’ve ever used throw/catch is in my evil middleware Rack::Evil. And the name says it all.

Let’s take a real example from Rails :

1
2
3
4
5
6
7
8
def find_with_associations(options = {})
  catch :invalid_query do
    join_dependency = JoinDependency.new(self, merge_includes(scope(:find, :include), options[:include]), options[:joins])
    rows = select_all_rows(options, join_dependency)
    return join_dependency.instantiate(rows)
  end
  []
end

Just by looking at this method, you’ll have absolutely no idea who’s gonna be throwing :invalid_query. It could be any method subsequently called while the block is being executed. Only way to know is by doing a global search for throw :invalid_query.

Rails uses throw/catch here because it wants to return an empty array when something somewhere goes wrong. And the thing that can possibly go wrong is so deep down inside, throw/catch provides an easy way out without much refactoring. However, easy is not always the best way or the proper way.

If we look at the relevant code from the involved methods :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
def select_all_rows(options, join_dependency)
  connection.select_all(
    construct_finder_sql_with_included_associations(options, join_dependency),
    "#{name} Load Including Associations"
  )
end

def construct_finder_sql_with_included_associations(options, join_dependency)
  scope = scope(:find)
  sql = "SELECT #{column_aliases(join_dependency)} FROM #{(scope && scope[:from]) || options[:from] || quoted_table_name} "

  ....
  if !using_limitable_reflections?(join_dependency.reflections) && ((scope && scope[:limit]) || options[:limit])
    add_limited_ids_condition!(sql, options, join_dependency)
  end
  ....

  sanitize_sql(sql)
end

def add_limited_ids_condition!(sql, options, join_dependency)
  unless (id_list = select_limited_ids_list(options, join_dependency)).empty?
    sql << "#{condition_word(sql)} #{connection.quote_table_name table_name}.#{primary_key} IN (#{id_list}) "
  else
    throw :invalid_query
  end
end

This doesn’t seem that bad on the first look. But think again. Apart from the control flow retardness, the method add_limited_ids_condition adds an extra responsibility to the caller – catching invalid_query. And this is very easy to miss too – as seen with the very same method in question here – calculations.rb. Add a few of more throw/catch and you get a proper spaghetti code.

I think the better way to write the above code is :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
def find_with_associations(options = {})
  join_dependency = JoinDependency.new(self, merge_includes(scope(:find, :include), options[:include]), options[:joins])
  rows = select_all_rows(options, join_dependency)
  rows ? join_dependency.instantiate(rows) : []
end

def select_all_rows(options, join_dependency)
  finder_sql = construct_finder_sql_with_included_associations(options, join_dependency)
  connection.select_all(finder_sql, "#{name} Load Including Associations") if finder_sql
end

def construct_finder_sql_with_included_associations(options, join_dependency)
  ....
  limitable = !using_limitable_reflections?(join_dependency.reflections) && ((scope && scope[:limit]) || options[:limit])

  unless limitable && add_limited_ids_condition!(sql, options, join_dependency).blank?
    ....
    sanitize_sql(sql)
  end
end

def add_limited_ids_condition!(sql, options, join_dependency)
  id_list = select_limited_ids_list(options, join_dependency)
  sql << "#{condition_word(sql)} #{connection.quote_table_name table_name}.#{primary_key} IN (#{id_list}) " if id_list.present?
end

I’d normally say that you should be flexible about following such rules about using a pattern or not using some. But this is an exception. Using throw/catch is just fucking wrong. Plain and simple.

Ruby I don't like #1 - Explicit 'return' 14

Posted by pratik
on Monday, August 03

In Ruby, you don’t have to specify an explicit return value from a method. Ruby will just return the last evaluated statement. Similarly, if an explicit return statement will make itself the last evaluated statement – i.e return control to the caller with the specified return value.

However, I’m not a big fan of explicit return statements. In my experience, the only place where they make sense is in the first line of the method, where the control is returned to the caller if the supplied arguments are not valid/expected. Consider the following method :

1
2
3
4
def read(file_name, options = nil)
  return nil unless File.exist?(file_name)
  ....
end

I think the above is the only case where I feel it’s ok to use an explicit return as it’s much better than the alternative – wrapping the entire method in a big if block. Also, you don’t really need to specify nil. The above can be rewritten as :

1
2
3
4
def read(file_name, options = nil)
  return unless File.exist?(file_name)
  ....
end

Now the real problem is visible when you look at the full method :

1
2
3
4
5
6
7
8
9
def read(file_name, options = nil)
  return nil unless File.exist?(file_name)

  if expires_in(options) > 0
    return nil
  end

  File.open(file_name, 'rb') { |f| Marshal.load(f) }
end

A much simpler version of the above method is :

1
2
3
4
5
def read(file_name, options = nil)
  if File.exist?(file_name) && expires_in(options) <= 0
    File.open(file_name, 'rb') { |f| Marshal.load(f) }
  end
end

Of course there’s no one ring to rule them all. It might be desirable to use multiple returns in a method. But every time you do that, take a moment to make sure it’s making the code easier to read.

Stop returning false from your before filters 3

Posted by pratik
on Friday, July 31

In the past version of Rails you had to explicitly return false from before filters to halt the filter chain and make sure the action doesn’t get run. The code looked somewhat like :

1
2
3
4
5
6
7
8
9
10
11
12
class AdminController < ApplicationController
  before_filter :check_admin

  private

  def check_admin
    unless current_user.admin?
      redirect_to home_path
      return false
    end
  end
end

But since Rails 2.0, If you call render, head or redirect_to from a before_filter, the filter chain will be halted. So the above controller will now look like :

1
2
3
4
5
6
7
8
9
class AdminController < ApplicationController
  before_filter :check_admin

  private

  def check_admin
    redirect_to home_path unless current_user.admin?
  end
end

You should go ahead and update all your before filters to reflect this change. return false is nothing but code smell.

Running Rails performance tests on real data 5

Posted by pratik
on Wednesday, July 29

It’s not quite straight forward to run Rails performance tests on real production data. By default, performance tests will use the test database and wipe it before using it, thus making it impossible to put real data in the test database.

To run performance tests on real data:

  • Create a file called performance_test_helper.rb inside your test/ directory with the following:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# START : HAX HAX HAX
# Load Rails environment in 'test' mode
RAILS_ENV = "test"
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
# Re-establish db connection for 'performance' mode
silence_warnings { RAILS_ENV = "performance" }
ActiveRecord::Base.establish_connection
# STOP : HAX HAX HAX

require_dependency 'application'

require 'test/unit'
require 'active_support/test_case'
require 'action_controller/test_case'
require 'action_controller/integration'

require 'performance_test_help'
  • Replace the following lines in performance tests:
1
2
require 'test_helper'
require 'performance_test_help'

with


require 'performance_test_helper'
  • Supply configuration for ‘performance’ environment in your database.yml:
1
2
3
4
5
6
7
8
performance:
  adapter: mysql
  encoding: utf8
  database: database_with_real_data
  pool: 5
  username: root
  password:
  socket: /tmp/mysql.sock

The above steps will also make sure that the performance tests use the database database_with_real_data. Also, the database will not get wiped on every run. You’ll also have to ensure the test database – database_with_real_data – is up-to-date with the schema changes.

Now you can just run your performance tests using the usual rake tasks:

1
2
$ rake test:benchmark
$ rake test:profile

Nested Layouts 10

Posted by pratik
on Tuesday, July 07

Don’t think the nested layouts get any simpler in Rails.

1
2
3
4
5
6
module ApplicationHelper
  def parent_layout(layout)
    @content_for_layout = self.output_buffer
    self.output_buffer = render(:file => "layouts/#{layout}")
  end
end

Now using the parent_layout helper method inside your layouts for nesting :

1
2
3
4
5
# items.html.erb
<h1>Just my items</h1>
<%= yield %>

<% parent_layout 'master' %>

and the parent layout of items – the master layout :

1
2
3
4
5
6
7
# master.html.erb
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head><title>Hello World</title></head>
  <body>
    <%= yield %>
  </body>
</html>

Now, the items layout will always be wrapped under the master layout. Just make sure that the parent_layout call is always on the last line using <%. This technique also works for nesting deeper than a single level.