Rails / Sinatra / Metal Shootout
I’m currently in Las Vegas attending RailsConf 2009. This morning I heard Heroku’s Adam Wiggins give an excellent overview of Rails Metal, Rack and Sinatra.
Some time ago now, Rails adopted Rack as its middleware layer. For those not in the know (myself included before Adam’s talk), according to RailsGuides Rack is:
Rack provides a minimal, modular and adaptable interface for developing web applications in Ruby. By wrapping HTTP requests and responses in the simplest way possible, it unifies and distills the API for web servers, web frameworks, and software in between (the so-called middleware) into a single method call.
Metal is essentially a thin wrapper around the Rack middleware layer of Rails. Why is this important? Well, by dropping down into Metal it’s possible to completely bypass the Rails framework and squeeze the absolute maximum performance out of your stack. Specifically this can be useful for a common request to your application where the response time is crucial and you want to avoid the extra overhead of passing through the Rails routing mechanism before serving a response.
» The Test
OK, so enough talk, what do the numbers look like? This is by no means meant to be a thorough test of all possible permutations of Rails, Sinatra and Metal, rather I’m interested in replacing a simple API method with a Sinatra application and a Metal endpoint. The API I’m testing is the Galaxy Zoo API layer. Within the Galaxy Zoo API we have the concept of ‘Assets’. An Asset is something like a SDSS galaxy image and a frequently accessed API url looks like:
http://api_url/api/assets/:id
This API call returns a simple XML snippet that looks something like this:
<?xml version="1.0" encoding="UTF-8"?>
<asset>
<id>1</id>
<location>http://s3.aws.com/1.jpg</location>
<project_id>1</project_id>
<external_ref></external_ref>
</asset>
I used Apache benchmark to test each option. Passenger 2.2.2 / Rails 2.3.2 and my MacBook Pro (2.53 GHz) were used to serve the application. Also, to ensure a reasonably fair test I rebooted the OS for each variant and ‘warmed up’ Apache by running the test 4 times before taking the benchmark results from the 5th and final time I issued this command:
ab -n 1000 -c 4 http://api_url/api/assets/1
This is basically making 1000 requests with 4 concurrent connections.
» The Results
OK, so first up I used a standard Rails controller action, the code for which is shown below:
def show
@asset = Asset.find(params[:id])
respond_to do |format|
format.xml { render :xml => @asset.to_xml }
end
end
This came out at a very reasonable 230 requests per second:
Requests per second: 229.64 [#/sec] (mean)
Next up I added in a Sinatra ‘application’ to respond to the api/assets/:id url. Because of the way that Rails uses Rack, the Sinatra/Metal endpoints are picked up before the Rails routing mechanism kicks in, no modification is therefore required to the routes.rb config to make the Sinatra application pick up the request url.
By default Sinatra/Metal endpoints are picked up if they are placed in the RAILS_ROOT/app/metal/ and have a class name that represents the filename for the Sinatra application:
RAILS_ROOT/app/metal/sinatra_asset.rb
require 'sinatra'
class SinatraAsset < Sinatra::Application
set :environment, 'production'
get '/api/assets/:id' do
Asset.find(params[:id]).to_xml
end
end
Benchmarking Sinatra produces the following results:
Requests per second: 416.61 [#/sec] (mean)
Wow! So we’ve gone from ~230 requests per second using a standard Rails controller action up to over 400 requests per second using Sinatra. This is obviously a pretty serious speed bump and for really not very much work.
Finally I tested a Metal endpoint to intercept the same request url. Once again, Metal endpoints need to be installed in:
RAILS_ROOT/app/metal/metal_asset.rb
class MetalAsset
def self.call(env)
url_pattern = /^\/api\/assets\/(\d+)$/
if m = env['PATH_INFO'].match(url_pattern)
asset = Asset.find(m[1])
[ 200, {"Content-Type" => "text/xml"}, asset.to_xml]
else
[ 404, {}, '']
end
end
end
So Sinatra was fast - how fast is Metal? Well it’s pretty nippy:
Requests per second: 522.12 [#/sec] (mean)
» Conclusions
As I mentioned earlier, this is by no means meant to be a through test of how Rails controller actions perform compared to their Sinatra and Metal equivalents, however the numbers are pretty spectacular: a bare Metal endpoint more than doubles the number of requests this application can handle per second. This is not to say that the Sinatra results weren’t pretty damn good too - using Sintara gave an 80% speed boost for this simple API request.
It seems clear that a significant speed boost can be had by getting down to ‘the metal’. Personally I prefer the clear syntax of Sinatra over the URL regex that Metal requires to achieve the same result, although the additional ~100 requests per second that Metal offers over Sinatra is hard to ignore.
David Heinemeier Hansson talked this week about the refactoring that’s going on with the Rails routing mechanism for the upcoming Rails 3 release so it’s possible that these numbers could significantly change when Rails 3 makes it into the wild. For now though, if you’ve got a Rails application with a frequently accessed url, drop in a Sinatra application or a Metal end point and watch it fly!
