Tuesday, July 29, 2008

Ruby Unit Converting Hash

I'm currently working on a project where I need to convert from things in one set of units to any other set of units ( eg centimeters to inches and so forth)

I had a bunch of small helper functions to convert from X to Y, but these kept growing every time we needed to handle something which hadn't been anticipated.

This kind of thing is also exponential, as if we have 4 'unit types' and we add a 5th one, we need to add 8 new methods to convert each other type to and from the new type

A few hours of refactoring later, I have this, which I think is kind of cool, and will enable me to delete dozens of small annoying meters_to_pts methods all over the place.

Disclaimer: This is definitely not good OO. A hash is not and never should be a unit converter. In the production code I will refactor this to build an actual Unit Converter class which stores a hash internally :-)


# Builds a unit converter object given the specified relationships
#
# converter = UnitConverter.create({
#  # to convert FROM a TO B, multiply by C
#  :pts    => {:inches => 72},
#  :inches => {:feet   => 12},
#  :cm     => {:inches => 2.54, 
#              :meters => 100},
#  :mm     => {:cm     => 10},
# })
#
# You can then do
#
# converter.convert(2, :feet, :inches) 
# => 24
#
# The interesting part is, it will follow any links which can be inferred
# and also generate inverse relationships, so you can also (with the exact same hash) do
#
# converter.convert(2, :meters, :pts) # relationship inferred from meters => cm => inches => pts
# => 5669.29133858268
#
class UnitConverter < Hash
  
  # Create a conversion hash, and populate with derivative and inverse conversions
  def self.create( hsh )
    returning new(hsh) do |h|
      # build and merge the matching inverse conversions
      h.recursive_merge! h.build_inverse_conversions
      
      # build and merge implied conversions until we've merged them all
      while (convs = h.build_implied_conversions) && convs.any?
        h.recursive_merge!( convs )
      end
    end
  end
  
  # just create a simple conversion hash, don't build any implied or inverse conversions
  def initialize( hsh )
    merge!( hsh )
  end
  
  # Helper method which does self.inject but flattens the nested hashes so it yields with |memo, from, to, rate|
  def inject_tuples(&block)
    h = Hash.new{ |h, key| h[key] = {} }
    
    self.inject(h) do |m, (from, x)|
      x.each do |to, rate|
        yield m, from, to, rate
      end
      m
    end
  end
  
  # Builds any implied conversions and returns them in a new hash
  # If no *new* conversions can be implied, will return an empty hash
  # For example
  # {:mm => {:cm => 10}, :cm => {:meters => 100}} implies {:mm => {:meters => 1000 }}
  # so that will be returned
  def build_implied_conversions
    inject_tuples do |m, from, to, rate|
      if link = self[to]
        link.each do |link_to, link_rate|
          # add the implied conversion to the 'to be added' list, unless it's already contained in +self+,
          # or it's converting the same thing (inches to inches) which makes no sense
          if (not self[from].include?(link_to)) and (from != link_to)
            m[from][link_to] = rate * link_rate 
          end
        end
      end
      m
    end
  end
  
  # build inverse conversions
  def build_inverse_conversions
    inject_tuples do |m, from, to, rate|
      m[to][from] = 1.0/rate
      m
    end
  end
  
  # do the actual conversion
  def convert( value, from, to )
    value * self[to][from]
  end
end

I'm not sure if deriving it from Hash is the right way to go, but it basically is just a big hash full of all the inferred conversions, so I'll leave it at that.


Update

Woops, this code requires 'returning' which is part of rails' ActiveSupport, and an extension to the Hash class called recursive_merge!, which I found on an internet blog comment somewhere (so it's only fitting that I share back with this unitconverter)

Code for recursive_merge


class Hash
  def recursive_merge(hsh)
    self.merge(hsh) do |key, oldval, newval|
      oldval.is_a?(Hash) ? 
        oldval.recursive_merge(newval) :
        newval
    end
  end
  
  def recursive_merge!(hsh)
    self.merge!(hsh) do |key, oldval, newval|
      oldval.is_a?(Hash) ? 
        oldval.recursive_merge!(newval) :
        newval
    end
  end
end

Code for returning

class Object
  def returning( x )
    yield x
    x
  end
end

Monday, July 14, 2008

HaveBetterXpath

I'm rspeccing some REST controllers which return XML, and wanting to use XPath to validate the responses.

I came across this

http://blog.wolfman.com/articles/2008/01/02/xpath-matchers-for-rspec

Thanks to him. It worked nicely (couldn't be bothered messing about with hpricot to get that to go), but I didn't like the API as much as I could have.

Example of that API:

response.body.should have_xpath('/root/node1')
response.body.should match_xpath('/root/node1', "expected_value" )
response.body.should have_nodes('/root/node1/child', 3 )

I didn't like the fact that there were 3 distinct matchers, and that match_xpath didn't work with regexes. I re-worked it, so the API is now

response.body.should have_xpath('/root/node1')
response.body.should have_xpath('/root/node1').with("expected_value") # can also pass a regex
response.body.should have(3).elements('/root/node1/child') # Note actually extends string class and uses normal rspec have matcher

Extending the String class to support elements(xpath) is a win also because it lets you do things like


response.body.elements('/child').each { |e| more complex assert for e here }

Without further ado, new code here:


# Code borrowed from
# http://blog.wolfman.com/articles/2008/01/02/xpath-matchers-for-rspec
# Modified to use one matcher and tweak syntax

require 'rexml/document'
require 'rexml/element'

module Spec
  module Matchers

    # check if the xpath exists one or more times
    class HaveXpath
      def initialize(xpath)
        @xpath = xpath
      end

      def matches?(response)
        @response = response
        doc = response.is_a?(REXML::Document) ? response : REXML::Document.new(@response)
        
        if @expected_value.nil?
          not REXML::XPath.match(doc, @xpath).empty?
        else # check each possible match for the right value
          REXML::XPath.each(doc, @xpath) do |e|
            @actual_value = e.is_a?(REXML::Element) ? 
              e.text : 
              e.to_s # handle REXML::Attribute and anything else
  
            if @expected_value.kind_of?(Regexp) && @actual_value =~ @expected_value
              return true
            elsif @actual_value == @expected_value.to_s
              return true
            end
          end
          
          false # our loop didn't hit anything, mustn't be there
        end
      end
      
      def with_value( val )
        @expected_value = val
        self
      end
      alias :with :with_value

      def failure_message
        if @expected_value.nil?
          "Did not find expected xpath #{@xpath}"
        else
          "The xpath #{@xpath} did not have the value '#{@expected_value}'\nIt was '#{@actual_value}'"
        end
      end

      def negative_failure_message
        if @expected_value.nil?
          "Found unexpected xpath #{@xpath}"
        else
          "Found unexpected xpath #{@xpath} matching value #{@expected_value}"
        end
      end

      def description
        "match the xpath expression #{@xpath}, optionally matching it's value"
      end
    end

    def have_xpath(xpath)
      HaveXpath.new(xpath)
    end
    
    # Utility function, so we can do this: 
    # response.body.should have(3).elements('/images/')
    class ::String
      def elements(xpath)
        REXML::XPath.match( REXML::Document.new(self), xpath)
      end
      alias :element :elements
    end

  end
end

Monday, July 07, 2008

How to: load the session from a query string instead of a cookie

We use SWFUpload to upload some images in a login-restricted part of the site.

There is a problem however, in that we weren't able to get SWFUpload to send the normal browser cookie along with it's HTTP file uploads, so the server couldn't tell which user was logged in.

The 'normal' solution to this is to add the session key to the query string, and have the server load the session from the query string if the cookie isn't present, only ruby/rails doesn't support doing that.

a nice guy with the handle 'mcr' in #rubyonrails on irc.freenode.org worked out how to make this work, by patching ruby's cgi/session.rb

Instructions

  1. Copy cgi/session.rb out of your ruby standard library into your rails app's lib folder
  2. explicitly load the file out of lib, which will then overwrite the built in code

Needless to say this will stop working if the ruby standard library version of cgi/session changes, but I don't see that as being very likely

Patch in unified diff format:


--- /usr/lib/ruby/1.8/cgi/session.rb 2006-07-30 10:06:50.000000000 -0400
+++ lib/cgi/session.rb 2008-07-07 21:07:12.000000000 -0400
@@ -25,6 +25,9 @@
 
 require 'cgi'
 require 'tmpdir'
+require 'tempfile'
+require 'stringio'
+require 'strscan'
 
 class CGI
 
@@ -243,6 +246,20 @@
     #       undef_method :fieldset
     #   end
     #
+    def query_string_as_params(query_string)
+      return {} if query_string.blank?
+      
+      pairs = query_string.split('&').collect do |chunk|
+ next if chunk.empty?
+ key, value = chunk.split('=', 2)
+ next if key.empty?
+ value = value.nil? ? nil : CGI.unescape(value)
+ [ CGI.unescape(key), value ]
+      end.compact
+
+      ActionController::UrlEncodedPairParser.new(pairs).result
+    end
+
     def initialize(request, option={})
       @new_session = false
       session_key = option['session_key'] || '_session_id'
@@ -253,6 +270,7 @@
  end
       end
       unless session_id
+ #debugger XXX
  if request.key?(session_key)
    session_id = request[session_key]
    session_id = session_id.read if session_id.respond_to?(:read)
@@ -260,6 +278,12 @@
  unless session_id
    session_id, = request.cookies[session_key]
  end
+
+ unless session_id
+   params = query_string_as_params(request.query_string)
+   session_id = params[session_key]
+ end
+
  unless session_id
    unless option.fetch('new_session', true)
      raise ArgumentError, "session_key `%s' should be supplied"%session_key


Sunday, July 06, 2008

How to: Avoid getting your database wiped when migrating to rails 2.1

We recently migrated some projects from rails 1.2 to 2.1.

In doing this, we encountered a bug where sometimes (in production only) running rake db:migrate goes wrong, and re-runs all your migrations

The unhappy side effect of it re-running ALL the migrations, is that it effectively re-creates your entire database, and you lose all your data. USEFUL

I didn't have the time or the luxury to figure out quite why this was happening, if anyone does, please comment and let me know what it was. Apparently there's been a few other blogs mentioning it, but I don't have any of them at hand.

The workaround is to manually create the schema_migrations table before you run rake db:migrate in rails 2.1.

If you put the following script in your RAILS_ROOT/db directory, and run it, it will do that.

Enjoy. (Disclaimer: if there's a bug in the script, and it does anything awful, it's not my fault! You have been warned!)

require File.dirname(__FILE__) + '/../config/environment'

# Define some models
class SchemaInfo < ActiveRecord::Base
  set_table_name 'schema_info'
end
class SchemaMigration < ActiveRecord::Base; end

# Create the schema_migrations table
ActiveRecord::Migration.class_eval do
  create_table 'schema_migrations', :id => false do |t|
    t.column :version, :string, :null => false
  end
end

# Work out the migrated version and populate the migrations table

v = SchemaInfo.find(:first).version.to_i
puts "Current schema version is #{v}"
raise "Version number doesn't seem right!" if v == 0

1.upto(v) do |i|
 SchemaMigration.create!( :version => i )
 puts "Added entry for migration #{i}"
end

# Drop the schema info table, as rails-2.1 won't automatically do it thanks to our hacking
ActiveRecord::Migration.class_eval do
  drop_table 'schema_info'
end

How To: Create old rails apps when you have newer gems installed

My dev server has the gems for rails 1.2.6, 2.0.2 and 2.1.0 all installed.

You can see which ones you have by running

gem list --local | grep rails

The problem is, when I create new rails apps, it always uses the latest version. If I explicitly want to create a 1.2.6 or 2.0.2 app, then I can do it like this

rails _1.2.6_ some_old_app

Useful.

For the technically nosey, we can see how this works by reading the source of /usr/bin/rails, which is here

require 'rubygems'
version = "> 0"
if ARGV.first =~ /^_(.*)_$/ and Gem::Version.correct? $1 then
  version = $1
  ARGV.shift
end
gem 'rails', version
load 'rails'

How to: Rails 2.0 and 2.1 resources with semicolons

Rails 1.X used semicolons as method seperators for resources, so you'd get

http://somesite/things/1;edit

Rails 2.X switches this to

http://somesite/things/1/edit

This is nice and all, but some of us have actual client applications which we can't all just upgrade instantly

To make the semicolon-routes still work in rails 2.X, so you don't break all your clients, do this

At the TOP of routes.rb, before the ActionController::Routing::Routes.draw block

# Backwards compatibility with old ; delimited routes
ActionController::Routing::SEPARATORS.concat %w( ; , )

and, at the BOTTOM of routes.rb BEFORE the end

# Backwards compatibility with old ; delimited routes
map.connect ":controller;:action"
map.connect ":controller/:id;:action"

Profit!