Consume non rails-style REST API's  4


ActiveResource is a great concept which consumes rails-style REST API but unfortunately most of the REST API's are not rails-style. This means that very frequently you will end up modifying ActiveResource to consume non rails-style REST API's. This article is about understanding ActiveResource and how to tweak/extend it to consume non rails-style REST API's. We will mainly concentrate on reading data i.e. the GET method.

Table of Contents

  1. Introduction
  2. Consume non rails-style REST API
    1. Create URL for remote resources
    2. Make a GET request
    3. Handling (Custom) Response
    4. Parse Response
    5. Create ActiveResource object from parsed response
    6. Other things to keep in mind
  3. Custom HTTP GET method tweaks
  4. Data Format
Filed in ruby tutorials
Tagged as activeresource api 
Posted on 11 March
4 comment Bookmark   AddThis Social Bookmark Button

archive

Consume non rails-style REST API's

ActiveResource is a great concept which consumes rails-style REST API but unfortunately most of the REST API's are not rails-style. This means that very frequently you will end up modifying ActiveResource to consume non rails-style REST API's. This article is about understanding ActiveResource and how to tweak/extend it to consume non rails-style REST API's. We will mainly concentrate on reading data i.e. the GET method.

Table of Contents

  1. Introduction
  2. Consume non rails-style REST API
    1. Create URL for remote resources
    2. Make a GET request
    3. Handling (Custom) Response
    4. Parse Response
    5. Create ActiveResource object from parsed response
    6. Other things to keep in mind
  3. Custom HTTP GET method tweaks
  4. Data Format

Introduction

Let me recall the purpose of ActiveResource as stated in ActiveResource README :

Active Resource attempts to provide a coherent wrapper object-relational mapping for REST web services. It follows the same philosophy as Active Record, in that one of its prime aims is to reduce the amount of code needed to map to these resources. This is made possible by relying on a number of code- and protocol-based conventions that make it easy for Active Resource to infer complex relations and structures.
Or, Model classes are mapped to remote REST resources by Active Resource much the same way Active Record maps model classes to database tables
The CRUD Mapping to REST (or ActiveRecord Mapping to ActiveResource) :
Create POST
Read GET -- our target
Update PUT
Delete DELETE

A minimalistic ActiveResource Model class looks as follows:
1
2
3
4
5
6
class Product < ActiveResource::Base
  self.site = "http://www.quarkrank.com/"
end
# Now, one can query quarkrank.com's api to get all products, or complete details for a particular product by simply doing a find:
# Product.find(:all) => http://www.quarkrank.com/products.xml
# Product.find("canon-powershot-sd1000") => http://www.quarkrank.com/products/canon-powershot-sd1000.xml 

Nested Resources: Some resources depend on other resources for e.g. comments on a blog would always depend on of the blog post. The comments can't exist independently. So, url for comments would be something like: www.myblog.com/posts/a_post/comments. Which means that url for finding comments would require an blog_post id.

And if one is accessing nested resources, model class would look like:
1
2
3
4
5
6
class Review < ActiveResource::Base
  # here reviews exist for a given product only.
  self.site = "http://www.quarkrank.com/products/:product_id/"
end
# Now, you can ask for reviews on canon-powershot-sd1000 by doing find:
# Review.find(:all, :params=>{:product_id=>'canon-powershot-sd1000'})
For better understanding further, I would really appreciate if you could go through Ryan's presentation on ActiveResource and Railscasts ActiveResource episodes for better understanding of ActiveResource basics.


Method Call Flow in a Get Request
Lets say, we are doing a find query on some ActiveResource Model.
Note: (phrases in braces denote the actual method calls being made)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
 find
  |  
  |-- find single or all items(find_single/find_every)
        |
        |-- create_url (element_path/collection_path)
        |
        |-- get_response_from_url (connection.get)
               |
               |-- make_http_request_to_url (http.get)
               |
               |-- handle_response(response) -> exceptions are raised here if we get 4xx/5xx response codes.
               |
               |-- get_body(response.body)
               |
               |-- decode_output, from xml/json to ruby hash (format.decode)
        |
        |-- convert_hash_to_active_resource_object (instantiate_record/instantiate_collection)

Consume non rails-style REST API

As we plan to talk about the GET operation, lets get deeper into the following steps :

Creating URL for remote resources

Sometimes, we might need to change the REST style url generation. At time of writing this article, most of the popular API's do not follow the rails restful url generation. So, the first step is to create the URL before a third party resource call is made. The URL is constructed using element_path or collection_path methods, depending on whether the response has 1 element or a set of elements respectively.
So, here is the code and little explanation of element_path method.
1
2
3
4
5
6
# code of element_path function
def element_path(id, prefix_options = {}, query_options = nil)
  prefix_options, query_options = split_options(prefix_options) if query_options.nil?
  # path to the resource, which we want to access is evaluated in this statement: 
  "#{prefix(prefix_options)}#{collection_name}/#{id}.#{format.extension}#{query_string(query_options)}"
end
Explanation: Lets look at each of the variable/method used in last statement above.
1
2
3
4
5
6
7
8
9
prefix(prefix_options) depends on self.site variable and value(s) of nested resource variable.
   =>Evaluates the "fixed" prefix path to the resource (if any, mentioned in self.site variable) and/or
      in case you are using nested style queries, replaces variables with their values
== Examples:
1. self.site = "http://www.quarkrank.com/folders/api"
prefix(prefix_options) => "/folders/api/"
2. self.site = "http://www.quarkrank.com/folders/:folder_id"
find(1,:folder_id=>5)
prefix(prefix_options) => "/folders/5/"
1
2
3
4
collection_name => evaluates to pluralize form of classname
== Examples:
1. class Comment < ActiveResource::Base;end
collection_name => "comments"
1
2
id => id of the item we are querying, usually mentioned in find (example: User.find(5))
format_extension => what format request you are making request for (default is xml).
1
2
3
4
5
6
7
8
9
10
query_string(query_options) => generates get styled query string from remaining params.
== Examples:
1. self.site = "http://www.quarkrank.com/folders/api"
    find(1)
    query_string(query_options) => ""
2. self.site = "http://www.quarkrank.com/folders/:folder_id"
    find(1,:folder_id=>5, :filename=>"nakul")
    query_string(query_options) => "?filename=nakul"

## collection_path method is quite similar
So, in case you want to modify the element_path, just redefine the method in your model class with custom definition. Please look into ActiveYoutube class code as an example.

Make a GET request

After creating url, request is send using Net::HTTP (ssl requests are supported).
Note that, private method "request" is called for making a Net::HTTP request, which logs the request being made and response from api server. This logging might lead to exception, because of the following line in the code.
1
2
# in case, result.message contains some characters like "%A", this leads to exception
logger.info "--> #{result.code} #{result.message} (#{result.body ? result.body : 0}b %.2fs)" % time if logger
So, in case you are getting exception try to switch off the logs.

Handling (Custom) Response

ActiveResource relies on response code to figure out errors/success/redirection but this might not always be true. Most of the API's do not respect this. Its quite common to see possible errors like unauthorized access, forbidden access, server error etc in success response.
For example, on unauthorized access, Flickr's API returns 200 OK response code with xml response describing the failure. (Facebook API also belongs to this league)

How to handle these errors? : ActiveResource currently doesn't supports callback hooks like after_find etc. So, one cannot hook the custom handlers for response handling. While, work is in progress for having callbacks support in ActiveResource, but till then we need to handle them on our own. So, for Flickr API, one solution will look like:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
## define a ActiveResource::Flickr  class, which raises exception if response is not OK.
  class ActiveFlickr<ActiveResource::Base
    class << self
      alias :old_find :find
      def find(*arguments)
        output = old_find(*arguments)
        if output.respond_to? :err
          case output.err.code.to_i
            when 100
              raise(ActiveResource::UnauthorizedAccess.new(output.err, output.err.msg))
            when 112
              raise(ActiveResource::MethodNotAllowed.new(output.err, output.err.msg))
            else
              raise(ConnectionError.new(output.err, "Unknown response code: #{outout.err.code}"))
          end
        end
      end
   end

# now other ActiveResource models would inherit from ActiveFlickr, rather ActiveResource::Base.
1
2
3
4
5
6
7
8
9
10
11
## also ActiveResource Exceptions, currently doesn't logs/prints the "message", which is passed as second argument. 
# You might want to modify the behavior to print error message also.
module ActiveResource
  class ConnectionError
    def to_s
      str = "Failed with #{response.code} #{response.message if response.respond_to?(:message)}\n"
      str += @message unless @message.nil?
      str
    end
  end
end

Parse Response

Next step is to decode the XML/JSON response into ruby object. Decoding is done in get/post method call in connection.rb. XML to hash conversion is done using XmlSimple with some modifications.
There is not much documentation on this conversion but more enthusiastic people can look at from_xml method in: vendor/rails/activesupport/lib/active_support/core_ext/hash/conversions.rb

Create ActiveResource object from parsed response

Convert appropriate elements into ActiveResource objects. Its done using 'load' method in ActiveResource::Base, which takes ruby object as input and maps it into ActiveResource object.

Other things to keep in mind

In last step of find(:all) method call, i.e. conversion of ruby object to ActiveResource Object, instantiate_collection method is called on ruby object. Here, ActiveResource expects an array. This may not be true for many of API's like Amazon, Youtube. So, you might need to rewrite this function:
1
2
3
4
5
6
7
8
class ActiveResource::Base
  def self.instantiate_collection(collection, prefix_options = {})
    unless collection.kind_of? Array      [instantiate_record(collection, prefix_options)]
    else
      collection.collect! { |record| instantiate_record(record, prefix_options) }
    end
  end
end

Custom HTTP GET method tweaks

Since simple CRUD/lifecycle methods cannot accomplish every task, ActiveResource supports defining your own custom REST methods. Sometimes we will be using CustomHTTP requests for executing a custom action for a particular resource.

Example: Getting comments for a particular blog post. A sample request could be: www.myblog.com/post/active_resouce/coments.xml. Here, we find a particular blog post and then ask for comments on it. So, ActiveResouce code would be:
1
2
3
4
5
# Type1: 
BlogPost.find('active_resouce').get(:comments)
# More examples:
Person.find(1).put(:promote, :position => 'Manager') # PUT /people/1/promote.xml
Person.find(1).delete(:deactivate) # DELETE /people/1/deactivate.xml
Or, sometimes, we might just want the list of active users on website right now.
1
2
#Type2
Person.get(:active)  # GET /people/active.xml

find method makes a call to get "id"

The "Type1" custom REST requests actually makes 2 remote requests:
  • Find the resource for which we want to make a custom request: This is used to find the id of the resource to be used in next step
  • The actual custom rest request
Here, sometimes we might want to skip step1 if we already know the "id" of the resource. So, one can define different find method which just sets the id param to be used to custom rest request and call get method:
1
2
3
4
5
6
7
  def find_custom(arg)
    object = self.new
    object.id = arg
    object
  end
# Example: For youtube videos, if we want comments for a particular video, we would do 
# Video.find_custom("ZTUVgYoeN_o").get(:comments)

"get" method doesn't converts hash into objects.

ActiveResource::CustomMethods get request sometimes does not converts the decoded ruby object (from xml) to activeresource objects. You can modify the behavior to get activeresource object
1
2
3
4
5
6
7
8
9
10
11
12
  def get(method_name, options = {})
    self.class.new.load(connection.get(custom_method_element_url(method_name, options), self.class.headers))
  end

  def self.get(method_name, options = {})
    object_array = connection.get(custom_method_collection_url(method_name, options), headers)
    if object_array.class.to_s=="Array"
      object_array.collect! {|record| self.class.new.load(record)}
    else
      self.class.new.load(object_array)
    end
  end

Data Format

Currently 2 formats are supported by ActiveResource: JSON and XML Do you want to use another format? Pretty easy, you need to define 4 methods: extension, mime_type, encode and decode. Encoding is for converting hash into required format and Decoding is for decoding the response in this format into a hash object. Example of JSON format
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
module ActiveResource
  module Formats
    module JsonFormat
      extend self
      
      def extension
        "json"
      end
      
      def mime_type
        "application/json"
      end
      
      def encode(hash)
        hash.to_json
      end
      
      def decode(json)
        ActiveSupport::JSON.decode(json)
      end
    end
  end
end

Thanks to Brian Nochugi for his frequent discussion/doubts on ActiveResource. It helped us to properly formulate the article.