Consume non rails-style REST API's  53


ActiveResource is a great concept which consumes rails-style REST API but unfortunately most of the REST API's are not rails-style. This means that very frequently you will end up modifying ActiveResource to consume non rails-style REST API's. This article is about understanding ActiveResource and how to tweak/extend it to consume non rails-style REST API's. We will mainly concentrate on reading data i.e. the GET method.

Table of Contents

  1. Introduction
  2. Consume non rails-style REST API
    1. Create URL for remote resources
    2. Make a GET request
    3. Handling (Custom) Response
    4. Parse Response
    5. Create ActiveResource object from parsed response
    6. Other things to keep in mind
  3. Custom HTTP GET method tweaks
  4. Data Format

Introduction

Let me recall the purpose of ActiveResource as stated in ActiveResource README :

Active Resource attempts to provide a coherent wrapper object-relational mapping for REST web services. It follows the same philosophy as Active Record, in that one of its prime aims is to reduce the amount of code needed to map to these resources. This is made possible by relying on a number of code- and protocol-based conventions that make it easy for Active Resource to infer complex relations and structures.
Or, Model classes are mapped to remote REST resources by Active Resource much the same way Active Record maps model classes to database tables
The CRUD Mapping to REST (or ActiveRecord Mapping to ActiveResource) :
Create POST
Read GET -- our target
Update PUT
Delete DELETE

A minimalistic ActiveResource Model class looks as follows:
1
2
3
4
5
6
7

class Product < ActiveResource::Base
  self.site = "http://www.quarkrank.com/"
end
# Now, one can query quarkrank.com's api to get all products, or complete details for a particular product by simply doing a find:
# Product.find(:all) => http://www.quarkrank.com/products.xml
# Product.find("canon-powershot-sd1000") => http://www.quarkrank.com/products/canon-powershot-sd1000.xml 

Nested Resources: Some resources depend on other resources for e.g. comments on a blog would always depend on of the blog post. The comments can't exist independently. So, url for comments would be something like: www.myblog.com/posts/a_post/comments. Which means that url for finding comments would require an blog_post id.

And if one is accessing nested resources, model class would look like:
1
2
3
4
5
6
7

class Review < ActiveResource::Base
  # here reviews exist for a given product only.
  self.site = "http://www.quarkrank.com/products/:product_id/"
end
# Now, you can ask for reviews on canon-powershot-sd1000 by doing find:
# Review.find(:all, :params=>{:product_id=>'canon-powershot-sd1000'})
For better understanding further, I would really appreciate if you could go through Ryan's presentation on ActiveResource and Railscasts ActiveResource episodes for better understanding of ActiveResource basics.


Method Call Flow in a Get Request
Lets say, we are doing a find query on some ActiveResource Model.
Note: (phrases in braces denote the actual method calls being made)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

 find
  |  
  |-- find single or all items(find_single/find_every)
        |
        |-- create_url (element_path/collection_path)
        |
        |-- get_response_from_url (connection.get)
               |
               |-- make_http_request_to_url (http.get)
               |
               |-- handle_response(response) -> exceptions are raised here if we get 4xx/5xx response codes.
               |
               |-- get_body(response.body)
               |
               |-- decode_output, from xml/json to ruby hash (format.decode)
        |
        |-- convert_hash_to_active_resource_object (instantiate_record/instantiate_collection)

Consume non rails-style REST API

As we plan to talk about the GET operation, lets get deeper into the following steps :

Creating URL for remote resources

Sometimes, we might need to change the REST style url generation. At time of writing this article, most of the popular API's do not follow the rails restful url generation. So, the first step is to create the URL before a third party resource call is made. The URL is constructed using element_path or collection_path methods, depending on whether the response has 1 element or a set of elements respectively.
So, here is the code and little explanation of element_path method.
1
2
3
4
5
6
7

# code of element_path function
def element_path(id, prefix_options = {}, query_options = nil)
  prefix_options, query_options = split_options(prefix_options) if query_options.nil?
  # path to the resource, which we want to access is evaluated in this statement: 
  "#{prefix(prefix_options)}#{collection_name}/#{id}.#{format.extension}#{query_string(query_options)}"
end
Explanation: Lets look at each of the variable/method used in last statement above.
1
2
3
4
5
6
7
8
9
10

prefix(prefix_options) depends on self.site variable and value(s) of nested resource variable.
   =>Evaluates the "fixed" prefix path to the resource (if any, mentioned in self.site variable) and/or
      in case you are using nested style queries, replaces variables with their values
== Examples:
1. self.site = "http://www.quarkrank.com/folders/api"
prefix(prefix_options) => "/folders/api/"
2. self.site = "http://www.quarkrank.com/folders/:folder_id"
find(1,:folder_id=>5)
prefix(prefix_options) => "/folders/5/"
1
2
3
4
5

collection_name => evaluates to pluralize form of classname
== Examples:
1. class Comment < ActiveResource::Base;end
collection_name => "comments"
1
2
3

id => id of the item we are querying, usually mentioned in find (example: User.find(5))
format_extension => what format request you are making request for (default is xml).
1
2
3
4
5
6
7
8
9
10
11

query_string(query_options) => generates get styled query string from remaining params.
== Examples:
1. self.site = "http://www.quarkrank.com/folders/api"
    find(1)
    query_string(query_options) => ""
2. self.site = "http://www.quarkrank.com/folders/:folder_id"
    find(1,:folder_id=>5, :filename=>"nakul")
    query_string(query_options) => "?filename=nakul"

## collection_path method is quite similar
So, in case you want to modify the element_path, just redefine the method in your model class with custom definition. Please look into ActiveYoutube class code as an example.

Make a GET request

After creating url, request is send using Net::HTTP (ssl requests are supported).
Note that, private method "request" is called for making a Net::HTTP request, which logs the request being made and response from api server. This logging might lead to exception, because of the following line in the code.
1
2
3

# in case, result.message contains some characters like "%A", this leads to exception
logger.info "--> #{result.code} #{result.message} (#{result.body ? result.body : 0}b %.2fs)" % time if logger
So, in case you are getting exception try to switch off the logs.

Handling (Custom) Response

ActiveResource relies on response code to figure out errors/success/redirection but this might not always be true. Most of the API's do not respect this. Its quite common to see possible errors like unauthorized access, forbidden access, server error etc in success response.
For example, on unauthorized access, Flickr's API returns 200 OK response code with xml response describing the failure. (Facebook API also belongs to this league)

How to handle these errors? : ActiveResource currently doesn't supports callback hooks like after_find etc. So, one cannot hook the custom handlers for response handling. While, work is in progress for having callbacks support in ActiveResource, but till then we need to handle them on our own. So, for Flickr API, one solution will look like:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

## define a ActiveResource::Flickr  class, which raises exception if response is not OK.
  class ActiveFlickr<ActiveResource::Base
    class << self
      alias :old_find :find
      def find(*arguments)
        output = old_find(*arguments)
        if output.respond_to? :err
          case output.err.code.to_i
            when 100
              raise(ActiveResource::UnauthorizedAccess.new(output.err, output.err.msg))
            when 112
              raise(ActiveResource::MethodNotAllowed.new(output.err, output.err.msg))
            else
              raise(ConnectionError.new(output.err, "Unknown response code: #{outout.err.code}"))
          end
        end
      end
   end

# now other ActiveResource models would inherit from ActiveFlickr, rather ActiveResource::Base.
1
2
3
4
5
6
7
8
9
10
11
12

## also ActiveResource Exceptions, currently doesn't logs/prints the "message", which is passed as second argument. 
# You might want to modify the behavior to print error message also.
module ActiveResource
  class ConnectionError
    def to_s
      str = "Failed with #{response.code} #{response.message if response.respond_to?(:message)}\n"
      str += @message unless @message.nil?
      str
    end
  end
end

Parse Response

Next step is to decode the XML/JSON response into ruby object. Decoding is done in get/post method call in connection.rb. XML to hash conversion is done using XmlSimple with some modifications.
There is not much documentation on this conversion but more enthusiastic people can look at from_xml method in: vendor/rails/activesupport/lib/active_support/core_ext/hash/conversions.rb

Create ActiveResource object from parsed response

Convert appropriate elements into ActiveResource objects. Its done using 'load' method in ActiveResource::Base, which takes ruby object as input and maps it into ActiveResource object.

Other things to keep in mind

In last step of find(:all) method call, i.e. conversion of ruby object to ActiveResource Object, instantiate_collection method is called on ruby object. Here, ActiveResource expects an array. This may not be true for many of API's like Amazon, Youtube. So, you might need to rewrite this function:
1
2
3
4
5
6
7
8
9

class ActiveResource::Base
  def self.instantiate_collection(collection, prefix_options = {})
    unless collection.kind_of? Array      [instantiate_record(collection, prefix_options)]
    else
      collection.collect! { |record| instantiate_record(record, prefix_options) }
    end
  end
end

Custom HTTP GET method tweaks

Since simple CRUD/lifecycle methods cannot accomplish every task, ActiveResource supports defining your own custom REST methods. Sometimes we will be using CustomHTTP requests for executing a custom action for a particular resource.

Example: Getting comments for a particular blog post. A sample request could be: www.myblog.com/post/active_resouce/coments.xml. Here, we find a particular blog post and then ask for comments on it. So, ActiveResouce code would be:
1
2
3
4
5
6

# Type1: 
BlogPost.find('active_resouce').get(:comments)
# More examples:
Person.find(1).put(:promote, :position => 'Manager') # PUT /people/1/promote.xml
Person.find(1).delete(:deactivate) # DELETE /people/1/deactivate.xml
Or, sometimes, we might just want the list of active users on website right now.
1
2
3

#Type2
Person.get(:active)  # GET /people/active.xml

find method makes a call to get "id"

The "Type1" custom REST requests actually makes 2 remote requests:
  • Find the resource for which we want to make a custom request: This is used to find the id of the resource to be used in next step
  • The actual custom rest request
Here, sometimes we might want to skip step1 if we already know the "id" of the resource. So, one can define different find method which just sets the id param to be used to custom rest request and call get method:
1
2
3
4
5
6
7
8

  def find_custom(arg)
    object = self.new
    object.id = arg
    object
  end
# Example: For youtube videos, if we want comments for a particular video, we would do 
# Video.find_custom("ZTUVgYoeN_o").get(:comments)

"get" method doesn't converts hash into objects.

ActiveResource::CustomMethods get request sometimes does not converts the decoded ruby object (from xml) to activeresource objects. You can modify the behavior to get activeresource object
1
2
3
4
5
6
7
8
9
10
11
12
13

  def get(method_name, options = {})
    self.class.new.load(connection.get(custom_method_element_url(method_name, options), self.class.headers))
  end

  def self.get(method_name, options = {})
    object_array = connection.get(custom_method_collection_url(method_name, options), headers)
    if object_array.class.to_s=="Array"
      object_array.collect! {|record| self.class.new.load(record)}
    else
      self.class.new.load(object_array)
    end
  end

Data Format

Currently 2 formats are supported by ActiveResource: JSON and XML Do you want to use another format? Pretty easy, you need to define 4 methods: extension, mime_type, encode and decode. Encoding is for converting hash into required format and Decoding is for decoding the response in this format into a hash object. Example of JSON format
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

module ActiveResource
  module Formats
    module JsonFormat
      extend self
      
      def extension
        "json"
      end
      
      def mime_type
        "application/json"
      end
      
      def encode(hash)
        hash.to_json
      end
      
      def decode(json)
        ActiveSupport::JSON.decode(json)
      end
    end
  end
end

Thanks to Brian Nochugi for his frequent discussion/doubts on ActiveResource. It helped us to properly formulate the article.
Filed in ruby tutorials
Tagged as   
Posted on 11 March
53 comment Bookmark   AddThis Social Bookmark Button Updated on 23 September
Comments

Leave a response

  1. Funny stuffOctober 23, 2009 @ 07:11 PM

    Thanks for the tutorial. You provide high quality stuff.

  2. Multivariate TestingNovember 17, 2009 @ 09:42 AM

    i am a student of computer science and i am presently doing a course on it.I have been trying to figure out how the date format works but after seeing this i got an idea i thnk now i can go ahead with my project

  3. Content SolutionsNovember 17, 2009 @ 09:46 AM

    Nice tutorial.I learned a lot after reading it but i have to make one practical program to test out these.kepp the blog updated so that we can learn by reading the posts done by you

  4. Inpatient Heroin Detox ProgramsNovember 19, 2009 @ 07:36 AM

    Excellent tutorial mate! I just saved $30 as I was supposed to hire someone to fix this on my site. Thanks.

  5. mazda 3November 19, 2009 @ 08:24 PM

    I also think that ActiveResource is a great way to access remote data. In fact I am using it to store and access data in Amazon’s SimpleDB webservice.

  6. volinmazDecember 10, 2009 @ 08:40 AM

    That is why i love QuarkRuby, we can get all information here.

  7. SlidesDecember 14, 2009 @ 12:02 PM

    Thanks. There is need those. Because my web sites everydays errors. Very thanks.

  8. VM WareDecember 16, 2009 @ 04:33 AM

    Quarkruby is easy to learn and easy to use but there are some hiccups as well hopefully in coming days these issuees will be resolved.Slowly qurakruby will gain ground and will a highly used language

  9. california traffic school February 09, 2010 @ 09:50 AM

    Can we define different find method which just sets the id param to be used to custom rest request and call get method in this?

  10. bodybuilding supplementsFebruary 24, 2010 @ 08:43 PM

    Very nice tutorial thank you

  11. Seo ArticlesFebruary 27, 2010 @ 07:50 AM

    Wow, stumbled upon this stuff through Google. Had been looking for it since quite a while. Thanks a ton for the share :)

  12. handbagsMarch 01, 2010 @ 03:43 AM

    I also think that ActiveResource is a great way to access remote data.

  13. WatchmenMarch 02, 2010 @ 02:13 AM

    Totally agree VM Ware, QuarkRuby is so easy to learn.

  14. Obama college grantsMarch 03, 2010 @ 06:27 PM

    Programming is so difficult. I’m so glad you guys are helping me out with these tutorials. They really help a lot.

  15. DaleMarch 03, 2010 @ 07:16 PM

    Thanks for the tutorial! its really detailed & easy to follow!

  16. uninstall programsMarch 03, 2010 @ 10:27 PM

    this helped out I was having an issue with ActiveResource and found this – much thanks.

  17. kids backpacksMarch 07, 2010 @ 11:55 PM

    Nice code, thanks for the post.

    -brad

  18. john ryanMarch 08, 2010 @ 02:47 AM

    very useful code! thanks for sharing it to us!

  19. shakir anjumMarch 08, 2010 @ 09:26 AM

    It follows the same philosophy as Active Record, in that one of its prime aims is to reduce the amount of code needed to map to these resources. This is made possible by relying on a number of code- and protocol-based conventions that make it easy for Active Resource to infer complex relations and structures.

    granite countertops nj

  20. New Business GrantsMarch 09, 2010 @ 10:06 PM

    This is really helpful! I’ve been wondering how to do this! Thanks!

  21. BaccaratMarch 11, 2010 @ 04:38 AM

    Hello, I am also a computer science student doing a course on ActiveResource. I have been trying to figure out how the date format is set, but after reading this post I got it finally. Thanks, I can go ahead with my homework now.

  22. BestBingoSpotsMarch 13, 2010 @ 12:06 AM

    Thanks for the info. This really helped me out.

    Sky Bingo

  23. Trueprotein Discount codeMarch 15, 2010 @ 10:54 PM

    thanks for the guide, i know it will come in handy.

  24. edinburgh dentistMarch 17, 2010 @ 08:48 AM

    How can I make acts_as_solr able to search for different languages(ex: Arabic)?

  25. ent problemsMarch 21, 2010 @ 04:43 AM

    very nice tutorial , nice,

  26. Custom DressesMarch 23, 2010 @ 05:59 AM

    very nice tutorial

  27. dvd to itouchMarch 24, 2010 @ 07:08 PM

    The tutorials on ActiveResource are simply amazing. Very easy to use and easy to understand. Thanks for sharing, it was very helpful to me. Keep up the good work…

  28. lice treatmentMarch 31, 2010 @ 05:50 PM

    its very informative post, it educated individuals. they are easy to handle, easy to understand and easy to use.

    lice treatment

  29. CaviarMarch 31, 2010 @ 11:52 PM

    I am always searching for quality content and that’s really helpful for me. Thanks a lot.

  30. annunityApril 07, 2010 @ 06:36 AM

    thanks for the post, Very useful and informative.The tutorial is quite amazing and very resourceful. thanks for sharing it.

  31. get him back forever reviewApril 13, 2010 @ 06:07 AM

    This is a fantastic article. Thanks for putting it together.

  32. Dental jobsApril 14, 2010 @ 08:18 AM

    Currently 2 formats are supported by ActiveResource: JSON and XML Do you want to use another format? Pretty easy, you need to define 4 methods: extension, mime_type, encode and decode. Encoding is for converting hash into required format and Decoding is for decoding the response in this format into a hash object. Example of JSON format

  33. ray ban wayfarerApril 17, 2010 @ 07:51 AM

    This is a fantastic article. Thanks for putting it together

  34. website tips and tricksApril 19, 2010 @ 05:50 AM

    Excellent tutorial mate! I just saved $30 as I was supposed to hire someone to fix this on my site. Thanks.

  35. pdf search engineApril 19, 2010 @ 06:03 AM

    Im impressed, you know what youre talking about. I like the concept very much.

  36. Holiday Villa JaveaApril 20, 2010 @ 02:00 PM

    Great, just like website tips, I was struggling to fix this on my site, so thanks.

  37. Internet hostingApril 27, 2010 @ 07:52 PM

    $30 as I was supposed to hire someone to fix this on my site. Thanks.

  38. Driver Not FoundApril 28, 2010 @ 02:11 PM

    It’s good to see there are people who will take the time to cook up a real and decent tutorial, thanks for sharing this.

    Regards

  39. Javea Property For SaleApril 29, 2010 @ 08:47 AM

    Thanks for sharing this, it’s a nice tutorial

    Javea Property For Sale
    
  40. keychainsApril 29, 2010 @ 12:26 PM

    thanks for your post, good article.

    Keychains

    Best keychains for you, thanks.

  41. hypnosis reviewsApril 29, 2010 @ 01:42 PM

    Thanks for sharing this information. I found it very informative as I have been researching a lot lately on practical matters such as you talk about.

  42. CamApril 30, 2010 @ 09:44 AM

    More and more all this stuff starts coming together and being a great help. Safety reasons suggest using non skid tape for its anti-slip properties.

  43. Kerala TourismApril 30, 2010 @ 08:56 PM

    This is a nice post and thank you very much for sharing this with us.

  44. Discount Evening DressesMay 01, 2010 @ 03:33 PM

    u have done well. thanks for sharing.

  45. Shoebuy CouponMay 06, 2010 @ 06:03 AM

    Thanks for sharing this, it’s a nice tutorial

  46. how to solve sudokuMay 06, 2010 @ 06:58 AM

    ye sthank you very much for the great post

  47. selçukMay 08, 2010 @ 05:52 PM

    thank you very much man

  48. Fat Burning Furnace Scam ReviewMay 09, 2010 @ 12:03 AM

    Can i use APIs only for the facebook ?

  49. kerala used carMay 13, 2010 @ 09:02 AM

    i was about to give 20 $ to a guy from digital point. U saved me. thanks

  50. Cow PrintMay 14, 2010 @ 01:44 AM

    Very great article, really helpful – thanks!

  51. Get him back forever reviewMay 22, 2010 @ 05:17 PM

    Great article! Tanks :)

  52. hid lightsMay 23, 2010 @ 01:59 PM

    Yes no doubt real and perfect article.

  53. logo designsMay 24, 2010 @ 07:30 AM

    [...] Excellent second tutorial! You have been bookmarked on digg and delicious. Keep up the good work. :) [...]

Comment