Skip to content

Latest commit

 

History

66 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Micky

Micky makes simple HTTP requests (GET/HEAD), follows redirects, handles exceptions (invalid hosts/URIs, server errors, timeouts, redirect loops), automatically parses responses (JSON, etc.), is very lightweight, and has no dependency.

Micky is for those times you would have used Net::HTTP or OpenURI, but don’t want to bother handling all the sneaky things mentionned above, and don’t want to add heavy dependencies to your app.

Installation

Add this line to your application’s Gemfile:

gem 'micky'

And then execute:

$ bundle

Or install it yourself as:

$ gem install micky

Usage

Micky provides two methods: get and head.

On successful requests, it will return a subclass of Net::HTTPReponse. For any error it might encounter during the request (invalid hosts/URIs, server errors, timeouts, redirect loops), it will return nil.

response = Micky.get('http://google.com')
response.content_type # "text/html"
response.body         # "<!doctype html><html ..."

response = Micky.get('http://invalidhost.foo')
response # nil

Classic example

if Micky.head(params[:website_url])
  # User provided a valid URL
  url = URI(params[:website_url])
  url.path = '/favicon.ico'

  if favicon = Micky.get(url)
    # Do whatever with the raw `favicon.body`, for whatever reason
  else
    # This site has no favicon, or a broken one, too bad
  end
else
  # Some error happened, display error message to user
end

Headers and query strings

Request headers and query string params can be passed as :headers and :query.

Micky.get('http://drpm.me/unwz.jpg', headers: { 'Accept' => 'text/html' })
Micky.get('http://urls.api.twitter.com/1/urls/count.json', query: { url: 'dropmeme.com' })

Limiting what an untrusted server can cost you

By default Micky reads a response to its end, and :timeout bounds each socket operation rather than the call as a whole. Against a server you don’t control, neither is a limit: a body can be as large as the server cares to send, and a server that answers slowly but never stalls is never timed out at all.

:max_response_size is the number of body bytes Micky will read. Past that it stops reading and drops the connection, so an oversized response costs you the limit and not the body. The call returns nil, or raises Micky::TooLargeResponse when :raise_errors is set:

Micky.get(url, max_response_size: 512 * 1024) # nil if the body is larger

Pass :truncate to get the first :max_response_size bytes instead. A truncated response is useful for some formats and useless for others, so this is opt-in: fine for HTML, not for an image.

response = Micky.get(url, max_response_size: 512 * 1024, truncate: true)
response.body.bytesize # at most 524288
response.truncated?    # true if the server had more to send

:total_timeout is a wall clock for the whole call, redirects included. It is the only option that bounds a server which drips one byte at a time, or a long chain of individually-reasonable redirects:

Micky.get(url, timeout: 5, total_timeout: 10)

On expiry Micky returns nil, or raises Micky::TotalTimeout when :raise_errors is set. It defaults to 20 seconds; pass nil for no limit.

:max_response_size has no default, since downloading a large file can be the point. For most callers it is the only option worth setting explicitly.

OAuth Authorization header

Micky supports creating a OAuth Authorization header with the help of the SimpleOAuth gem.

Micky.get(
  'https://api.twitter.com/1.1/statuses/user_timeline.json',
  oauth: {
    consumer_key: 'l0tSAl3tT3RsAnD1G1tS',
    consumer_secret: 'l0tSAl3tT3RsAnD1G1tS',
    token: 'l0tSAl3tT3RsAnD1G1tS',
    token_secret: 'l0tSAl3tT3RsAnD1G1tS',
  },
)

To use the :oauth argument, just ensure simple_oauth is available:

gem 'simple_oauth'

Automatically parse responses into Ruby objects

Micky::Response#body always returns the response as a string. To parse this string into a Ruby object, use Micky::Response#data.

Responses with Content-Type: application/json are automatically parsed by Ruby’s JSON library.

response = Micky.get('http://urls.api.twitter.com/1/urls/count.json?url=dropmeme.com')
response.content_type # 'application/json'

# plain string
response.body # '{"count":33,"url":"http://dropmeme.com/"}'

# proper hash
response.data # {"count"=>33, "url"=>"http://dropmeme.com/"}

Add custom parsers

To add custom response parsers for specific content-types, insert lambdas in the Micky.parsers hash.

For instance, to parse HTML documents with Nokogiri:

Micky.parsers['text/html'] = -> (body) {
  Nokogiri::HTML(body)
}

Overwrite the default application/json parser to use Oj:

Micky.parsers['application/json'] = -> (body) {
  begin
    Oj.load(body)
  rescue Oj::ParseError
  end
}

Parse images into MiniMagick instances:

image_parser = -> (body) {
  begin
    MiniMagick::Image.read(body)
  rescue MiniMagick::Invalid
  end
}

%w[image/png image/jpeg image/jpg image/gif].each do |type|
  Micky.parsers[type] = image_parser
end

TODO

  • Support :basic_auth and :digest_auth through HTTPauth
  • Better document configuration options in README

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

© 2014 Rafaël Blais Masson. Micky is released under the MIT license.

About

Lightweight and worry-free HTTP client for Ruby

Topics

Resources

Stars

5 stars

Watchers

3 watching

Forks

Releases

Packages

Used by

Contributors

Languages