Giter VIP home page Giter VIP logo

remcached's Introduction

remcached

  • Ruby EventMachine memCACHED client implementation
  • provides a direct interface to the memcached protocol and its semantics
  • uses the memcached binary protocol to reduce parsing overhead on the server side (requires memcached >= 1.3)
  • supports multiple servers with simple round-robin key hashing (TODO: implement the libketama algorithm) in a fault-tolerant way
  • writing your own abstraction layer is recommended
  • uses RSpec
  • partially documented in RDoc-style

Callbacks

Each request may be passed a callback. These are not two-cased (success & failure) EM deferrables, but standard Ruby callbacks. The rationale behind this is that there are no usual success/failure responses, but you will want to evaluate a response[:status] yourself to check for cache miss, version conflict, network disconnects etc.

A callback may be kept if it returns :proceed to catch multi-response commands such as STAT.

remcached has been built with fault tolerance in mind: a callback will be called with just {:status => Memcached::Errors::DISCONNECTED} if the network connection has went away. Thus, you can expect your callback will be called, except of course you're using quiet commands. In that case, only a "non-usual response" from the server or a network failure will invoke your block.

Multi commands

The technique is described in the binary protocol spec in section 4.2. Memcached.multi_operation will help you exactly with that, sending lots of those quiet commands, except for the last, which will be a normal command to trigger an acknowledge for all commands.

This is of course implemented per-server to accomodate load-balancing.

Usage

First, pass your memcached servers to the library:

Memcached.servers = %w(localhost localhost:11212 localhost:11213)

Note that it won't be connected immediately. Use Memcached.usable? to check. This however complicates your own code and you can check response[:status] == Memcached::Errors::DISCONNECTED for network errors in all your response callbacks.

Further usage is pretty straight-forward:

Memcached.get(:key => 'Hello') do |response|
  case response[:status]
    when Memcached::Errors::NO_ERROR
      use_cached_value response[:value] # ...
    when Memcached::Errors::KEY_NOT_FOUND
      refresh_cache! # ...
    when Memcached::Errors::DISCONNECTED
      proceed_uncached # ...
    else
      cry_for_help # ...
    end
  end
end
Memcached.set(:key => 'Hello', :value => 'World',
              :expiration => 600) do |response|
  case response[:status]
    when Memcached::Errors::NO_ERROR
      # That's good
    when Memcached::Errors::DISCONNECTED
      # Maybe stop filling the cache for now?
    else
      # What could've gone wrong?
    end
  end
end

Multi-commands may require a bit of precaution:

Memcached.multi_get([{:key => 'foo'},
                     {:key => 'bar'}]) do |responses|
  # responses is now a hash of Key => Response
end

It's not guaranteed that any of these keys will be present in the response. Moreover, they may be present even if they are a usual response because the last request is always non-quiet.

HAPPY CACHING!

remcached's People

Contributors

astro avatar boutil avatar sumo-mbryant avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

remcached's Issues

multi_* methods working?

Do the multi methods work? I must be missing something.. Looking at the source, only the last request decrements pending_requests? I should see "value" => "Not Found" for all of these keys, no?

  Memcached.multi_get([{:key => 'foo'},{:key => 'bar'}, {:key => 'test'}]) do |res|
          p res
  end

# => {"foo"=>{:status=>65535}, "bar"=>{:status=>65535}, "test"=>{:status=>65535}}

clarify license

Currently I cannot find what license remcached is distributed under. Can you clarify it?

3 tests are failing with ruby 2.0

Running tests for ruby1.9.1 using debian/ruby-tests.rake ...
/usr/bin/ruby1.9.1 -S rspec ./spec/client_spec.rb ./spec/memcached_spec.rb ./spec/packet_spec.rb
.................................

Finished in 5.97 seconds
33 examples, 0 failures
/usr/bin/ruby2.0 -I/usr/lib/ruby/vendor_ruby /usr/lib/ruby/vendor_ruby/gem2deb/test_runner.rb
Running tests for ruby2.0 using debian/ruby-tests.rake ...
/usr/bin/ruby2.0 -S rspec ./spec/client_spec.rb ./spec/memcached_spec.rb ./spec/packet_spec.rb
..................FF...........F.

Failures:

  1. Memcached::Packet when generating a request example 4.2.1 should serialize correctly
    Failure/Error: "Hello"
    expected: "\x80\u0000\u0000\u0005\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0005\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000Hello"
    got: "\x80\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00Hello" (using ==)

    ./spec/packet_spec.rb:25:in `block (4 levels) in <top (required)>'

  2. Memcached::Packet when generating a request example 4.3.1 (add) should serialize correctly
    Failure/Error: "World"
    expected: "\x80\u0002\u0000\u0005\b\u0000\u0000\u0000\u0000\u0000\u0000\u0012\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u07AD\xBE\xEF\u0000\u0000\u000E\u0010HelloWorld"
    got: "\x80\x02\x00\x05\b\x00\x00\x00\x00\x00\x00\x12\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xDE\xAD\xBE\xEF\x00\x00\x0E\x10HelloWorld" (using ==)

    ./spec/packet_spec.rb:48:in `block (4 levels) in <top (required)>'

  3. Memcached::Packet when parsing a response example 4.2.1 should parse the body correctly
    Failure/Error: @pkt[:value].should == "World"
    expected: "World"
    got: "orld" (using ==)

    ./spec/packet_spec.rb:121:in `block (4 levels) in <top (required)>'

Finished in 6.59 seconds
33 examples, 3 failures

Failed examples:

rspec ./spec/packet_spec.rb:18 # Memcached::Packet when generating a request example 4.2.1 should serialize correctly
rspec ./spec/packet_spec.rb:38 # Memcached::Packet when generating a request example 4.3.1 (add) should serialize correctly
rspec ./spec/packet_spec.rb:119 # Memcached::Packet when parsing a response example 4.2.1 should parse the body correctly
/usr/bin/ruby2.0 -S rspec ./spec/client_spec.rb ./spec/memcached_spec.rb ./spec/packet_spec.rb failed
ERROR: Test "ruby2.0" failed. Exiting.

1.9 compatible?

Specs don't run / block on 1.9:

spec spec/client_spec.rb 
F

1)
NoMethodError in 'Memcached::Client when getting stats before(:all)'
undefined method `[]' for nil:NilClass
./spec/client_spec.rb:7:in `run'
./spec/client_spec.rb:20:

Finished in 21.633754 seconds

0 examples, 1 failure

Breaking under memcached 1.4.12

I think there may be a break in the remcached client when used in conjunction with the latest memcached release (1.4.12). When I launch my Goliath project after installing the latest memcache, I receive the following error when the memcached connection pool is established:

   /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/remcached-0.4.1/lib/remcached/client.rb:79:in `receive_data': undefined method `[]' for nil:NilClass (NoMethodError)
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run_machine'
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/eventmachine-1.0.0.beta.3/lib/eventmachine.rb:199:in `run'
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/em-synchrony-1.0.0/lib/em-synchrony.rb:27:in `synchrony'
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/goliath-0.9.4/lib/goliath/server.rb:73:in `start'
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/goliath-0.9.4/lib/goliath/runner.rb:236:in `run_server'
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/goliath-0.9.4/lib/goliath/runner.rb:170:in `run'
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/goliath-0.9.4/lib/goliath/application.rb:123:in `run!'
   from /Users/awatson/.rvm/gems/ruby-1.9.2-p180@proj/gems/goliath-0.9.4/lib/goliath/application.rb:141:in `block in '

I can see that Goliath makes the initial connection to memcache via memcache logging, but it seems that something bad happens in the client during the response. If I back down memcache to 1.4.10, the problem disappears.

Thanks,
Adam

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.