thiagopradi / octopus Goto Github PK

Consider the following code:

g = Group.find(9)
Octopus.using(:slave1) do
  g.members.count  # (1) This reads from master, not the slave
  Group.find(8)  # (2) This also reads from master, not the slave
end

I have traced Octopus a little bit and this comes down to reload_connection being called on the model whenever an association is accessed.

I hope this is not by design, because it pretty much invalidates the use of Octopus for fine-grained load-balancing of read-only slaves. It looks like Octopus wants models to remember which shard they came from; that's fine if that "pinning" only applies to that actual model (although in my case I would like for case 1 to also read from the slave), but it reverts the connection for all models and all subsequent access within the using block.

In my opinion both (1) and (2) are bugs, but I can accept the behaviour in (1); ideally I would like an option to have it behave the way I like, however.

Background: We use Postgres 9.0 streaming replication, but due to the replication lag, only certain page components may be rendered using slave queries. So we want to wrap certain blocks with Octopus.using. Those page components necessarily refer to objects outside their scope; for example, there's a global, per-thread current_user context.

Hi,

With the new version 0.4.0 of the gem, I can't add Octopus in Rails 3.0.9 apps. When I add ar-octopus in Gemfile and Bundle, I have this error:

Bundler could not find compatible versions for gem "activerecord":
In Gemfile:
ar-octopus depends on
activerecord (= 3.0.6)

rails (= 3.0.9) depends on
  activerecord (3.0.9)

Any solutions?

Best regards,
Noé

Using Postgres9 and rails 3.0.7.
I noticed that after activating octopus (editing the shards.yml) and running the first migration, the schema.rb has "id => false" on each and every table/model.

It looks like using 0.2.0 breaks sharding for us. From what I can tell, looks like it's always using what's in database.yml instead of shards.yml. The short of it is that commenting out lib/octopus/model.rb line 93:

write_inheritable_attribute(:establish_connection, true)

fixes this, but I doubt that's the correct solution.

We're using Rails 3.0.0. We do not call establish_connection() in any of our models to establish different connections from what ActiveRecord::Base uses. However, it looks like in our case Rails is calling ActiveRecord::Base.establish_connection to, well, establish the connection. When it does so, the monkeypatch sets that attribute and establishes the connection. Later, the logic in ActiveRecord::Base.should_use_normal_connection? sees that the establish_connection attribute is set, and then never returns a connection proxy. I don't think we're doing anything weird to get Rails to call establish_connection on our behalf. Stack traces at the point of that monkeypatch don't seem to indicate that any code beyond Rails is actually being called.

For running some rake tasks, like db:test:clone_structure in Rails, or for specifying a particular model should always use a particular database, establish_connection() in ActiveRecord::Base handles this. However, when using Octopus, ActiveRecord::Base.connection, even in these cases, will refer to the Octopus proxy and the wrong database.

When running 'rake spec' in Rails project, with only development and production environments defined to use Octopus, db:test:clone_structure ends up trying to load the database on the development database instead of into the test database.

Since my theory, based on looking at the rake task, is that establish_connection() is at fault, I've created a test for it and pushed that test to my fork of Octopus. Not sure how to fix right now.

We use a master/slave database configuration, and Octopus is working well there.

We're having a small problem currently where there are a few tables which live on a slave database (by design), and we aren't seeing a way to declare octopus_use_shard :slave or anything of the sort. I understand we can define the full connection spec, but I'd rather just use a shard name.

Without such a setting, ActiveRecord tries to consult the master database to determine the structure of the table. Which of course produces a SQL error. Can you recommend anything simple we can do directly in our model with the existing codebase? Or suggest an approach or provide a patch to resolve it?

Thanks,

Randy

Does anyone know if Octopus can be used to support failover in a master-master circular replication environment? The docs say Octopus supports Master Slave replication with one master. But if the master goes down, that means there is no place to write to. So it seems logical to support a multi-master environment.

Ideally, one could just specify two master databases in database.yml, and if the connection to the first one fails, rails would automatically switch to the second until the first comes back. This seems simpler than master-slave replication support to me, since it doesn't require distinguishing between reads and writes. So I'm surprised I can't find any discussion of it.

If Octopus is not the correct tool for this purpose, I'd really appreciate it if someone could direct me to another tool which can accomplish this.

Thanks everyone.

Justin

Hey,

Using the gem in Amazon RDS.
When Amazon RDS sets up a read slave, there are times when you have 1 minute of replication latency.

Even though 1 minute is a rare case, 10 seconds latencies happen a few times a day.

From looking and the source code, this is obviously not handles, so I am thinking about implementing a Dirty kind of flag, so when the flag is true, you also read from the master.

Opinions? Advice?

Hi,

we are using Octopus for replication, which is mostly working fine, but sometimes we get an "Lost connection to MySQL server during query" (~ for 0.1% of the requests we get).

I found this post, which seems to be the issue we have: http://stackoverflow.com/questions/4983385/lost-connection-to-mysql-server-during-query-error-with-rails3-octopus-replicat

We already have reconnect: true in our config, but you were writing that this is only working with unicorn, but not with passenger.
Could you find a solution for passenger?

Our app stack:

Rails 3.0.10
Octopus 0.3.4
Passenger 3.0.9
MySQL 5.1.41

Our shards.yml:

octopus:
replicated: true
fully_replicated: true
verify_connection: true
environments:
- production

production:
slave1:
encoding: utf8
adapter: mysql
database: database
username: slave
password:
socket: /var/run/mysqld/mysqld.sock
reconnect: true

Backtrace:

…/active_record/connection_adapters/abstract_adapter.rb: 207:in log_instrumentation' …lib/active_record/connection_adapters/mysql_adapter.rb: 289:inexecute'
…lib/active_record/connection_adapters/mysql_adapter.rb: 633:in select' …rd/connection_adapters/abstract/database_statements.rb: 7:inselect_all'
…ive_record/connection_adapters/abstract/query_cache.rb: 56:in select_all' …le/ruby/1.8/gems/ar-octopus-0.3.4/lib/octopus/proxy.rb: 177:insend'
…le/ruby/1.8/gems/ar-octopus-0.3.4/lib/octopus/proxy.rb: 177:in send_queries_to_selected_slave' …le/ruby/1.8/gems/ar-octopus-0.3.4/lib/octopus/proxy.rb: 140:inmethod_missing'
…1.8/gems/activerecord-3.0.10/lib/active_record/base.rb: 473:in find_by_sql_ActiveRecord_self_name_find_by_sql' …gems/activerecord-3.0.10/lib/active_record/relation.rb: 64:into_a'
…rd-3.0.10/lib/active_record/relation/finder_methods.rb: 341:in find_first' …rd-3.0.10/lib/active_record/relation/finder_methods.rb: 122:infirst'
…rd-3.0.10/lib/active_record/relation/finder_methods.rb: 242:in send' …rd-3.0.10/lib/active_record/relation/finder_methods.rb: 242:infind_by_attributes'
…1.8/gems/activerecord-3.0.10/lib/active_record/base.rb: 993:in send' …1.8/gems/activerecord-3.0.10/lib/active_record/base.rb: 993:inmethod_missing_without_create_or_update'
…11110100150/vendor/plugins/create_or_update_by/init.rb: 37:in method_missing' …111110100150/app/controllers/application_controller.rb: 79:infind_site'

Thanks, Enrico

Hi,

I was wanting to give this gem a try, sounds promising.
I've got this database.yml:

production:
  adapter: mysql2
  encoding: utf8
  reconnect: false
  database: app_production
  pool: 5
  username: app
  password: app
  host: "192.168.254.10"

I've got this shards.yml:

octopus:
  replicated: true

  environments:
    - production

  production:
    slave1:
      host: 192.168.254.10
      adapter: mysql2
      database: app_production
      user: app
      password: app2386
    slave2:
      host: 192.168.254.11
      adapter: mysql2
      database: app_production
      user: app
      password: app2386

I've setup mysql master/slave and imported the dump of my application.
192.168.254.10 is the master, 192.168.254.11 is the slave. (I've also ran this app with slave1 commented, since I thought it may clash with the one configured in database.yml)

When running the application, I get the following in the logging:
Mysql2::Error (Access denied for user 'root'@'rails1' (using password: YES)):

(rails1 is the hostname of the server)
It seems like it doesn't pick up on what I've got configured

This is my bundle:

$ bundle show
Gems included by the bundle:
  * actionmailer (3.1.1)
  * actionpack (3.1.1)
  * activemodel (3.1.1)
  * activerecord (3.1.1)
  * activeresource (3.1.1)
  * activesupport (3.1.1)
  * albino (1.3.3)
  * ar-octopus (0.4.0 433befd)
  * arel (2.2.1)
  * awesome_print (0.4.0)
  * builder (3.0.0)
  * bundler (1.0.21)
  * cancan (1.6.7)
  * carmen (0.2.8)
  * chronic (0.6.5)
  * coffee-script (2.2.0)
  * coffee-script-source (1.1.3)
  * dnsruby (1.52)
  * erubis (2.7.0)
  * execjs (1.2.9)
  * geocoder (1.0.5)
  * haml (3.1.3)
  * hike (1.2.1)
  * httparty (0.8.1)
  * i18n (0.6.0)
  * ipaddress (0.8.0)
  * jquery-rails (1.0.17)
  * json (1.6.1)
  * kaminari (0.12.4)
  * libv8 (3.3.10.2)
  * mail (2.3.0)
  * mime-types (1.17.2)
  * multi_json (1.0.3)
  * multi_xml (0.4.1)
  * mysql2 (0.3.10)
  * nokogiri (1.5.0)
  * paper_trail (2.5.0)
  * polyamorous (0.5.0)
  * polyglot (0.3.3)
  * posix-spawn (0.3.6)
  * progressbar (0.9.1)
  * rack (1.3.5)
  * rack-cache (1.1)
  * rack-mount (0.8.3)
  * rack-protection (1.1.4)
  * rack-ssl (1.3.2)
  * rack-test (0.6.1)
  * rails (3.1.1)
  * railties (3.1.1)
  * rake (0.9.2.2)
  * ransack (0.5.8)
  * rdoc (3.11)
  * redcarpet (1.17.2)
  * redis (2.2.2)
  * redis-namespace (1.0.3)
  * resque (1.19.0 af5a9a8)
  * ruby-ip (0.9.0)
  * sass (3.1.10)
  * sass-rails (3.1.4)
  * sinatra (1.3.1)
  * sprockets (2.0.3)
  * squeel (0.9.3)
  * test-unit (2.4.1)
  * therubyracer (0.9.9)
  * thor (0.14.6)
  * tilt (1.3.3)
  * treetop (1.4.10)
  * tzinfo (0.3.31)
  * uglifier (1.0.4)
  * vegas (0.1.8)
  * whenever (0.7.0)

First of all, thank you for this wonderful gem.

In my setting, I use Amazon RDS and a Read Replica. I want both to serve reading but only the master to serve writing. Is it possible to use the master as a slave (in shards.yml) as well?

Thank you for your help,

Thawatchai

I'm using shard-selection in my migrations. During the development process, all has been hunky-dory. But I recently started setting up the production database using shard.yml:

octopus:
  environments:
    - development
    - production
  development:
    first_shard:
      host: localhost
      adapter: mysql2
      user: root
      database: dev_first
    second_shard:
      host: localhost
      adapter: mysql2
      user: root
      database: dev_second
  production:
    first_shard:
      host: [Amazon RDS endpoint]
      adapter: mysql2
      user: [Amazon username]
      password: [Amazon password]
      database: prod_first
    second_shard:
      host: [Amazon RDS endpoint]
      adapter: mysql2
      user: [Amazon username]
      password: [Amazon password]
      database: prod_second

When I do migrations on non-sharded models, they go through just fine. But when my sharded migrations run, I get a credential error.

Access denied for user 'root'@'[IP address]' (using password: NO)

It appears that the sharded migrations are attempting to use the development credentials on the production database (using 'root' as a username), even though the system is running in production mode.

Any ideas?

For example if my controller loads @author from slave. Will it support @author.books (in the view) using slave too? Now I get a connection to master db and master doesnt have this kind of table (looks for books but in master_db). Thanks in advance.

In our current application we have a master wich is a PostgreSQL database and two shards wich are on Ms SQLServer. The problem we are facing is that the PostgreSQL syntax is being used on the queries performed on the shards.

We confirmed this behavior by setting the master as one a SQL Server too.

Here is the query performed on a shard (SQLServer database) generated when a PostgreSQL adapter is master:
SELECT [FASE2].* FROM [FASE2] JOIN fase2_enviar ON fase2.id = fase2_enviar.id LIMIT 50

And here is the 'correct' query when all adapters are SQLServer:
SELECT TOP(50) [FASE2].* FROM [FASE2] JOIN fase2_enviar ON fase2.id = fase2_enviar.id

It seems that Octopus Logger prints the shard used on a new line, below the SQL statement. This is one of my outputs while doing development:

```SQL (0.7ms) describeusers_cases_list_selected_session_attributes
Shard: master - SQL (1.2ms) SHOW TABLES
Shard: master - User Load (1.4ms) SELECT`users`.* FROM`users`WHERE`users`.`id`= 6 LIMIT 1
Shard: master - SQL (1.2ms) SHOW TABLES
Shard: master - Chart Load (0.6ms) SELECT`charts`.* FROM`charts`WHERE`charts`.`id` = 13 LIMIT 1
Shard: slave1 -

That was really confusing to me at the beginning. I thought that Octopus didn't write to the correct shards. But I investigated the logs on MySQL servers (master and slave) and verified that it worked ok. So, the problem is with Logger.

Note: I am using ActiveRecord 3.0.9

How it handles the failover situations? e.g. lost connection, server is shutting down or gone away etc.

  def select_connection()
    @shards[shard_name].verify_active_connections! if @verify_connection 
    @shards[shard_name].connection()
  end

Can you add a page in the wiki discussing unit testing with Octopus sharding. I've got sharding enabled in my development environment but the test environment databases are not being correctly initialized. Should it work transparently or are there some custom setup/teardown steps.

Will try to take a look but I'm not sure what changes may have done this.... just observed behavior, warrants investigation.

This is my favorite feature from db-charmer that is missing in octopus. When a query happens on a DB other than master, the debug log contains a prefix in dark blue that tells me which DB was used. The relevant function is here:

https://github.com/kovyrin/db-charmer/blob/master/lib/db_charmer/abstract_adapter/log_formatting.rb

It would be wonderful if octopus also did this.

For example if you have 2 schemas where table names are the same but different structure. ScopeProxy correctly changes shard when we user Model.using(:slave) but keeps old attributes of master shard in that model. Strange thou if I do Model.using(:slave).connection it refreshes attributes.

One quick issue I've encountered -- I mainly use the Octopus.using(:shard_name) construct in my controllers where I make ActiveRecord queries, like:

Octopus.using(:shard) do
  @label = Label.find(params[:label_id])
end

However when I use @Label in a view template, I get an error because the labels table does not exist in the master shard. I fought with this for a couple of hours, since in the Rails console this works without any issues.

Then I realized the problem (I think). Rails 3 lazy loads database queries. So Label.find(id) does not query the database until the variable is used in the view template, and by then it appears that the Octopus database scoping has expired.

Doing this fixes the issue:

Octopus.using(:shard) do
  @label = Label.find(params[:label_id])
  puts @label
end

I would rather not have to print every object to the console before using it, though. Am I missing something here? Should I be using Octopus.using blocks in view code instead of controller code?

I checked out the octopus replication example and tried rake db:octopus and added passwds to my configs and rake db:octopus complains

srv01:~/workspace/octopus_replication_example$ rake db:octopus
DEPRECATION WARNING: Calling a method in Rails::Application is deprecated, please call it directly in your application constant OctopusReplicationExample::Application. (called from /home/ddutta/workspace/octopus_replication_example/Rakefile:7)
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: NO)'
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: NO)'
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: NO)'
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: NO)'
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: NO)'
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

shards config

octopus:
replicated: true
octopus_enviroments:
- development
- production
development:
shards:
slave1:
adapter: mysql
host: localhost
username: root
password: root
database: octopus_replication_2
slave2:
adapter: mysql
host: localhost
username: root
password: root
database: octopus_replication_3
slave3:
adapter: mysql
host: localhost
username: root
password: root
database: octopus_replication_4
slave4:
adapter: mysql
host: localhost
username: root
password: root
database: octopus_replication_5

Hi, I am running into a problem with Octopus not running sharded migrations. The sample app under octopus works perfectly if it's configure to run with sqlite.

But when I configure the database.yml and shards.yml to work with postgres, the migrations are run on the master database only. All my sharded databases are not migrated.

Attached are my shards.yml and database.yml, and Gemfile:
https://gist.github.com/866687

Those are the only files I changed for my sample app.

Can someone please tell me what am I doing wrong?

Thanks in advance.

Hello,

I'm using rails 3.0.10 with octopus. I have an error like that:

ActiveRecord::StatementInvalid: Mysql2::Error: UPDATE command denied to user 'reader'@'server' for table 'articles': UPDATE articles SET view_count = COALESCE(view_count, 0) + 1 WHERE articles.id = 233322

reader@server is the user for read operations, not writing

I don't have any other errors for writing operations in my website. This is the only one and it happens on this line of code:
Article.increment_counter(:view_count, @article.id)

Is there a bug here?
Thanks

Jerome

Hi,

I've been getting replication working with your gem and I think i've found a bug. I followed the instructions and these are my settings in shards.yml

octopus:
replicated: true
enviroments:
- development
- production
development:
slave:
adapter: mysql2
host: localhost
database: webapp_development
production:
slave:
adapter: mysql2
host: localhost
database: webapp_production

In database.yml I have test configured to use webapp_test:

development:
adapter: mysql2
encoding: utf8
database: webapp_development
username: root
password:
host: localhost
pool: 5
timeout: 5000

test: &test
adapter: mysql2
encoding: utf8
database: webapp_test
username: root
password:
host: localhost
pool: 5
timeout: 5000

production:
adapter: mysql2
encoding: utf8
database: webapp_production
username: root
password: root
host: some_other_host # the local slave is configured in shards.yml
pool: 5
timeout: 5000

cucumber:
<<: *test

At first it seemed that everything was working as it should when I was testing the replication manually by running through our application but then I ran into a problem when trying to run our tests.

When I try to run our cucumber tests with 'rake cucumber' it drops all the tables from my webapp_test database and deletes all the data from my webapp_development database. When I fill the databases again, each time I try to run the tests, the tables in webapp_test disappear. When I started investigating I also noticed that my webapp_development database no longer had any data.

If I run the same command without octopus this does not happen.

I continued some testing and found that if I run "RAILS_ENV=test rake cucumber" instead, then the tests work again and my webapp_development database is not cleared of data.

To get around the problem, i've updated my cucumber.rake file as follows:

desc 'Alias for cucumber:ok'
task :cucumber do
if (ENV['RAILS_ENV'] == "test")
Rake::Task['cucumber:ok'].invoke
else
system("rake cucumber RAILS_ENV=test")
end
end

But again, this is only needed if I'm running with the Octopus gem. I believe it's the command 'rake db:test:prepare' which isn't working correctly when Octupus is used (which is the first command cucumber runs before running the tests).

Running

rake db:create

the result is

beta_d already exists

even if the database exists. If I disable octopus, commenting the "- development" below, rake runs correctly.

octopus:
environments:

• development

  • production

  development:
  dummy:
  adapter: postgresql
  host: localhost
  database: xxx_d
  username: foobar
  password: foobar
  encoding: unicode
  beta:
  adapter: postgresql
  host: localhost
  database: beta_d
  username: foobar
  password: foobar
  encoding: unicode

I got following error when running rake task.

rake aborted!
undefined method announce' for classClass'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/core_ext/module/aliasing.rb:31:in alias_method' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/core_ext/module/aliasing.rb:31:inalias_method_chain'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/ar-octopus-0.3.4/lib/octopus/migration.rb:9:in singletonclass' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/ar-octopus-0.3.4/lib/octopus/migration.rb:3:inextended'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/ar-octopus-0.3.4/lib/octopus/migration.rb:73:in extend' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/ar-octopus-0.3.4/lib/octopus/migration.rb:73:in<top (required)>'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:240:in require' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:240:inblock in require'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:223:in block in load_dependency' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:640:innew_constants_in'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:223:in load_dependency' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:240:inrequire'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/ar-octopus-0.3.4/lib/octopus.rb:66:in <top (required)>' /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.14/lib/bundler/runtime.rb:68:inrequire'
/usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.14/lib/bundler/runtime.rb:68:in block (2 levels) in require' /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.14/lib/bundler/runtime.rb:66:ineach'
/usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.14/lib/bundler/runtime.rb:66:in block in require' /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.14/lib/bundler/runtime.rb:55:ineach'
/usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.14/lib/bundler/runtime.rb:55:in require' /usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.14/lib/bundler.rb:120:inrequire'
/home/arhannik/IPCM/rails/Rails_working_area/config/application.rb:7:in <top (required)>' /home/arhannik/IPCM/rails/Rails_working_area/Rakefile:4:inrequire'
/home/arhannik/IPCM/rails/Rails_working_area/Rakefile:4:in <top (required)>' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/rake_module.rb:25:inload'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/rake_module.rb:25:in load_rakefile' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:495:inraw_load_rakefile'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:78:in block in load_rakefile' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:129:instandard_exception_handling'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:77:in load_rakefile' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:61:inblock in run'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:129:in standard_exception_handling' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:59:inrun'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/gems/rake-0.9.2/bin/rake:32:in <top (required)>' /home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/bin/rake:19:inload'
/home/arhannik/IPCM/rails/Rails_working_area/activerecord-sqlite3-adapter/ruby/1.9.1/bin/rake:19:in `

Here is my gemfile.lock

specs:
capybara (1.1.1)
mime-types (>= 1.16)
nokogiri (>= 1.3.3)
rack (>= 1.0.0)
rack-test (>= 0.5.4)
selenium-webdriver (> 2.0)
xpath (> 0.1.4)

GEM
remote: http://rubygems.org/
specs:
actionmailer (3.1.0)
actionpack (= 3.1.0)
mail (> 2.3.0)
actionpack (3.1.0)
activemodel (= 3.1.0)
activesupport (= 3.1.0)
builder (> 3.0.0)
erubis (> 2.7.0)
i18n (> 0.6)
rack (> 1.3.2)
rack-cache (> 1.0.3)
rack-mount (> 0.8.2)
rack-test (> 0.6.1)
sprockets (> 2.0.0)
activemodel (3.1.0)
activesupport (= 3.1.0)
bcrypt-ruby (> 3.0.0)
builder (> 3.0.0)
i18n (> 0.6)
activerecord (3.1.0)
activemodel (= 3.1.0)
activesupport (= 3.1.0)
arel (> 2.2.1)
tzinfo (> 0.3.29)
activeresource (3.1.0)
activemodel (= 3.1.0)
activesupport (= 3.1.0)
activesupport (3.1.0)
multi_json (> 1.0)
acts_as_audited (2.0.0)
rails (>= 3.0.3)
addressable (2.2.6)
ar-octopus (0.3.4)
activerecord (>= 2.3)
archive-tar-minitar (0.5.2)
arel (2.2.1)
bcrypt-ruby (3.0.1)
builder (3.0.0)
childprocess (0.2.2)
ffi (> 1.0.6)
coffee-script (2.2.0)
coffee-script-source
execjs
coffee-script-source (1.1.2)
columnize (0.3.4)
daemons (1.0.10)
database_cleaner (0.6.7)
diff-lcs (1.1.3)
erubis (2.7.0)
execjs (1.2.9)
multi_json (> 1.0)
factory_girl (2.2.0)
activesupport
factory_girl_rails (1.3.0)
factory_girl (> 2.2.0)
railties (>= 3.0.0)
faker (1.0.1)
i18n (> 0.4)
ffi (1.0.9)
gem_plugin (0.2.3)
hike (1.2.1)
i18n (0.6.0)
jquery-rails (1.0.13)
railties (> 3.0)
thor (> 0.14)
jshint (0.1.1)
json (1.6.1)
json_pure (1.6.1)
launchy (2.0.5)
addressable (> 2.2.6)
lexr (0.2.3)
libv8 (3.3.10.2)
linecache19 (0.5.12)
ruby_core_source (>= 0.1.4)
mail (2.3.0)
i18n (>= 0.4.0)
mime-types (> 1.16)
treetop (> 1.4.8)
math_engine (0.2.0)
lexr (>= 0.2.2)
mime-types (1.16)
mongrel (1.2.0.pre2)
daemons (> 1.0.10)
gem_plugin (> 0.2.3)
multi_json (1.0.3)
nokogiri (1.5.0)
polyglot (0.3.2)
rack (1.3.5)
rack-cache (1.0.3)
rack (>= 0.4)
rack-mount (0.8.3)
rack (>= 1.0.0)
rack-ssl (1.3.2)
rack
rack-test (0.6.1)
rack (>= 1.0)
rails (3.1.0)
actionmailer (= 3.1.0)
actionpack (= 3.1.0)
activerecord (= 3.1.0)
activeresource (= 3.1.0)
activesupport (= 3.1.0)
bundler (> 1.0)
railties (= 3.1.0)
rails3-jquery-autocomplete (0.9.1)
rails (> 3.0)
railties (3.1.0)
actionpack (= 3.1.0)
activesupport (= 3.1.0)
rack-ssl (> 1.3.2)
rake (>= 0.8.7)
rdoc (> 3.4)
thor (> 0.14.6)
rake (0.9.2)
rcov (0.9.11)
rdoc (3.11)
json (> 1.4)
rspec (2.6.0)
rspec-core (> 2.6.0)
rspec-expectations (> 2.6.0)
rspec-mocks (> 2.6.0)
rspec-core (2.6.4)
rspec-expectations (2.6.0)
diff-lcs (> 1.1.2)
rspec-mocks (2.6.0)
rspec-rails (2.6.1)
actionpack (> 3.0)
activesupport (> 3.0)
railties (> 3.0)
rspec (> 2.6.0)
ruby-debug-base19 (0.11.25)
columnize (>= 0.3.1)
linecache19 (>= 0.5.11)
ruby_core_source (>= 0.1.4)
ruby-debug19 (0.11.6)
columnize (>= 0.3.1)
linecache19 (>= 0.5.11)
ruby-debug-base19 (>= 0.11.19)
ruby_core_source (0.1.5)
archive-tar-minitar (>= 0.5.2)
rubyzip (0.9.4)
sass (3.1.10)
sass-rails (3.1.0)
actionpack (> 3.1.0)
railties (> 3.1.0)
sass (>= 3.1.4)
selenium-client (1.2.18)
selenium-webdriver (2.9.1)
childprocess (>= 0.2.1)
ffi (= 1.0.9)
json_pure
rubyzip
sprockets (2.0.3)
hike (> 1.2)
rack (> 1.0)
tilt (> 1.1, != 1.3.0)
sqlite3 (1.3.4)
therubyracer (0.9.4)
libv8 (> 3.3.10)
thor (0.14.6)
tilt (1.3.3)
treetop (1.4.10)
polyglot
polyglot (>= 0.3.1)
tzinfo (0.3.30)
uglifier (1.0.4)
execjs (>= 0.3.0)
multi_json (>= 1.0.2)
will_paginate (3.0.pre2)
xpath (0.1.4)
nokogiri (~> 1.3)

PLATFORMS
ruby

DEPENDENCIES
acts_as_audited (= 2.0.0)
ar-octopus
capybara!
coffee-script (= 2.2.0)
database_cleaner
factory_girl_rails
faker
jquery-rails (= 1.0.13)
jshint
launchy
math_engine (= 0.2.0)
mongrel (>= 1.2.0.pre2)
rails (= 3.1.0)
rails3-jquery-autocomplete (= 0.9.1)
rake (= 0.9.2)
rcov
rspec (= 2.6.0)
rspec-rails
ruby-debug19
sass-rails (= 3.1.0)
selenium-client (= 1.2.18)
sqlite3 (= 1.3.4)
therubyracer (= 0.9.4)
uglifier
will_paginate (= 3.0.pre2)

It seems that there is a problem that is related to "environments" tag inside the shards.yml file. It confuses "rake" which incorrectly uses development database instead of test database to run tests.

The shards.yml file that I use is the following:

octopus:
  replicated: true
  fully_replicated: false
  verify_connection: true
  environments:
    - development
  development:
    slave1:
      adapter: mysql2
      encoding: utf8
      reconnect: false
      database: shop_development
      pool: 5
      username: root
      password:
      host: 10.0.0.122

With the lastest version from master, which mostly works with rails 3.1.x, migrations are broken.

ActiveRecord::ConnectionNotEstablished
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:321:in checkout_new_connection' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:265:inblock (2 levels) in checkout'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:261:in loop' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:261:inblock in checkout'
/Users/anthonymorrisjohnson/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:260:incheckout'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:162:in connection' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/bundler/gems/octopus-433befdc0970/lib/octopus/proxy.rb:86:inselect_connection'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/bundler/gems/octopus-433befdc0970/lib/octopus/proxy.rb:149:in method_missing' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/schema_dumper.rb:36:ininitialize'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/schema_dumper.rb:21:in new' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/schema_dumper.rb:21:indump'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/railties/databases.rake:349:in block (4 levels) in <top (required)>' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/railties/databases.rake:347:inopen'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/railties/databases.rake:347:in block (3 levels) in <top (required)>' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:205:incall'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:205:in block in execute' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:200:ineach'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:200:in execute' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:158:inblock in invoke_with_call_chain'
/Users/anthonymorrisjohnson/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:151:ininvoke_with_call_chain'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:144:in invoke' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/activerecord-3.1.1/lib/active_record/railties/databases.rake:162:inblock (2 levels) in <top (required)>'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:205:in call' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:205:inblock in execute'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:200:in each' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:200:inexecute'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:158:in block in invoke_with_call_chain' /Users/anthonymorrisjohnson/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/monitor.rb:201:inmon_synchronize'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:151:in invoke_with_call_chain' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/task.rb:144:ininvoke'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:116:in invoke_task' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:94:inblock (2 levels) in top_level'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:94:in each' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:94:inblock in top_level'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:133:in standard_exception_handling' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:88:intop_level'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:66:in block in run' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:133:instandard_exception_handling'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/lib/rake/application.rb:63:in run' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/gems/rake-0.9.2.2/bin/rake:33:in<top (required)>'
/Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/bin/rake:19:in load' /Users/anthonymorrisjohnson/.rvm/gems/ruby-1.9.2-p290/bin/rake:19:in

The issue appears to be that the connection is getting closed directly, rather than going through the proxy. I'm working on fixing it, but any insight would be much appreciated.

I am unable to get Octopus to work inside of a block.
When I do this it does not use the shard, it just uses the default db in database.yml.
Octopus.using(:slave_two) do
User.create(:name => "Mike")
end

The same thing happens when I use the around_filter inside the Controller. It doesn't use the shard I specify at all.

The only way I'm able to specify the shard db is by using this syntax:
User.using(:shard1).find_by_name("Joao")

Do you guys know of anything that would cause this? It would simplify my code a lot of I could use the other block methods.

My shards.yml file looks like this example:
octopus:
environments:
- development
- production
development:
shard_one:
host: localhost
adapter: mysql
database: app_development
production:
shard_one:
host: localhost
adapter: mysql
database: app_production

Either ThinkingSphinx or Octopus needs to support the other, since they don't work together currently:

https://github.com/freelancing-god/thinking-sphinx/pull/131

It doesn't seem that octopus raises any exceptions when the shards.yml file is incorrect? I changed a slave database which I know is not correct and it seems octopus just defaulted to the master one in database.yml.

Also, trying to use a non-existing shard does not yield an error?

User.using(:i_dont_exist).find(1) doesn't inform that it's actually returning the object from the master database in database.yml.

In proxy.rb

def should_send_queries_to_replicated_databases?(method)
  @replicated && method.to_s =~ /select/
end

Will match queries such as:

 INSERT INTO "table1" ("column1", "column2", ...)  SELECT "column3", "column4", ... FROM "table2"

Perhaps prefixing the regex with:

def should_send_queries_to_replicated_databases?(method)
  @replicated && method.to_s.strip =~ /^select/
end

Would be safer?

Hello, we 'd like to use octopus for production (3.0.9+), which is the suggested version? 0.4.0 seems to have a hard dependency on activerecord 3.0.6

Is master (9dd5249) stable enough for usage?

I assume there should be setter in Octopus module that will set a new @config variable like Octopus.set_config({"development"} => { "new_shard_name" }... ), and then use Project.using(:new_shard_name)..... cuz my problem is that Im loading info for connection from master db, and I dont use shards.yml.

Hi.

When I use preload_app true in unicorn config, I got this error in logs:

ActiveRecord::ConnectionNotEstablished

What I did wrong?

Octopus seems to override any explicit configuration.
We are using a legacy database and have a special need for this configuration.

With the following shards.yml:

octopus:
  replicated: true
  fully_replicated: true

  environments:
      - production
      - development

  development:
    slave1:
      adapter: mysql2
      encoding: utf8
      reconnect: false
      database: cornice_development
      pool: 5
      username: root
      password: 
      #socket: /tmp/mysql.sock
      host: localhost

  production:
    slave1:

My test database is wiped clean, including the schema, when I try to the rspec test suite or "rake db:test:prepare". If I remove the development entries from shards.yml, db:test:prepare correctly loads the schema in schema.rb into the test database.

Feature Request:

In a sharded environment, currently the schema_info is stored on whatever database is defined in the database.yml. It would great if the code used the shards schema_info. This would allow for adding new shards and gracefully recovering from a migration error on a specific shard.

Hey there, great work on Octopus so far! Thank you!

I noticed there is some work in place for using travis-ci including multiple gemfiles and .travis.yml. I searched travis for octopus and found nothing. At this point I decided to fork octopus just to run it on travis-ci. It seems to be failing and I am interested in getting it to work. Is there something that you all have run into that prevents testing octopus on travis-ci? Would you accept a pull request with a working travis-ci configuration? I think I may be sending you more pull requests in the future and would really like to use travis-ci in the process. Any thoughts, objections or information you all have for me would be welcomed.

Thanks,
bramswenson

Hello,

I have a pretty serious problem, I'm having random inserts into slave databases once in a while, I recently noticed that 2 or 3 of my slaves weren't up do date from master because there were duplicated keys in some tables caused by inserts that supposed to go to the master.

Please help!.

Here are my files:

config/database.yml

production:
  adapter: mysql2
  encoding: utf8
  database: db.name
  username: db.master.user
  password: db.master.pass
  host: db.master.domain

config/shards.yml

octopus:
  verify_connection: true
  replicated: true
  production:
    shards:
      slave:
        adapter: mysql2
        username: db.local.user
        password: db.local.pass
        host: localhost
        database: db.name

I'm using Octopus 0.3.2
There is some places where I use Model.using(:master).where ... to get the data from the master directly because over heavy load the replication (1 master 3 slaves) takes a while.
Can the use of using(:master) messing up all?

Thanks.

Well I have "projects" table with different schemas in master and in slave. The problem is when Im using Project.using(:slave) then Octopus ScopeProxy returns correct @Shard as :slave but not corrent @klass as Project(bla bla bla all fields from MASTER SHARD). but should return proper fields. How can I override this?

Your code

def initialize_replication(config)
  @replicated = true
  if config.has_key?("fully_replicated")
    @fully_replicated = config["fully_replicated"]
  else
    @fully_replicated = true
  end
  @slaves_list = @shards.keys.map {|sym| sym.to_s}.sort 
  @slaves_list.delete('master')   
end

It sets fully_replicated to true if you don't have the config in place, but your documentation says it needs to be there to turn it on.

Which is correct?

Investigating code since I haven't been able to get Octopus to work yet for me.

$ rake test
no such table: users : DELETE FROM "users" WHERE 1=1

I add and have improved this file.

RAILS_ROOT/lib/tasks/db_test_set_rails_env.rake

namespace :db do
  namespace :test do
    task :set_rails_env do
      Rails.env = 'test'
      Thread.current[:connection_proxy] = nil
    end
    task :load => ['db:test:set_rails_env', 'db:test:purge']
  end
end

ruby 1.8.7
rails 3.0.8

I'm writing an application using JRuby as a base where each client will be using their own database. I've got all other database sections working, however Octopus eludes me. Reason for doing so is that certain requirements force us to isolate client data. As clients get added, it would be best if the shard list were to be dynamically updatable.

I have followed the wiki and attempted to use (located in config/initializers/octopus.rb)

Octopus.setup do |config|
  config.environments = [:development, :staging]
  config.shards = "crazy_shard" => {"adapter" => "mysql", "database" => "development", "username" => "root", "host" => "localhost"}}
end

however I get an error whenever attempting to use the shard (via any means)

The driver encountered an unknown error: undefined method `symbolize_keys!' for # ActiveSupport::HashWithIndifferentAccess:0x29895454 

Now the weird part is that if I take that connection info and shove it into a shards.yml like so, everything works:

development:

shards:

crazy_shard:

host: localhost

adapter: mysql

database: Construcsys-Web_development

user: root

When using octopus on a legacy database schema (which does not use ActiveRecord naming conventions), my model often calls set_table_name, like so:

class BaseRecord < ActiveRecord::Base
  set_table_name :baserecords
end

Using octopus to access a BaseRecord row, the table name used in the generated SQL queries is always "base_records":

ruby-1.9.2-p136 :051 > BaseRecord.table_name
 => "base_records" 
ruby-1.9.2-p136 :052 > BaseRecord.using(:shard_one).first
ActiveRecord::StatementInvalid: Mysql::Error: Table 'shard_one_dev.base_records' doesn't exist: SELECT  `base_records`.* FROM `base_records` LIMIT 1

When "rake db:migrate" was done, "schema.rb" not correct was made.

schema.rb

create_table "users", :id => false, :force => true do |t|
  t.integer  "id",  :null => false
  t.text    "name"
  ...
  ...
end 

I want you to become it like this.

create_table "users", :force => true do |t|
  t.text    "name"
  ...
  ...
end 

The cause is in the following processing.

ActiveRecord::SchemaDumper

      def table(table, stream)
        columns = @connection.columns(table)
        begin
          tbl = StringIO.new

          # first dump primary key column
          if @connection.respond_to?(:pk_and_sequence_for)
            pk, pk_seq = @connection.pk_and_sequence_for(table)
          elsif @connection.respond_to?(:primary_key)
            pk = @connection.primary_key(table)
          end

pk is nil.
However, if respond_to? is not passed, it is correctly returned.

@connection #=> Octopus::Proxy
@connection.respond_to?(:primary_key)  #=> false
@connection.primary_key  #=> "id"

It went well when Octopus::Proxy was corrected as follows.

class Octopus::Proxy
  def respond_to?(method)
    if should_clean_connection?(method)
      conn = select_connection()
      self.last_current_shard = self.current_shard
      clean_proxy()
      conn.respond_to?(method)
    elsif should_send_queries_to_replicated_databases?(method)
      false     
    else
      select_connection().respond_to?(method)
    end
  end
end

ruby 1.8.7
rails 3.0.8

I'm using a replicated setup and am trying to send a specific query to the master.

My shards.yml file is as follows (except that I made it generic):

octopus:
replicated: true
fully_replicated: false
environments:
- development
- production
development:
active_master:
host: localhost
adapter: mysql
database: my_development
pool: 10
username: root
password:
encoding: utf8
reconnect: true
passive_master:
host: localhost
adapter: mysql
database: my_development
pool: 10
username: root
password:
encoding: utf8
reconnect: true
production:
passive_master:
host: localhost
adapter: mysql
database: my_production
pool: 15
username: root
password:
encoding: utf8
reconnect: true

To test, I loaded script/console production

I run a simple query such as: o = Order.using(:master).find(:first) and this query works just fine. I can then query the orders model all I want. However, as soon as I try and query any other model, I get the following error (regardless of if I have "using" in the code or not. Here's the error:

a = Account.using(:master).find(:first)
NoMethodError: undefined method current_shard=' for #<Account:0xb75ced8> from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/attribute_methods.rb:255:inmethod_missing'
from /var/ruby-enterprise/lib/ruby/gems/1.8/gems/ar-octopus-0.3.4/lib/octopus/model.rb:31:in set_current_shard' from /var/ruby-enterprise/lib/ruby/gems/1.8/gems/ar-octopus-0.3.4/lib/octopus/model.rb:39:inafter_initialize'
from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/callbacks.rb:347:in send' from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/callbacks.rb:347:incallback'
from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:1662:in send' from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:1662:ininstantiate'
from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:661:in find_by_sql' from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:661:incollect!'
from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:661:in find_by_sql' from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:1553:infind_every'
from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:1510:in find_initial' from /mnt/data/client-vhosts/httpdocs/my_production/vendor/rails/activerecord/lib/active_record/base.rb:613:infind'
from /var/ruby-enterprise/lib/ruby/gems/1.8/gems/ar-octopus-0.3.4/lib/octopus/scope_proxy.rb:29:in send' from /var/ruby-enterprise/lib/ruby/gems/1.8/gems/ar-octopus-0.3.4/lib/octopus/scope_proxy.rb:29:inmethod_missing'
from /var/ruby-enterprise/lib/ruby/gems/1.8/gems/ar-octopus-0.3.4/lib/octopus/proxy.rb:97:in run_queries_on_shard' from /var/ruby-enterprise/lib/ruby/gems/1.8/gems/ar-octopus-0.3.4/lib/octopus/scope_proxy.rb:28:inmethod_missing'
from (irb):4>>

This happens the same on both Windows running Ruby 1.8.7 and Linux running Enterprise Ruby 1.8.7
I'm using Rails 2.3.2

I've tried using both the gem and the latest from github, but both have the same problem.

I'm reprogramming a huge application in Rails 3, and we have 3 servers, 1 master 2 slaves, every slave has 1 web server, every web server reads from his own db and writes to the master.
I think I have configured my app for using octopus, but I can't get the logger to work, so I'm not sure if everything is working or not.
I ended adding ActiveRecord::Base.logger = Octopus::Logger.new('[FULL_ROUTE_TO_DEVELOPMENT_LOG]') and the log isn't showing me nothing.

Please help

When I provide intentionally bad configuration options to a slave database configuration exceptions aren't thrown. All requests route to the master database.

I've been trying to test octopus against my replicated database setup. I have a delay in replication so I can confirm reads are performed against the slave database and writes are performed against the master database.

I've launched the console and created objects like:

um = User.find_by_id(1).using(:master)

um.partner_id was originally 1

um.partner_id = 234

um.save

us = User.find_by_id(1).using(:slave1)

replication delay is 30 seconds. the slave database hasn't been updated yet. a direct query of the database confirms this

us.partner_id

234

The slave database shouldn't be updated yet. For that instance, user.partner_id should still be 1.

In order to double-check slave queries, I've provided intentionally incorrect values for database name, username, and password and I don't get any exceptions. Using an invalid symbol for the #using() clause also fails to throw an exception.

User.find_by_id(1).using(:mater)

succeeds where I would expect it to fail.