daddye / mini_record Goto Github PK

Just some suggestions, since schema's are defined in the model, it might be nice using Mongoid's (or similar) syntactical approach (as an alternative option).

Mongoid calls it Fields - should be more or less the same as what we can do with ActiveRecord. It could look something like:

class User < ActiveRecord::Base
  field :first_name        # defaults to String type
  field :last_name,        :type => String # Default, don't need to specify
  field :money_in_cents,   :type => Integer
  field :alcohol_in_blood, :type => Float
  field :birthdate,        :type => Date
  field :created_at,       :type => DateTime
  field :updated_at,       :type => DateTime

  # Arrays/Hashes not supported by SQL so omitted
end

class Post < ActiveRecord::Base
  field :title
  field :content, :type => :text # Mongoid doesn't have that,
                                 # String can be huge in MongoDB
  field :user_id,
    :type  => Integer, # Foreign Key
    :index => true     # Apply index without needing to use the #index method

  # Or if you want to use the #index method instead of :index => true
  index :user_id
end

The only thing that would be different would be Text since we have t.string and t.text in ActiveRecord, so if you'd take a similar approach you could actually simplify this by just using symbols instead of classes to define the field type, like so:

class User < ActiveRecord::Base
  field :first_name
  field :last_name,  :type => :string # default
  field :biography,  :type => :text
  field :birthdate,  :type => :date
  field :created_at, :type => :datetime
end

Then you could simply run something like send(options[:type]) and you wouldn't have to worry about Class name availability like Text, which doesn't exist, since String translates to VARCHAR and not TEXT in RDBMS.

This doesn't add any new functionality, but the syntax might be more appealing in the model layer.

The signature of TableDefinition.initialize was changed
from def initialize(types)
to def initialize(types, name, temporary, options)
in this commit: rails/rails@14d7dc0

Travis now use ActiveSupport 4.0, this is why my previous commit was failed with Travis: https://travis-ci.org/DAddYE/mini_record/jobs/8435470

Lets say I have 2 classes

class SchoolTeacher < ActiveRecord::Base
  self.table_name = 'school_teachers_foo'
  field :name
  has_and_belongs_to_many :school_students
end

class SchoolStudent < ActiveRecord::Base
  self.table_name='school_students_foo'
  field :name
  has_and_belongs_to_many :school_teachers
end

If I were to auto_upgrade on the SchoolStudent class for instance

SchoolStudent.auto_upgrade!

It creates a join_table 'school_students_foo_school_teachers'
[MiniRecord] Creating Join Table school_students_foo_school_teachers with keys school_student_id

However, Rails 4 does not look for a table with the same name

SchoolStudent.create(name: 'James')
SchoolStudent.first.school_teachers
irb(main):015:0> SchoolStudent.first.school_teachers
Mysql2::Error: Table 'hpbx_development.school_students_foo_teachers_foo' doesn't exist: SHOW FULL FIELDS FROM `school_students_foo_teachers_foo`
ActiveRecord::StatementInvalid: Mysql2::Error: Table 'hpbx_development.school_students_foo_teachers_foo' doesn't exist: SHOW FULL FIELDS FROM `school_students_foo_teachers_foo`

To summarize

• mini_record creates 'school_students_foo_school_teachers'
• activerecord looks for 'school_students_foo_teachers_foo'

There are 2 issues with mini_record join table naming convention

1. It does not take into account prefixes (school in my example's case)
2. It does not use the table_name for the association, it only uses the association name (school_teachers vs school_teachers_foo)

I understand I can change the join_table option when defining the has_and_belongs_to_many relationship, but I'd like to not have to set this option if I don't have to.

I don't know if anyone is interested, but I am going to work on a PR for this.

field :content, :was => 'body'
field :content2, :was => ['content1', 'content']

Any reason we should not try to implement it?

Cool project, it should have been in AR core since 1970.

Hey, didn't know how else to contact you? I can't find this app anymore, would you be able to send it to me?

Thanks,
Craig

It would be useful to have an option to upgrade models without actually destroy columns that are no more declared in models.

This would be great to

• grant a safer approach in production
• ease the effort of switching back to plain activerecord migrations

It would be also fine providing a rake task to clean the database tables from deleted columns

@DAddYE There is some issue about primary keys.

According to readme:

Option :as or :type if not provided is :string by default, you can use all ActiveRecord types:

But col :name, as: :primary_key doesn't work because of ActiveRecord::ConnectionAdapters::TableDefinition#primary_key has one argument only, but we send two.

Then if I fix this issue (by removing options argument) I get next error:

ActiveRecord::StatementInvalid: SQLite3::SQLException: table "caras" has more than one primary key: CREATE TABLE "caras" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "name" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL) 

What do you think?

Once I started using mini_record on a larger sqlite3 db (about 400k rows), I noticed that startups were taking an extremely long time -- more than an hour. I've tracked it down to mini_record always thinking that every string column has changed, thus triggering a connection.change_column, which in AR in sqlite3 means defining a new table and copying all the existing data over to it, then dropping the old and renaming.

repo is simple, run this script a couple times (i'm using activerecord 4.1.2):

require 'mini_record'
ActiveRecord::Base.logger = Logger.new(STDOUT)

class Log < ActiveRecord::Base
  establish_connection adapter: :sqlite3,
    database: 'logs.sqlite3'

  field :old_path
end
Log.auto_upgrade!

Every time you run it you'll see [MiniRecord] Detected schema change for logs.old_path#type from "varchar(255)" to "", and then the migration.

So I have:

class Call < ActiveRecord::Base

  col :receptionist_id, as: :integer, limit: 11
  col :closed, :holiday, as: :boolean, default: 0
  col :to, :from, :sid, :api_version, :status
  col :start, :end, as: :datetime
  col :duration, as: :integer
  timestamps

end

And on every page load it runs:

  DEBUG -  (18.0ms)  ALTER TABLE `calls` CHANGE `receptionist_id` `receptionist_id` int(11) DEFAULT NULL
  DEBUG -  (13.4ms)  ALTER TABLE `calls` CHANGE `closed` `closed` tinyint(1) DEFAULT 0
  DEBUG -  (14.5ms)  ALTER TABLE `calls` CHANGE `holiday` `holiday` tinyint(1) DEFAULT 0

I've also been running into this (which I think is related) as it runs before the stuff above:

  DEBUG -   RELOAD (0.0883ms) /Users/cj/Dropbox/sd/acd/receptionist/app/models/calls.rb
DEPRECATION WARNING: Database connections will not be closed automatically, please close your
database connection at the end of the thread by calling `close` on your
connection.  For example: ActiveRecord::Base.connection.close
. (called from mon_synchronize at /Users/cj/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/monitor.rb:211)
DEPRECATION WARNING: Database connections will not be closed automatically, please close your
database connection at the end of the thread by calling `close` on your
connection.  For example: ActiveRecord::Base.connection.close
. (called from mon_synchronize at /Users/cj/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/monitor.rb:211)
DEPRECATION WARNING: Database connections will not be closed automatically, please close your
database connection at the end of the thread by calling `close` on your
connection.  For example: ActiveRecord::Base.connection.close
. (called from mon_synchronize at /Users/cj/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/monitor.rb:211)
DEPRECATION WARNING: Database connections will not be closed automatically, please close your
database connection at the end of the thread by calling `close` on your
connection.  For example: ActiveRecord::Base.connection.close

It would be nice to have support for add_foreign_key. We could check to see if this gem exists and add the extra helper method, just like add_index. Thoughts?

So I just wanted to mention that text and any type of varchar is also limitless and that this is the recommended way of adding string columns in postgres.

Thinking of what I saw over on: https://github.com/DAddYE/mini_record/blob/master/lib/mini_record/auto_schema.rb#L142

With an empty database this doesn't seem to work because ActiveRecord + mini_record seems to insist that the table exist before hand. Is there a way to get it to create the tables from an empty database?

Thanks! I love this project.

Calling Model.auto_upgrade! for a second (or any subsequent time) raises a NameError after the initial run.

irb(main):001:0> Location.auto_upgrade!
NameError: no member 'limit' in struct
	from (irb):1

Model:

class Location < ApplicationRecord
  col :name,        as: :string, null: false
  col :description, as: :text
  timestamps

  validates :name, presence: true

  has_many :spaces
end

Full stack:

NameError: no member 'limit' in struct
Did you mean?  limit=
/usr/local/bundle/bundler/gems/mini_record-cc3edb544089/lib/mini_record/auto_schema.rb:253:in `[]'
/usr/local/bundle/bundler/gems/mini_record-cc3edb544089/lib/mini_record/auto_schema.rb:253:in `block in field_attr_changes'
/usr/local/bundle/bundler/gems/mini_record-cc3edb544089/lib/mini_record/auto_schema.rb:252:in `each'
/usr/local/bundle/bundler/gems/mini_record-cc3edb544089/lib/mini_record/auto_schema.rb:252:in `field_attr_changes'
/usr/local/bundle/bundler/gems/mini_record-cc3edb544089/lib/mini_record/auto_schema.rb:393:in `block in auto_upgrade!'
/usr/local/bundle/bundler/gems/mini_record-cc3edb544089/lib/mini_record/auto_schema.rb:391:in `each'
/usr/local/bundle/bundler/gems/mini_record-cc3edb544089/lib/mini_record/auto_schema.rb:391:in `auto_upgrade!'
/app/lib/tasks/mini_record.rake:10:in `block (3 levels) in <top (required)>'
/app/lib/tasks/mini_record.rake:4:in `each'
/app/lib/tasks/mini_record.rake:4:in `block (2 levels) in <top (required)>'
/usr/local/bundle/gems/rake-12.1.0/exe/rake:27:in `<top (required)>'

Versions:

rails: 5.1.4
mini_record: 0.5.0

In my bundler file

gem 'mini_record', '0.2.1'

When I play around with my models I get an error warning me that timestamps method is not defined. If I specify the git repo and rebundle it works

gem 'mini_record', :git => 'https://github.com/DAddYE/mini_record.git'

Also you will need to bump the version number and add documentation of this helper to the README

In auto_upgrade! if you choose a field with type :boolean it will call
connection.add_column table_name, column.name, column.type.to_sym, options

However, options will include :limit => 1 and in turn it will cause a query to PostgreSQL looking like
SELECT 'boolean(1)'::regtype::oid instead of the correct SELECT 'boolean'::regtype::oid.

I have a fix for this, but I am also working out some other issues with PostgreSQL and mini_record.

Hello. It's look like changing column type is not working at PostgreSQL. For example:

class Some < ActiveRecord::Base
  field :super_field, as: :integer
  auto_upgrade!
end

Works well. Then I change :integer to :text, reload app and Some class... And nothing happend. :( Is it for me only or it's a bug?

Any other changings (adding, removing columns etc.) works perfectly.

This is because when it is checking the existing indexes with connection.indexes it is not using the table_name like ActiveRecord does when creating the index to check if the names are the same. Consider the class below

class Foobar < ActiveRecord::Base
  self.table_name = 'foobars_changed'
  belongs_to :notify_subjectable, polymorphic: true
  suppress_index :notify_subjectable
  add_index :subscribable_id
  add_index :subscribable_type
end

If I Foobar.auto_upgrade! it will create on index named "index_foobars_changed_on_notify_subjectable_id". If I do Foobar.auto_upgrade! again, the code will look for "index_foobars_on_notify_subjectable_id" not "index_foobars_changed_on_notify_subjectable_id", so it will try and add index_foobars_changed_on_notify_subjectable_id again.

[MiniRecord] Adding index index_foobars_on_notify_subjectable_id :notify_subjectable_id on foobars_changed
ArgumentError: Index name 'index_foobars_changed_on_notify_subjectable_id' on table 'foobars_changed' already exists

I have a PR ready for this.

When trying to use MiniRecord with Rails 5 ActiveRecord, we get this error:

/Users/fred/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/mini_record-0.4.7/lib/mini_record/auto_schema.rb:22:in `init_table_definition': Unsupported number of args for ActiveRecord::ConnectionAdapters::TableDefinition.new() (ArgumentError)
    from /Users/fred/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/mini_record-0.4.7/lib/mini_record/auto_schema.rb:35:in `table_definition'
    from /Users/fred/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/mini_record-0.4.7/lib/mini_record/auto_schema.rb:121:in `block in field'
    from /Users/fred/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/mini_record-0.4.7/lib/mini_record/auto_schema.rb:111:in `each'
    from /Users/fred/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/mini_record-0.4.7/lib/mini_record/auto_schema.rb:111:in `field'
    from /Users/fred/dev/work/bitmaker/wdi-august-2016/11-intro-to-databases/film.rb:7:in `<class:Film>'
    from /Users/fred/dev/work/bitmaker/wdi-august-2016/11-intro-to-databases/film.rb:6:in `<top (required)>'
    from /Users/fred/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/fred/.rbenv/versions/2.3.1/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from server.rb:1:in `<main>'

This error comes from the init_table_definition class method in the MiniRecord::AutoSchema module.

It looks like Rails 5 changed the number of arguments in ActiveRecord::ConnectionAdapters::TableDefinition#initialize!

@DAddYE When I auto_upgrade! existing tables that use STI type column isn't created. Isn't it wrong?

My code changes the table_name class variable of an ActiveRecord::Base, auto_upgrade! was likely to fail if you have a has_and_belongs_to_many. This is because I was using gsub! on table_name itself and not a clone of the table names in my calculating the join table name.

Basically the situation is like the following:

Before:
SchoolTeacher.table_name is 'school_teachers'
SchoolStudent.table_name is 'school_students'.

The code I added would make the join table 'school_students_teachers' like Rails 4 expects. However, afterwards there is an unintended consequence.

After:
SchoolTeacher.table_name is 'teachers' NOT GOOD!!!
SchoolStudent.table_name is 'school_students'.

See PR #54
See commit fc4ccf5

I have a fix for this!

Hey,

So I've been using mini_record for fast prototyping, but when switching to production I tend to like migrations just so someone can't accidentally delete an entire column. What are you thoughts on giving mini_record the ability to generate a migration file with all the current models schema?

@DAddYE - I have added macro facility to our fork. If you like it I will create a PR.

This is to support gems (https://github.com/hzamani/acts_as_relation) or other class methods that need migrations like composed_of and acts_as_superclass.

Added mini_macro method (macro just seemed destined to clash with something somewhere) that just sends the first arg, self and rest to a module method in MiniRecord::Macros module (needs to be reopened in client code), e.g.

module MiniRecord
  module Macros
    def self.acts_as_superclass(model_class, *args)
      opts = args.extract_options!
      model_class.acts_as_superclass(opts)

      association_name = options[:as] || model_class.acts_as_association_name
      model_class.field "#{association_name}_id", type: :integer
      model_class.field "#{association_name}_type"
    end
  end
end

and used...

class Item < ActiveRecord::Base
  mini_macro :acts_as_superclass, as: :itemizable
  field :sku
  field :description
end

class PostalItem < ActiveRecord::Base
  acts_as :item, as: :itemizable
  field :carrier
  field :volume
end

class DownloadItem < ActiveRecord::Base
  acts_as :item, as: :itemizable
  field :bucket, :path
  field :checksum
  field :permission
end

Auto upgrade your schema, so if you know what you are doing you don't [lost => lose] your existing data!

Hi,

How would you go about adding indexes to specific columns?

class Customer::Base < ActiveRecord::Base
  self.abstract_class = true
  self.table_name_prefix = 'customer_'
end

class Customer::Account < Customer::Base
  field :name
end

class Customer::ServiceRequest < Customer::Base
  field :preferred_time, :as => :time
end

Customer::Account.auto_upgrade!
Customer::ServiceRequest.auto_upgrade!

The Customer::ServiceRequest creates name column which is defined in Customer::Account

The result is table with one extra column customer_service_request [name:string, prefered_time:time]

Is somehow possible to prevent the columns leaking to other tables?
Thank you.

I use ActiveRecord in Sinatra app with sinatra-activerecord. But I couldn't find any way to use mini_record in Sinatra. So How can I use mini_record instead of active record in Sinatra apps?

It seems that running auto_upgrade! can create problems for has_and_belongs_to_many associations. In particular, if you have a habtm association between groups and users, but then call ActiveRecord::Base.auto_upgrade!, this will change the groups_users table to have the columns "group_id" and "left_side_id" (which should be "user_id"). This makes the User#groups method fail.

I figured that ActiveRecord::Base.auto_upgrade! would update all tables. But it seems like this way won't work. So is there an easier way to update all tables, other than listing them manually and then calling auto_upgrade! on them? (If you do ActiveRecord::Base.descendants.each(&:auto_upgrade!) it creates the same error. I suppose this is because it was trying to update the association table itself?) This way of doing it seems fragile and prone to error (say, if you add a new table and forget to manually add it to the list).

A related question is whether there is a way to know which tables have "migrations pending", i.e. which haven't been auto_upgrade'd to a schema matching the model source code.

Is it possible to run a custom query just like you can in migrations t.column :direction, "ENUM('inbound', 'outbound')" ?

Hi,

So using the current git version if you do col :first_name, :last_name, index: true it will create the first_name and last_name columns, but it only creates the first_name index when it should create the last_name index too.

hi DAddYE:

class Page < ActiveRecord::Base
  key :title
  has_and_belongs_to_many :photogalleries
end

class Photogallery < ActiveRecord::Base
  key :title
  has_and_belongs_to_many :pages 
end

when create db:

`add_index_options': Index name 'index_pages_photogalleries_on_pages_photogallery_id_and_photogallery_id' on table 'pages_photogalleries' is too long; the limit is 64 characters (ArgumentError)

how to add custom index in habtm ?

Thanks

Would be great to have mini_record recognize relationships and generate the necessary fields for it

The hobo_fields gem currently works with all AR relations (belongs to, Many to many, belongs to polymorphic). Overall its a pretty robust solution. The only reason I'm not using it now is that they have not upgraded for Rails 3.1

http://blog.teksol.info/2010/03/03/hobo-fields-lots-of-activerecord-goodness

Hello. I don't use logger in my AR connection at production, but MiniRecord don't check that moment in most situations:

  # auto_schema.rb:183
  logger.debug "[MiniRecord] Dropping table #{name}" # no check
  # many times...
  # auto_schema.rb:349
  logger.debug "[MiniRecord] Detected schema change for #{table_name}.#{field}##{att} " +
                                 "from #{old_value.inspect} to #{value.inspect}" if logger # check!
  # auto_schema,rb:359
  logger.debug "[MiniRecord] Changing column #{table_name}.#{field} to new type #{new_type}" # no check again

So I have exception:

NoMethodError: undefined method `debug' for nil:NilClass
from /usr/local/lib/ruby/gems/1.9.1/gems/mini_record-0.4.4/lib/mini_record/auto_schema.rb:243:in `auto_upgrade!'

I think, it will be very simple and usefull addition if you'll paste logger check on every situations it used.

Thank you!

Hello! MiniRecord is awesome, especially with MySQL. While using with SQLite I have some problems with it:

• When I use decimal columns, everytime I call auto_upgrade! it re-create tables. My English is terrible, so let me show example:

class Some < ActiveRecord::Base
  field :one, :two, :three, as: :decimal, scale: 2
end
...
Some.auto_upgrade! # happyness
Some.auto_upgrade! # oh:
# [MiniRecord] Detected schema change for somes.one#scale from nil to 2
# [MiniRecord] Changing column somes.one to new type decimal
# re-creating table with tempolary table for each column
Some.auto_upgrade! # and again...

• [critical] While creating tempolaty tables on re-creating there are exceptions on binary fields (there was MessagePack objects in my case):

SQLite3::SQLException: unrecognized token: "<binary_data_here>": INSERT INTO "altered_somes" ...

So, changing fields with MiniRecord is impossible with any binary data in table.

• When using MySQL there is a warning:

Error adding decimal column: precision cannot be empty if scale if specified

But when using SQLite there is no any warnings. And in AR's migrations there is no such warning too.

Of cource, all of that are not End of the World, because I can use MySQL instead, but maybe it will usefull information for you. :)

If you remove a has_and_belongs_to_many association the table is not deleted.

I want to do something like the following:

class CustomerVerification < Verification
    field :customer_uid,
          as: :integer,
          index: {
            column: [:customer_uid, :state],
            unique: true,
            where: 'state NOT IN (-1, -10)'
          }
end

Is it possible? With the postgresql adapter I can do the following:

add_index :verifications, [:customer_uid, :state], where: 'state NOT IN (-1, -10)', unique: true

Unfortunately this feature seems to be missing with mini_record?

All my foreign key indexes are getting nuked

Project.auto_upgrade!
   (6675.9ms)  ALTER TABLE `projects` ADD `attachments_count` int(11) DEFAULT 0
   (261.3ms)  DROP INDEX `index_projects_on_site_id` ON `projects`
   (98.4ms)  DROP INDEX `index_projects_on_estimate_id` ON `projects`
   (78.3ms)  DROP INDEX `index_projects_on_client_id` ON `projects`
   (79.3ms)  DROP INDEX `index_projects_on_vendor_id` ON `projects`
   (107.8ms)  DROP INDEX `index_projects_on_product_id` ON `projects`

:(

Do we have a method to automatically create the magic timestamp fields :created_at and :updated_at?

MiniRecord looks fantastic - thanks!!!

Question: During a migration, sometimes I need to to more than just modify the database schema. For example, I might want to migrate data from an old column to a new column, before deleting the old column.

class TestMigration < ActiveRecord::Migration
  def self.up
    add_column :phones, :number, :string
    execute ("update phones set number = concat(area_code, prefix, suffix)")
    remove_column :phones, :area_code
    remove_column :phones, :prefix
    remove_column :phones, :suffix
  end
end

Can I do something like this with mini_record ??

Whenever I load my environment and then access a model, various SQL alterations take place. If I quit the app, reboot it and access the model again the same thing happens (even though the schema is obviously already up to date). I'm not sure how but mini_record needs a better way to check if columns already match their definitions before altering them.

I think I have a good idea for handling changing column names, it also is good documentation for the person using the model.

lets say you have

class Post < ActiveRecord::Base
  col :comments
end

and you want to change it to just :comment you would do:

class Post < ActiveRecord::Base
  col :comment, was: :comments
end

I think it's simple and would work well, plus it would provide good documentation so they actually knew what it was before. If you then wanted to change it back you'd just do col :comments, was: :comment or if you wanted to change :comment to something else you'd just do col :something_else, was: :comment.

What do you think?