Giter VIP home page Giter VIP logo

mongoid_orderable's Introduction

Gem Version Build Status

Mongoid Orderable

Mongoid::Orderable is a ordered list implementation for your Mongoid 7+ projects.

Core Features

  • Sets a position index field on your documents which allows you to sort them in order.
  • Uses MongoDB's $inc operator to batch-update position.
  • Supports scope for position index, including changing scopes.
  • Supports multiple position indexes on the same document.
  • (Optional) Uses MongoDB transactions to ensure order integrity during concurrent updates.

Version Support

As of version 6.0.0, Mongoid::Orderable supports the following dependency versions:

  • Ruby 2.6+
  • Mongoid 7.0+
  • Rails 5.2+

For older versions, please use Mongoid::Orderable 5.x and earlier.

Transaction support requires MongoDB 4.2+ (4.4+ recommended.)

Usage

Getting Started

gem 'mongoid_orderable'

Include Mongoid::Orderable into your model and call orderable method. Embedded objects are automatically scoped to their parent.

class MyModel
  include Mongoid::Document
  include Mongoid::Orderable

  belongs_to :group
  belongs_to :drawer, class_name: "Office::Drawer",
             foreign_key: "legacy_drawer_key_id"

  orderable

  # if you set :scope as a symbol, it will resolve the foreign key from relation
  orderable scope: :drawer, field: :pos

  # if you set :scope as a string, it will use it as the field name for scope
  orderable scope: 'drawer', field: :pos

  # scope can also be a proc
  orderable scope: ->(doc) { where(group_id: doc.group_id) }

  # this one if you want specify indexes manually
  orderable index: false 

  # count position from zero as the top-most value (1 is the default value)
  orderable base: 0
end

You can also set default config values in an initializer, which will be applied when calling the orderable macro in a model.

# configs/initializers/mongoid_orderable.rb
Mongoid::Orderable.configure do |config|
  config.field = :pos
  config.base = 0
  config.index = false
end

Moving Position

item.move_to 2 # just change position
item.move_to! 2 # and save
item.move_to = 2 # assignable method

# symbol position
item.move_to :top
item.move_to :bottom
item.move_to :higher
item.move_to :lower

# generated methods
item.move_to_top
item.move_to_bottom
item.move_higher
item.move_lower

item.next_items # return a collection of items higher on the list
item.previous_items # return a collection of items lower on the list

item.next_item # returns the next item in the list
item.previous_item # returns the previous item in the list

Multiple Fields

You can also define multiple orderable fields for any class including the Mongoid::Orderable module.

class Book
  include Mongoid::Document
  include Mongoid::Orderable

  orderable base: 0
  orderable field: sno, as: :serial_no
end

The above defines two different orderable_fields on Book - position and serial_no. The following helpers are generated in this case:

book.move_#{field}_to
book.move_#{field}_to=
book.move_#{field}_to!

book.move_#{field}_to_top
book.move_#{field}_to_bottom
book.move_#{field}_higher
book.move_#{field}_lower

book.next_#{field}_items
book.previous_#{field}_items

book.next_#{field}_item
book.previous_#{field}_item

where #{field} is either position or serial_no.

When a model defines multiple orderable fields, the original helpers are also available and work on the first orderable field.

  @book1 = Book.create!
  @book2 = Book.create!
  @book2                 # => <Book _id: 53a16a2ba1bde4f746000001, serial_no: 1, position: 1>
  @book2.move_to! :top   # this will change the :position of the book to 0 (not serial_no)
  @book2                 # => <Book _id: 53a16a2ba1bde4f746000001, serial_no: 1, position: 0>

To specify any other orderable field as default pass the default: true option with orderable.

  orderable field: sno, as: :serial_no, default: true

Embedded Documents

class Question
  include Mongoid::Document
  include Mongoid::Orderable

  embedded_in :survey

  orderable
end

If you bulk import embedded documents without specifying their position, no field position will be written.

class Survey
  include Mongoid::Document

  embeds_many :questions, cascade_callbacks: true
end

To ensure the position is written correctly, you will need to set cascade_callbacks: true on the relation.

Disable Ordering

You can disable position tracking for specific documents using the :if and :unless options. This is in advanced scenarios where you want to control position manually for certain documents. In general, the disable condition should match a specific scope. Warning: If used improperly, this will cause your documents to become out-of-order.

class Book
  include Mongoid::Document
  include Mongoid::Orderable

  field :track_position, type: Boolean
  
  orderable if: :track_position, unless: -> { created_at < Date.parse('2020-01-01') }
end

Transaction Support

By default, Mongoid Orderable does not guarantee ordering consistency when doing multiple concurrent updates on documents. This means that instead of having positions 1, 2, 3, 4, 5, after running your system in production at scale your position data will become corrupted, e.g. 1, 1, 4, 4, 6. To remedy this, this Mongoid Orderable can use MongoDB transactions

Prerequisites

Configuration

You may enable transactions on both the global and model configs:

Mongoid::Orderable.configure do |config|
  config.use_transactions = true       # default: false
  config.transaction_max_retries = 10  # default: 10
end

class MyModel
  orderable :position, use_transactions: false
end

When two transactions are attempted at the same time, database-level WriteConflict failures may result and retries will be attempted. After transaction_max_retries has been exceeded, a Mongoid::Orderable::Errors::TransactionFailed error will be raised.

Locks

When using transactions, Mongoid Orderable creates a collection mongoid_orderable_locks which is used to store temporary lock objects. Lock collections use a TTL index which auto-deletes objects older than 1 day.

You can change the lock collection name globally or per model:

Mongoid::Orderable.configure do |config|
  config.lock_collection = "my_locks"  # default: "mongoid_orderable_locks"
end

class MyModel
  orderable :position, lock_collection: "my_model_locks"
end

MongoDB 4.2 Support

In MongoDB 4.2, collections cannot be created within transactions. Therefore, you will need to manually run the following command once to initialize the lock collection:

Mongoid::Orderable::Models::Lock.create!

This step is not necessary when using MongoDB 4.4+.

Contributing

Please fork the project on Github and raise a pull request including passing specs.

Copyright & License

Copyright (c) 2011 Arkadiy Zabazhanov, Johnny Shields, and contributors.

MIT license, see LICENSE for details.

mongoid_orderable's People

Contributors

bharat311 avatar danielspector avatar dblock avatar dsci avatar gpx avatar gurix avatar joeyaghion avatar johnnyshields avatar mattiassvedhem avatar omiyakejtakuma avatar pjkelly avatar pyromaniac avatar rafaelgaspar avatar rickcarlino avatar semaperepelitsa avatar tomasc avatar zhengjia avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

mongoid_orderable's Issues

deep embedded order able

Hey Hey, first thanks for this gem, this could save me a lot of code :-D But I found a little problem, that is I guess just not implemented at the moment. I would apply order able in a deep embedded document. The following spec explains the problem:

 class EmbedsOrderable
    include Mongoid::Document

    embeds_many :embedded_orderables
    embeds_many :deep_embedded_orderables
  end

  class DeepEmbeddedOrderable
    include Mongoid::Document

    embedded_in :embeds_orderable
    embeds_many :deeper_embedded_orderables
  end

  class DeeperEmbeddedOrderable
    include Mongoid::Document
    include Mongoid::Orderable

    embedded_in :deep_embedded_orderables
    orderable
  end

 describe DeepEmbeddedOrderable do
    before :each do
      EmbedsOrderable.delete_all
      eo = EmbedsOrderable.create!
      deep_eo = eo.deep_embedded_orderables.create!
      3.times do
        deep_eo.deeper_embedded_orderables.create!
      end
    end

    it 'moves an item returned by a query to position' do
      deep_eo = EmbedsOrderable.first.deep_embedded_orderables.first
      deeper_eo_1 = deep_eo.deeper_embedded_orderables.where(position: 1).first
      deeper_eo_2 = deep_eo.deeper_embedded_orderables.where(position: 2).first
      deeper_eo_3 = deep_eo.deeper_embedded_orderables.where(position: 3).first
      deeper_eo_1.move_to! 2
      deeper_eo_2.reload.position.should == 1
      deeper_eo_1.reload.position.should == 2
      deeper_eo_3.reload.position.should == 3
    end
  end

The next item that should swap after increasing is not updated.

 1) Mongoid::Orderable DeepEmbeddedOrderable moves an item returned by a query to position
     Failure/Error: deeper_eo_2.reload.position.should == 1
       expected: 1
            got: 2 (using ==)
     # ./spec/mongoid/orderable_spec.rb:454:in `block (3 levels) in <top (required)

I am trying to solve that problem by my own but this could take a while. Here is the example code: https://github.com/gurix/mongoid_orderable/compare/deep_orderable?expand=1

Orderable Updates Position when Destroying Parent Embedded In

When you call something like Grouping.find(1234).destroy the cascading call update the children position in the database rather then just destroying the parent data and moving along.

class Group
  include Mongoid::Document
  include Mongoid::Timestamps

  embeds_many :items, cascade_callbacks: true
end
class Item
  include Mongoid::Document
  include Mongoid::Timestamps
  include Mongoid::Orderable

  orderable

  embedded_in :group
end

This is not a the worst that can happen however it does slow down the destroy_all method and destroy method. It was a bit of extra work to track down but you can see it updating the position in the log file. One of the requirements is that you have to have more than one item embedded in the grouping but if you do it will result in the position field -1 for every nested item within the grouping excluding the first item that you destroyed.

Error with embedded document

I am trying to use this gem with embedded documents and I am receiving an error. The error that I am getting is

NoMethodError (undefined method `orderable_scope' for #<Summary:0x007fa6b3515160>):

I tried to fix the error but I am not sure what else need to be done. I have changed line https://github.com/pyromaniac/mongoid_orderable/blob/master/lib/mongoid/orderable.rb#L104 to

send(metadata.inverse).send(metadata.name).class.orderable_scope(self) 

as the scope is defined to the class.

This fixes the previous error but now I am receiving a new error

Mongoid::Errors::InvalidCollection (Access to the collection for Summary is not allowed since it is an embedded document, please access a collection from the root document.):

Can anyone shed some light on how I can fix this error?

keys must be strings or symbols

Getting keys must be strings or symbols error when using mongoid_orderable.

I think it has something to do with dataMapper which I am using besides mongid (porting data from anoher server with dataMapper)

I've notices collisions between dataMapper and mongoid, since they both extend Symbol with various methods, asc, desc etc. Could this be the same problem?

Here is a snippet from my stacktrace if it is any help:

keys must be strings or symbols
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/bson-1.6.2/lib/bson/bson_c.rb:24:in `serialize'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/bson-1.6.2/lib/bson/bson_c.rb:24:in `serialize'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongo-1.6.2/lib/mongo/collection.rb:436:in `update'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid-2.4.8/lib/mongoid/collections/master.rb:25:in `block in update'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid-2.4.8/lib/mongoid/collections/retry.rb:29:in `retry_on_connection_failure'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid-2.4.8/lib/mongoid/collections/master.rb:24:in `update'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid-2.4.8/lib/mongoid/collection.rb:149:in `update'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid_orderable-1.0.0/lib/mongoid_orderable/mongoid/contexts/mongo.rb:6:in `inc'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid_orderable-1.0.0/lib/mongoid_orderable/mongoid/criteria.rb:1:in `inc'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid_orderable-1.0.0/lib/mongoid/orderable.rb:125:in `apply_position'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/mongoid_orderable-1.0.0/lib/mongoid/orderable.rb:85:in `add_to_list'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/activesupport-3.2.2/lib/active_support/callbacks.rb:419:in `block in _run__3008157942946527494__save__4591629889417243504__callbacks'
/Users/hmm/.rbenv/versions/1.9.2-p290/lib/ruby/gems/1.9.1/gems/activesupport-3.2.2/lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_742'

Empty objects when destroying multiple documents.

So when I do something like this on a model that has orderable scoped to it's parent:

Place.first.photos.all.each { |p| p.destroy }
MONGODB (0ms) app['places'].update({"_id"=>BSON::ObjectId('4f99f0eea2b279ec3d000006')}, {"$pull"=>{"photos"=>{"_id"=>BSON::ObjectId('50531aae66eaa76979000001')}}})
MONGODB (0ms) app['places'].update({"_id"=>BSON::ObjectId('4f99f0eea2b279ec3d000006'), "photos._id"=>BSON::ObjectId('50531b1166eaa76979000002')}, {"$inc"=>{"photos.1.position"=>-1}})
MONGODB (0ms) app['places'].update({"_id"=>BSON::ObjectId('4f99f0eea2b279ec3d000006')}, {"$pull"=>{"photos"=>{"_id"=>BSON::ObjectId('50531b1166eaa76979000002')}}})

Then what I'm left with is (note: position is decremented).

=> #<Photo _id: , _type: nil, width: nil, height: nil, crop_x: "", crop_y: "", crop_h: "", crop_w: "", title: nil, image: nil, position: -1>

This is due to the fact that remove_from_list is implemented like this:

def remove_from_list
    orderable_scoped.where(orderable_column.gt => orderable_position).inc(orderable_column, -1)
end

And I guess that somehow that command conflicts with the delete that happens directly after the atomic operation.

Note, that in reality I'm not looping through the photos, The issue happens when I hit DELETE to /photos multiple times on each of the photos (it's in an API).

Do you have any Idea why this is happening? and do you have a solution?

Thanks a lot.

Index doesn't respect scope

The declared index is always on position only.

class Person
  include Mongoid::Document
  include Mongoid::Orderable
  field :parent
  orderable scope: :parent
end

Person.index_specifications.map(&:key)  # => [{:position=>1}]

Instead, I believe it should be prefixed by the scope (when defined). In the above example, it should resolve to:

[{:parent => 1, :position=>1}]

before_create support broken.

class FieldCategory
  include Mongoid::Orderable
  include Mongoid::Document

  field :name, type: String
  orderable scope: :organization
  before_validation :move_to_top, on: :create  # I would rather before_create this....
end
    it "sets list position to top by default" do
      cat1 = FactoryGirl.create(:field_category, name: "cat 1", organization: @org1)
      cat2 = FactoryGirl.create(:field_category, name: "cat 2", organization: @org1)
      cat3 = FactoryGirl.create(:field_category, name: "cat 3", organization: @org1)

      [cat1,cat2,cat3].map(&:reload)
      [cat1.position, cat2.position, cat3.position].should == [3,2,1]
    end

^ passes with before_validation, fails with before_create

Providing default column value when not specified.

Hi again,

I am getting duplicate column keys when I don't specify the column value on creation. Is this expected behavior?

class Sequence
  embeds_many :steps
end

class Step
  include Mongoid::Document
  include Mongoid::Orderable

  embedded_in :sequence

  field :rank, type: Integer
  orderable scope: :sequence, column: :rank
end

seq = Sequence.create
Step.new(sequence: seq).save # Can't use create() in my use case
Step.new(sequence: seq).save
seq.steps.pluck(:rank)
# => [1, 1]
# Not unique ^

can I use mongoid_orderable in hash field?

I have lots of records, they have different position in different table.

so the schema is like:

class Record
  field :tables, type: Hash, default: {}
  #tables: {
  #   [table_id]: { position: 0, hide: true }
  #}
end

can I use mongoid_orderable in Record.tables[table_id].position?

undefined method mongoid7? for an installation of Mongoid at 6.4.X

Hi there, when I tried to start a rails app after bundle install it errors out like this:

gems/mongoid_orderable-5.2.0/lib/mongoid_orderable.rb:18:in module:MongoidOrderable': undefined method mongoid7?' for Mongoid::Compatibility::Version:Module (NoMethodError)

This is against a mongoid gem at 6.4.4

diving into the code base, at line 18 of mongoid_orderable.rb it does show that it's checking against mongoid 7 only.

image

Does anyone else have the same problem?

Please make me an admin of this repo

@dblock I'd like to take over as an admin, can you please add me to this repo and to Rubygems?

I'd like release a version 6.0 which:

  • Upgrades to Mongoid 7.0 as minimum version
  • Optionally uses MongoDB transactions to fix the out-of-sync issues. I have a proof of concept for this which is working.

My business relies heavily on this gem and amazingly our clients have put up with ordering issues for years. Time to fix them.

BUG: I can't add item in the middle of the list.

Seems like a bug. Or no functionaly for that. Or i do smth wrong.
Steps to reproduce.
``1. rails c

Model

class Test
  ### INCLUDES AND EXTENDS
  include Mongoid::Document
  include Mongoid::Timestamps
  include Mongoid::Orderable

  orderable field: :position

  ### Fields
  field :rate, type: Integer
  field :position,   type: Integer
end
Test.create!(rate: 80)
Test.create!(rate: 70)
Test.create!(rate: 60)

# Bug
Test.create!(rate: 30, position: 2)
Responce
#<Test _id: 64db673bc9876f69e0f19c91, created_at: 2023-08-15 11:53:27 UTC, updated_at: 2023-08-15 11:53:27 UTC, position: 4, rate: 30>

Help me or I write this functionality by myself.

Trigger a removement on position set to nil

Hi,

When using Mongoid::Orderable, I need a way to trigger a removement on the remaining items in a list when a specifc item's position is set to nil. If I have a list of 5 items ordered 1 through 5, I want to set the list item at position 3 to nil and then remaining items should collapse/reorder themselves to be sequential.

Here is a sample test of the functionality I'm trying to implement. The first expectation will pass but the second expectation will fail because the position is set to nil but removement is not triggered. I attempted to use item.move_to! nil but the position remained the same.

What's the best way to implement the required behavior? I don't have the option to destroy the list item which I know would trigger the removement.

Thank you!

require 'spec_helper'

class OrderableTest
  include Mongoid::Document
  include Mongoid::Orderable

  orderable
end

describe OrderableTest do
  it "should move all of the list items properly" do
    5.times do
      OrderableTest.create!
    end

    expect(OrderableTest.all.map(:position).sort).to eq([1, 2, 3, 4, 5])

    ot = OrderableTest.where(:position => 3).first
    ot.unset(:position)

    expect(OrderableTest.all.map(:position).compact.sort).to eq([1, 2, 3, 4])
  end
end

Is it possible to add `orderable` to multiple columns on a single model?

I am trying to do something like the following:

I have a model Bar with a year column which is a denormalized value. I want to use orderable for the entire set and then also be able to use orderable by year

class Bar
  include Mongoid::Document
  include Mongoid::Orderable

  orderable :scope => :foo, :column => :position
  orderable scope: lambda { |document| where(foo_id: document.foo_id), year: document.year }, :column => :year_position
end

Is this possible?

Order gets out-of-sync with parallel updates

I've had a nagging issue with Mongoid::Orderable where, over time, randomly in my production database the position field will start looking like:

0, 0, 1, 1, 2, 3, 4, 4, 4, etc.

My theory as to why this happens is:

        def apply_position column, target_position
           ...
          elsif target_position < pos
            MongoidOrderable.inc(scope.where(col.gte => target_position, col.lt => pos), col, 1)
          elsif target_position > pos
            MongoidOrderable.inc(scope.where(col.gt => pos, col.lte => target_position), col, -1)
          end
        end

The apply_position method increments the range between "current_position" and "target_position". If someone is moving lots of items around quickly, I believe we can get a stale value for the current_position--possibly from a replica set instance, or maybe a race condition between sending two "move_to" requests to two separate web servers. In other words, in the current implementation we can't be 100% sure that the current_position is accurate.

For now, I'm going to work around this issue by adding a delay on the frontend, so when you move one object you can't trigger another request under the server responds. It would be better if we had an atomic way to read and set the position.

Suggest to use zero-based position instead of 1-based position?

I'd like to suggest to make the lowest position as 0 rather than 1, or at least provide an option to do so. Ruby, MongoDB, and Javascript all use zero-based arrays, so in various parts of my code I have to subtract 1 from object.position when reading, and add 1 when saving.

Problem when using default scope.

I have Items which can be published/unpublished with a default_scope: default_scope where(state: 'published')

If you try to update state on an unpublished Item it will not work.
This is because mongoid_orderable is executing a find in a before_save(updating position) callback, which will not find current document(since default scope is published and current document is unpublished).
All .find in mongoid_orderable should be unscoped.find to avoid this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.