diff --git a/CHANGELOG.md b/CHANGELOG.md index 3753475..94006e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,6 @@ -### 1.0.2 (Next) +### 2.0.0 (Next) +* [#38](https://github.com/mongoid/mongoid-scroll/pull/38): Allow to reverse the scroll - [@GCorbel](https://github.com/GCorbel). * Your contribution here. ### 1.0.1 (2023/03/15) diff --git a/README.md b/README.md index 7e177df..6657577 100644 --- a/README.md +++ b/README.md @@ -69,27 +69,38 @@ end Scroll by `:position` and save a cursor to the last item. ```ruby -saved_cursor = nil -Feed::Item.desc(:position).limit(5).scroll do |record, next_cursor| +saved_iterator = nil + +Feed::Item.desc(:position).limit(5).scroll do |record, iterator| # each record, one-by-one - saved_cursor = next_cursor + saved_iterator = iterator end ``` -Resume iterating using the previously saved cursor. +Resume iterating using saved cursor and save the cursor to go backward. + +```ruby +Feed::Item.desc(:position).limit(5).scroll(saved_iterator.next_cursor) do |record, iterator| + # each record, one-by-one + saved_iterator = iterator +end +``` + +Loop over the first records again. ```ruby -Feed::Item.desc(:position).limit(5).scroll(saved_cursor) do |record, next_cursor| +Feed::Item.desc(:position).limit(5).scroll(saved_iterator.previous_cursor) do |record, iterator| # each record, one-by-one - saved_cursor = next_cursor + saved_iterator = iterator end ``` The iteration finishes when no more records are available. You can also finish iterating over the remaining records by omitting the query limit. ```ruby -Feed::Item.desc(:position).scroll(saved_cursor) do |record, next_cursor| +Feed::Item.desc(:position).limit(5).scroll(saved_iterator.next_cursor) do |record, iterator| # each record, one-by-one + saved_iterator = iterator end ``` @@ -98,19 +109,19 @@ end Scroll a `Mongo::Collection::View` and save a cursor to the last item. You must also supply a `field_type` of the sort criteria. ```ruby -saved_cursor = nil -client[:feed_items].find.sort(position: -1).limit(5).scroll(nil, { field_type: DateTime }) do |record, next_cursor| +saved_iterator = nil +client[:feed_items].find.sort(position: -1).limit(5).scroll(nil, { field_type: DateTime }) do |record, iterator| # each record, one-by-one - saved_cursor = next_cursor + saved_iterator = iterator end ``` Resume iterating using the previously saved cursor. ```ruby -session[:feed_items].find.sort(position: -1).limit(5).scroll(saved_cursor, { field_type: DateTime }) do |record, next_cursor| +session[:feed_items].find.sort(position: -1).limit(5).scroll(saved_iterator.next_cursor, { field_type: DateTime }) do |record, iterator| # each record, one-by-one - saved_cursor = next_cursor + saved_iterator = iterator end ``` @@ -179,15 +190,15 @@ Feed::Item.desc(:created_at).scroll(cursor) # Raises a Mongoid::Scroll::Errors:: ### Standard Cursor -The `Mongoid::Scroll::Cursor` encodes a value and a tiebreak ID separated by `:`, and does not include other options, such as scroll direction. Take extra care not to pass a cursor into a scroll with different options. +The `Mongoid::Scroll::Cursor` encodes a value and a tiebreak ID separated by `:`, and does not include other options, such as scroll direction. Take extra care not to pass a cursor into a scroll with different options. ### Base64 Encoded Cursor The `Mongoid::Scroll::Base64EncodedCursor` can be used instead of `Mongoid::Scroll::Cursor` to generate a base64-encoded string (using RFC 4648) containing all the information needed to rebuild a cursor. ```ruby -Feed::Item.desc(:position).limit(5).scroll(Mongoid::Scroll::Base64EncodedCursor) do |record, next_cursor| - # next_cursor is of type Mongoid::Scroll::Base64EncodedCursor +Feed::Item.desc(:position).limit(5).scroll(Mongoid::Scroll::Base64EncodedCursor) do |record, iterator| + # iterator.next_cursor is of type Mongoid::Scroll::Base64EncodedCursor end ``` diff --git a/UPGRADING.md b/UPGRADING.md index 67334d4..286f7fc 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -1,5 +1,25 @@ # Upgrading +## Upgrading to >= 2.0.0 + +The second argument yielded in the block in `Mongoid::Criteria::Scrollable#scroll` and `Mongo::Scrollable#scroll` has changed from a cursor to an instance of `Mongoid::Criteria::Scrollable` which provides `next_cursor` and `previous_cursor`. The `next_cursor` method returns the same cursor as in versions prior to 2.0.0. + +For example, this code: + +```ruby +Feed::Item.asc(field_name).limit(2).scroll(cursor) do |_, next_cursor| + cursor = next_cursor +end +``` + +Should be updated to: + +``` +Feed::Item.asc(field_name).limit(2).scroll(cursor) do |_, iterator| + cursor = iterator.next_cursor +end +``` + ## Upgrading to >= 1.0.0 ### Mismatched Sort Fields @@ -9,6 +29,6 @@ Both `Mongoid::Criteria::Scrollable#scroll` and `Mongo::Scrollable` now raise a For example, the following code will now raise a `MismatchedSortFieldsError` because we set a different field name (`position`) from the `created_at` field used to sort in `scroll`. ```ruby -cursor.field_name = "position" +cursor.field_name = "position" Feed::Item.desc(:created_at).scroll(cursor) ``` diff --git a/lib/config/locales/en.yml b/lib/config/locales/en.yml index 99ee357..6de3698 100644 --- a/lib/config/locales/en.yml +++ b/lib/config/locales/en.yml @@ -27,4 +27,7 @@ en: message: "Unsupported field type." summary: "The type of the field '%{field}' is not supported: %{type}." resolution: "Please open a feature request in https://github.com/mongoid/mongoid-scroll." - + unsupported_type: + message: "Unsupported type." + summary: "The type supplied in the cursor is not supported: %{type}." + resolution: "The cursor type can be either ':previous' or ':next'." diff --git a/lib/mongo/scrollable.rb b/lib/mongo/scrollable.rb index 74a80a2..1f40b59 100644 --- a/lib/mongo/scrollable.rb +++ b/lib/mongo/scrollable.rb @@ -16,21 +16,41 @@ def scroll(cursor_or_type = nil, options = nil, &_block) cursor_options = { field_name: scroll_field, direction: scroll_direction }.merge(options) cursor = cursor && cursor.is_a?(cursor_type) ? cursor : cursor_type.new(cursor, cursor_options) raise_mismatched_sort_fields_error!(cursor, cursor_options) if different_sort_fields?(cursor, cursor_options) - # make a view - view = Mongo::Collection::View.new( - view.collection, - view.selector.merge(cursor.criteria), - sort: (view.sort || {}).merge(_id: scroll_direction), - skip: skip, - limit: limit - ) + + records = nil + if cursor.type == :previous + # scroll backwards by reversing the sort order, limit and then reverse again + pipeline = [ + { '$match' => view.selector.merge(cursor.criteria) }, + { '$sort' => { scroll_field => -scroll_direction } }, + { '$limit' => limit }, + { '$sort' => { scroll_field => scroll_direction } } + ] + aggregation_options = view.options.except(:sort) + records = view.aggregate(pipeline, aggregation_options) + else + # make a view + records = Mongo::Collection::View.new( + view.collection, + view.selector.merge(cursor.criteria), + sort: (view.sort || {}).merge(_id: scroll_direction), + skip: skip, + limit: limit + ) + end # scroll if block_given? - view.each do |record| - yield record, cursor_type.from_record(record, cursor_options) + previous_cursor = nil + records.each do |record| + previous_cursor ||= cursor_type.from_record(record, cursor_options.merge(type: :previous)) + iterator = Mongoid::Criteria::Scrollable::Iterator.new( + previous_cursor: previous_cursor, + next_cursor: cursor_type.from_record(record, cursor_options) + ) + yield record, iterator end else - view + records end end end diff --git a/lib/mongoid-scroll.rb b/lib/mongoid-scroll.rb index b2e7088..7671ab2 100644 --- a/lib/mongoid-scroll.rb +++ b/lib/mongoid-scroll.rb @@ -11,5 +11,6 @@ require 'mongoid/scroll/base64_encoded_cursor' require 'mongoid/criteria/scrollable/fields' require 'mongoid/criteria/scrollable/cursors' +require 'mongoid/criteria/scrollable/iterator' require 'mongo/scrollable' if Object.const_defined?(:Mongo) require 'mongoid/criteria/scrollable' diff --git a/lib/mongoid/criteria/scrollable.rb b/lib/mongoid/criteria/scrollable.rb index bd70ee1..3e89327 100644 --- a/lib/mongoid/criteria/scrollable.rb +++ b/lib/mongoid/criteria/scrollable.rb @@ -12,13 +12,19 @@ def scroll(cursor_or_type = nil, &_block) cursor_options = build_cursor_options(criteria) cursor = cursor.is_a?(cursor_type) ? cursor : new_cursor(cursor_type, cursor, cursor_options) raise_mismatched_sort_fields_error!(cursor, cursor_options) if different_sort_fields?(cursor, cursor_options) - cursor_criteria = build_cursor_criteria(criteria, cursor) + records = find_records(criteria, cursor) if block_given? - cursor_criteria.order_by(_id: scroll_direction(criteria)).each do |record| - yield record, cursor_from_record(cursor_type, record, cursor_options) + previous_cursor = nil + records.each do |record| + previous_cursor ||= cursor_from_record(cursor_type, record, cursor_options.merge(type: :previous)) + iterator = Mongoid::Criteria::Scrollable::Iterator.new( + previous_cursor: previous_cursor, + next_cursor: cursor_from_record(cursor_type, record, cursor_options) + ) + yield record, iterator end else - cursor_criteria + records end end @@ -60,10 +66,21 @@ def new_cursor(cursor_type, cursor, cursor_options) cursor_type.new(cursor, cursor_options) end - def build_cursor_criteria(criteria, cursor) + def find_records(criteria, cursor) cursor_criteria = criteria.dup cursor_criteria.selector = { '$and' => [criteria.selector, cursor.criteria] } - cursor_criteria + if cursor.type == :previous + pipeline = [ + { '$match' => cursor_criteria.selector }, + { '$sort' => { cursor.field_name => -cursor.direction } }, + { '$limit' => criteria.options[:limit] }, + { '$sort' => { cursor.field_name => cursor.direction } } + ] + aggregation = cursor_criteria.view.aggregate(pipeline) + aggregation.map { |record| Mongoid::Factory.from_db(cursor_criteria.klass, record) } + else + cursor_criteria.order_by(_id: scroll_direction(criteria)) + end end def cursor_from_record(cursor_type, record, cursor_options) diff --git a/lib/mongoid/criteria/scrollable/iterator.rb b/lib/mongoid/criteria/scrollable/iterator.rb new file mode 100644 index 0000000..fb11b91 --- /dev/null +++ b/lib/mongoid/criteria/scrollable/iterator.rb @@ -0,0 +1,14 @@ +module Mongoid + class Criteria + module Scrollable + class Iterator + attr_accessor :previous_cursor, :next_cursor + + def initialize(previous_cursor:, next_cursor:) + @previous_cursor = previous_cursor + @next_cursor = next_cursor + end + end + end + end +end diff --git a/lib/mongoid/scroll/base64_encoded_cursor.rb b/lib/mongoid/scroll/base64_encoded_cursor.rb index b0c2b47..28c5d3b 100644 --- a/lib/mongoid/scroll/base64_encoded_cursor.rb +++ b/lib/mongoid/scroll/base64_encoded_cursor.rb @@ -18,7 +18,8 @@ def initialize(value, options = {}) field_name: parsed['field_name'], direction: parsed['direction'], include_current: parsed['include_current'], - tiebreak_id: parsed['tiebreak_id'] && !parsed['tiebreak_id'].empty? ? BSON::ObjectId.from_string(parsed['tiebreak_id']) : nil + tiebreak_id: parsed['tiebreak_id'] && !parsed['tiebreak_id'].empty? ? BSON::ObjectId.from_string(parsed['tiebreak_id']) : nil, + type: parsed['type'].try(:to_sym) } else super nil, options @@ -32,7 +33,8 @@ def to_s field_name: field_name, direction: direction, include_current: include_current, - tiebreak_id: tiebreak_id && tiebreak_id.to_s + tiebreak_id: tiebreak_id && tiebreak_id.to_s, + type: type }.to_json) end end diff --git a/lib/mongoid/scroll/base_cursor.rb b/lib/mongoid/scroll/base_cursor.rb index b34e225..ce69d42 100644 --- a/lib/mongoid/scroll/base_cursor.rb +++ b/lib/mongoid/scroll/base_cursor.rb @@ -1,7 +1,7 @@ module Mongoid module Scroll class BaseCursor - attr_accessor :value, :tiebreak_id, :field_type, :field_name, :direction, :include_current + attr_accessor :value, :tiebreak_id, :field_type, :field_name, :direction, :include_current, :type def initialize(value, options = {}) @value = value @@ -10,6 +10,9 @@ def initialize(value, options = {}) @field_name = options[:field_name] @direction = options[:direction] || 1 @include_current = options[:include_current] || false + @type = options[:type] || :next + + raise Mongoid::Scroll::Errors::UnsupportedTypeError.new(type: @type) if ![:previous, :next].include?(@type) end def criteria @@ -86,20 +89,23 @@ def extract_field_options(options) field_type: field_type.to_s, field_name: field_name.to_s, direction: options[:direction] || 1, - include_current: options[:include_current] || false + include_current: options[:include_current] || false, + type: options[:type].try(:to_sym) || :next } elsif options && (field = options[:field]) { field_type: field.type.to_s, field_name: field.name.to_s, direction: options[:direction] || 1, - include_current: options[:include_current] || false + include_current: options[:include_current] || false, + type: options[:type].try(:to_sym) || :next } end end def compare_direction - direction == 1 ? '$gt' : '$lt' + dir = type == :previous ? -direction : direction + dir == 1 ? '$gt' : '$lt' end def tiebreak_compare_direction diff --git a/lib/mongoid/scroll/errors.rb b/lib/mongoid/scroll/errors.rb index 2b20e4d..ecb685a 100644 --- a/lib/mongoid/scroll/errors.rb +++ b/lib/mongoid/scroll/errors.rb @@ -6,3 +6,4 @@ require 'mongoid/scroll/errors/invalid_base64_cursor_error' require 'mongoid/scroll/errors/no_such_field_error' require 'mongoid/scroll/errors/unsupported_field_type_error' +require 'mongoid/scroll/errors/unsupported_type_error' diff --git a/lib/mongoid/scroll/errors/unsupported_type_error.rb b/lib/mongoid/scroll/errors/unsupported_type_error.rb new file mode 100644 index 0000000..3743e69 --- /dev/null +++ b/lib/mongoid/scroll/errors/unsupported_type_error.rb @@ -0,0 +1,11 @@ +module Mongoid + module Scroll + module Errors + class UnsupportedTypeError < Mongoid::Scroll::Errors::Base + def initialize(opts = {}) + super(compose_message('unsupported_type', opts)) + end + end + end + end +end diff --git a/lib/mongoid/scroll/version.rb b/lib/mongoid/scroll/version.rb index 7a7a2b7..96bfa3d 100644 --- a/lib/mongoid/scroll/version.rb +++ b/lib/mongoid/scroll/version.rb @@ -1,5 +1,5 @@ module Mongoid module Scroll - VERSION = '1.0.2'.freeze + VERSION = '2.0.0'.freeze end end diff --git a/spec/mongo/collection_view_spec.rb b/spec/mongo/collection_view_spec.rb index c8e938c..2f329e3 100644 --- a/spec/mongo/collection_view_spec.rb +++ b/spec/mongo/collection_view_spec.rb @@ -57,7 +57,7 @@ context 'default' do it 'scrolls all' do records = [] - Mongoid.default_client['feed_items'].find.scroll(cursor_type) do |record, _next_cursor| + Mongoid.default_client['feed_items'].find.scroll(cursor_type) do |record, iterator| records << record end expect(records.size).to eq 10 @@ -68,7 +68,7 @@ context field_type do it 'scrolls all with a block' do records = [] - Mongoid.default_client['feed_items'].find.sort(field_name => 1).scroll(cursor_type, field_type: field_type) do |record, _next_cursor| + Mongoid.default_client['feed_items'].find.sort(field_name => 1).scroll(cursor_type, field_type: field_type) do |record, iterator| records << record end expect(records.size).to eq 10 @@ -77,15 +77,15 @@ it 'scrolls all with a break' do records = [] cursor = nil - Mongoid.default_client['feed_items'].find.sort(field_name => 1).limit(5).scroll(cursor_type, field_type: field_type) do |record, next_cursor| + Mongoid.default_client['feed_items'].find.sort(field_name => 1).limit(5).scroll(cursor_type, field_type: field_type) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor expect(cursor).to be_a cursor_type end expect(records.size).to eq 5 - Mongoid.default_client['feed_items'].find.sort(field_name => 1).scroll(cursor, field_type: field_type) do |record, next_cursor| + Mongoid.default_client['feed_items'].find.sort(field_name => 1).scroll(cursor, field_type: field_type) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor expect(cursor).to be_a cursor_type end expect(records.size).to eq 10 @@ -93,7 +93,7 @@ end it 'scrolls in descending order' do records = [] - Mongoid.default_client['feed_items'].find.sort(field_name => -1).limit(3).scroll(cursor_type, field_type: field_type, field_name: field_name) do |record, _next_cursor| + Mongoid.default_client['feed_items'].find.sort(field_name => -1).limit(3).scroll(cursor_type, field_type: field_type, field_name: field_name) do |record, iterator| records << record end expect(records.size).to eq 3 @@ -106,6 +106,27 @@ expect(cursor.value).to eq record[field_name.to_s] expect(cursor.tiebreak_id).to eq record['_id'] end + it 'can scroll back with the previous cursor' do + first_iterator = nil + second_iterator = nil + third_iterator = nil + + Mongoid.default_client['feed_items'].find.sort(field_name => 1).limit(2).scroll(cursor_type, field_type: field_type) do |_, iterator| + first_iterator = iterator + end + + Mongoid.default_client['feed_items'].find.sort(field_name => 1).limit(2).scroll(first_iterator.next_cursor, field_type: field_type) do |_, iterator| + second_iterator = iterator + end + + Mongoid.default_client['feed_items'].find.sort(field_name => 1).limit(2).scroll(second_iterator.next_cursor, field_type: field_type) do |_, iterator| + third_iterator = iterator + end + + records = Mongoid.default_client['feed_items'].find.sort(field_name => 1) + expect(Mongoid.default_client['feed_items'].find.sort(field_name => 1).limit(2).scroll(second_iterator.previous_cursor, field_type: field_type).to_a).to eq(records.limit(2).to_a) + expect(Mongoid.default_client['feed_items'].find.sort(field_name => 1).limit(2).scroll(third_iterator.previous_cursor, field_type: field_type).to_a).to eq(records.skip(2).limit(2).to_a) + end end end end @@ -124,12 +145,12 @@ it "scrolls by #{sort_order}" do records = [] cursor = nil - Mongoid.default_client['feed_items'].find.sort(sort_order).limit(2).scroll(cursor_type) do |record, next_cursor| + Mongoid.default_client['feed_items'].find.sort(sort_order).limit(2).scroll(cursor_type) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor end expect(records.size).to eq 2 - Mongoid.default_client['feed_items'].find.sort(sort_order).scroll(cursor) do |record, _next_cursor| + Mongoid.default_client['feed_items'].find.sort(sort_order).scroll(cursor) do |record, iterator| records << record end expect(records.size).to eq 3 diff --git a/spec/mongoid/base64_encoded_cursor_spec.rb b/spec/mongoid/base64_encoded_cursor_spec.rb index b5b9876..a373130 100644 --- a/spec/mongoid/base64_encoded_cursor_spec.rb +++ b/spec/mongoid/base64_encoded_cursor_spec.rb @@ -3,17 +3,18 @@ describe Mongoid::Scroll::Base64EncodedCursor do context 'new' do context 'an empty cursor' do - let(:base64_string) { 'eyJ2YWx1ZSI6bnVsbCwiZmllbGRfdHlwZSI6IlN0cmluZyIsImZpZWxkX25hbWUiOiJhX3N0cmluZyIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOm51bGx9' } + let(:base64_string) { 'eyJ2YWx1ZSI6bnVsbCwiZmllbGRfdHlwZSI6IlN0cmluZyIsImZpZWxkX25hbWUiOiJhX3N0cmluZyIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOm51bGwsInR5cGUiOiJuZXh0In0=' } subject do Mongoid::Scroll::Base64EncodedCursor.new base64_string end its(:tiebreak_id) { should be_nil } its(:value) { should be_nil } its(:criteria) { should eq({}) } + its(:type) { should eq(:next) } its(:to_s) { should eq(base64_string) } end context 'a string field cursor' do - let(:base64_string) { 'eyJ2YWx1ZSI6ImEgc3RyaW5nIiwiZmllbGRfdHlwZSI6IlN0cmluZyIsImZpZWxkX25hbWUiOiJhX3N0cmluZyIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2M2RmODA5NDQzNDE3YzdkMmIxMDIifQ==' } + let(:base64_string) { 'eyJ2YWx1ZSI6ImEgc3RyaW5nIiwiZmllbGRfdHlwZSI6IlN0cmluZyIsImZpZWxkX25hbWUiOiJhX3N0cmluZyIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2M2RmODA5NDQzNDE3YzdkMmIxMDIiLCJ0eXBlIjoibmV4dCJ9' } let(:a_value) { 'a string' } let(:tiebreak_id) { BSON::ObjectId.from_string('64063df809443417c7d2b102') } let(:criteria) do @@ -32,10 +33,11 @@ its(:value) { should eq a_value } its(:tiebreak_id) { should eq tiebreak_id } its(:criteria) { should eq(criteria) } + its(:type) { should eq(:next) } its(:to_s) { should eq(base64_string) } end context 'an id field cursor' do - let(:base64_string) { 'eyJ2YWx1ZSI6IjY0MDY0NTg0MDk0NDM0MjgxZmE3MWFiMiIsImZpZWxkX3R5cGUiOiJCU09OOjpPYmplY3RJZCIsImZpZWxkX25hbWUiOiJpZCIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2NDU4NDA5NDQzNDI4MWZhNzFhYjIifQ==' } + let(:base64_string) { 'eyJ2YWx1ZSI6IjY0MDY0NTg0MDk0NDM0MjgxZmE3MWFiMiIsImZpZWxkX3R5cGUiOiJCU09OOjpPYmplY3RJZCIsImZpZWxkX25hbWUiOiJpZCIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2NDU4NDA5NDQzNDI4MWZhNzFhYjIiLCJ0eXBlIjoibmV4dCJ9' } let(:a_value) { BSON::ObjectId('64064584094434281fa71ab2') } let(:tiebreak_id) { a_value } let(:criteria) do @@ -52,10 +54,11 @@ its(:value) { should eq a_value } its(:tiebreak_id) { should eq tiebreak_id } its(:criteria) { should eq(criteria) } + its(:type) { should eq(:next) } its(:to_s) { should eq(base64_string) } end context 'an integer field cursor' do - let(:base64_string) { 'eyJ2YWx1ZSI6MTAsImZpZWxkX3R5cGUiOiJJbnRlZ2VyIiwiZmllbGRfbmFtZSI6ImFfaW50ZWdlciIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2M2RmODA5NDQzNDE3YzdkMmIxMDgifQ==' } + let(:base64_string) { 'eyJ2YWx1ZSI6MTAsImZpZWxkX3R5cGUiOiJJbnRlZ2VyIiwiZmllbGRfbmFtZSI6ImFfaW50ZWdlciIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2M2RmODA5NDQzNDE3YzdkMmIxMDgiLCJ0eXBlIjoibmV4dCJ9' } let(:a_value) { 10 } let(:tiebreak_id) { BSON::ObjectId('64063df809443417c7d2b108') } let(:criteria) do @@ -74,10 +77,11 @@ its(:value) { should eq a_value } its(:tiebreak_id) { should eq tiebreak_id } its(:criteria) { should eq(criteria) } + its(:type) { should eq(:next) } its(:to_s) { should eq(base64_string) } end context 'a date/time field cursor' do - let(:base64_string) { 'eyJ2YWx1ZSI6MTM4NzU5MDEyMy4wLCJmaWVsZF90eXBlIjoiRGF0ZVRpbWUiLCJmaWVsZF9uYW1lIjoiYV9kYXRldGltZSIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2NDNhNzA5NDQzNDIzOWYyZGJmODYifQ==' } + let(:base64_string) { 'eyJ2YWx1ZSI6MTM4NzU5MDEyMy4wLCJmaWVsZF90eXBlIjoiRGF0ZVRpbWUiLCJmaWVsZF9uYW1lIjoiYV9kYXRldGltZSIsImRpcmVjdGlvbiI6MSwiaW5jbHVkZV9jdXJyZW50IjpmYWxzZSwidGllYnJlYWtfaWQiOiI2NDA2NDNhNzA5NDQzNDIzOWYyZGJmODYiLCJ0eXBlIjoibmV4dCJ9' } let(:a_value) { DateTime.new(2013, 12, 21, 1, 42, 3, 'UTC') } let(:tiebreak_id) { BSON::ObjectId('640643a7094434239f2dbf86') } let(:criteria) do @@ -94,10 +98,11 @@ its(:value) { should eq a_value } its(:tiebreak_id) { should eq tiebreak_id } its(:criteria) { should eq(criteria) } + its(:type) { should eq(:next) } its(:to_s) { should eq(base64_string) } end context 'a date field cursor' do - let(:base64_string) { 'eyJ2YWx1ZSI6MTM4NzU4NDAwMCwiZmllbGRfdHlwZSI6IkRhdGUiLCJmaWVsZF9uYW1lIjoiYV9kYXRlIiwiZGlyZWN0aW9uIjoxLCJpbmNsdWRlX2N1cnJlbnQiOmZhbHNlLCJ0aWVicmVha19pZCI6IjY0MDY0MmM5MDk0NDM0MjEyYzRkNDQyMCJ9' } + let(:base64_string) { 'eyJ2YWx1ZSI6MTM4NzU4NDAwMCwiZmllbGRfdHlwZSI6IkRhdGUiLCJmaWVsZF9uYW1lIjoiYV9kYXRlIiwiZGlyZWN0aW9uIjoxLCJpbmNsdWRlX2N1cnJlbnQiOmZhbHNlLCJ0aWVicmVha19pZCI6IjY0MDY0MmM5MDk0NDM0MjEyYzRkNDQyMCIsInR5cGUiOiJuZXh0In0=' } let(:tiebreak_id) { BSON::ObjectId('640642c9094434212c4d4420') } let(:a_value) { Date.new(2013, 12, 21) } let(:criteria) do @@ -114,10 +119,11 @@ its(:value) { should eq a_value } its(:tiebreak_id) { should eq tiebreak_id } its(:criteria) { should eq(criteria) } + its(:type) { should eq(:next) } its(:to_s) { should eq(base64_string) } end context 'a time field cursor' do - let(:base64_string) { 'eyJ2YWx1ZSI6MTM4NzYwNTcyMy4wLCJmaWVsZF90eXBlIjoiVGltZSIsImZpZWxkX25hbWUiOiJhX3RpbWUiLCJkaXJlY3Rpb24iOjEsImluY2x1ZGVfY3VycmVudCI6ZmFsc2UsInRpZWJyZWFrX2lkIjoiNjQwNjNkNGEwOTQ0MzQxNjZiZDA1M2VkIn0=' } + let(:base64_string) { 'eyJ2YWx1ZSI6MTM4NzYwNTcyMy4wLCJmaWVsZF90eXBlIjoiVGltZSIsImZpZWxkX25hbWUiOiJhX3RpbWUiLCJkaXJlY3Rpb24iOjEsImluY2x1ZGVfY3VycmVudCI6ZmFsc2UsInRpZWJyZWFrX2lkIjoiNjQwNjNkNGEwOTQ0MzQxNjZiZDA1M2VkIiwidHlwZSI6Im5leHQifQ==' } let(:item_id) { BSON::ObjectId('640636f209443407333b46d4') } let(:a_value) { Time.new(2013, 12, 21, 6, 2, 3, '+00:00').utc } let(:tiebreak_id) { BSON::ObjectId('64063d4a094434166bd053ed') } @@ -136,6 +142,7 @@ its(:tiebreak_id) { tiebreak_id } its(:tiebreak_id) { should eq tiebreak_id } its(:criteria) { should eq(criteria) } + its(:type) { should eq(:next) } its(:to_s) { should eq(base64_string) } end context 'an invalid field cursor' do @@ -229,5 +236,22 @@ end.to raise_error Mongoid::Scroll::Errors::UnsupportedFieldTypeError, /The type of the field 'a_array' is not supported: Array./ end end + + it 'encode and decode type option' do + feed_item = Feed::Item.create! + cursor = Mongoid::Scroll::Base64EncodedCursor.from_record feed_item, field_name: 'id', field_type: BSON::ObjectId, type: :previous + expect(Mongoid::Scroll::Base64EncodedCursor.new(cursor.to_s).type).to eq(:previous) + end + context 'a cursor with previous set to true' do + let(:field_type) { BSON::ObjectId } + let(:field_name) { 'id' } + let(:feed_item) { Feed::Item.create! } + subject do + Mongoid::Scroll::Base64EncodedCursor.from_record feed_item, field_name: field_name, field_type: field_type, type: :previous + end + its(:value) { should eq feed_item._id } + its(:field_type) { should eq field_type.to_s } + its(:type) { should eq(:previous) } + end end end diff --git a/spec/mongoid/criteria_spec.rb b/spec/mongoid/criteria_spec.rb index 7f61625..26e5cd4 100644 --- a/spec/mongoid/criteria_spec.rb +++ b/spec/mongoid/criteria_spec.rb @@ -51,7 +51,7 @@ context 'default' do it 'scrolls all' do records = [] - Feed::Item.all.scroll(cursor_type) do |record, _next_cursor| + Feed::Item.all.scroll(cursor_type) do |record, _iterator| records << record end expect(records.size).to eq 10 @@ -63,11 +63,11 @@ criteria.limit(2).scroll(cursor_type) expect(criteria).to eq original_criteria cursor = nil - criteria.limit(2).scroll(cursor) do |_record, next_cursor| - cursor = next_cursor + criteria.limit(2).scroll(cursor) do |_record, iterator| + cursor = iterator.next_cursor end - criteria.scroll(cursor) do |_record, next_cursor| - cursor = next_cursor + criteria.scroll(cursor) do |_record, iterator| + cursor = iterator.next_cursor end expect(criteria).to eq original_criteria end @@ -83,7 +83,7 @@ context field_type do it 'scrolls all with a block' do records = [] - Feed::Item.asc(field_name).scroll(cursor_type) do |record, _next_cursor| + Feed::Item.asc(field_name).scroll(cursor_type) do |record, iterator| records << record end expect(records.size).to eq 10 @@ -92,14 +92,14 @@ it 'scrolls all with a break' do records = [] cursor = nil - Feed::Item.asc(field_name).limit(5).scroll(cursor_type) do |record, next_cursor| + Feed::Item.asc(field_name).limit(5).scroll(cursor_type) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor end expect(records.size).to eq 5 - Feed::Item.asc(field_name).scroll(cursor) do |record, next_cursor| + Feed::Item.asc(field_name).scroll(cursor) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor end expect(records.size).to eq 10 expect(records).to eq Feed::Item.all.to_a @@ -107,9 +107,9 @@ it 'scrolls from a cursor' do last_record = nil cursor = nil - Feed::Item.asc(field_name).limit(5).scroll(cursor_type) do |record, next_cursor| + Feed::Item.asc(field_name).limit(5).scroll(cursor_type) do |record, iterator| last_record = record - cursor = next_cursor + cursor = iterator.next_cursor end sixth_item = Feed::Item.asc(field_name).to_a[5] from_item = Feed::Item.asc(field_name).scroll(cursor).to_a.first @@ -118,9 +118,9 @@ it 'includes the current record when Mongoid::Scroll::Cursor#include_current is true' do last_record = nil cursor = nil - Feed::Item.asc(field_name).limit(5).scroll(cursor_type) do |record, next_cursor| + Feed::Item.asc(field_name).limit(5).scroll(cursor_type) do |record, iterator| last_record = record - cursor = next_cursor + cursor = iterator.next_cursor end fifth_item = last_record cursor.include_current = true @@ -129,7 +129,7 @@ end it 'scrolls in descending order' do records = [] - Feed::Item.desc(field_name).limit(3).scroll(cursor_type) do |record, _next_cursor| + Feed::Item.desc(field_name).limit(3).scroll(cursor_type) do |record, _iterator| records << record end expect(records.size).to eq 3 @@ -145,21 +145,42 @@ end it 'can be reused' do ids = Feed::Item.asc(field_name).limit(2).map(&:id) - Feed::Item.asc(field_name).limit(2).scroll(cursor_type) do |_, cursor| - cursor.include_current = true - expect(Feed::Item.asc(field_name).limit(2).scroll(cursor).pluck(:id)).to eq ids + Feed::Item.asc(field_name).limit(2).scroll(cursor_type) do |_, iterator| + iterator.next_cursor.include_current = true + expect(Feed::Item.asc(field_name).limit(2).scroll(iterator.next_cursor).pluck(:id)).to eq ids break end end it 'can be re-created and reused' do ids = Feed::Item.asc(field_name).limit(2).map(&:id) - Feed::Item.asc(field_name).limit(2).scroll(cursor_type) do |_, cursor| - new_cursor = cursor_type.new(cursor.to_s, field_type: field_type, field_name: field_name) + Feed::Item.asc(field_name).limit(2).scroll(cursor_type) do |_, iterator| + new_cursor = cursor_type.new(iterator.next_cursor.to_s, field_type: field_type, field_name: field_name) new_cursor.include_current = true expect(Feed::Item.asc(field_name).limit(2).scroll(new_cursor).pluck(:id)).to eq ids break end end + it 'can scroll back with the previous cursor' do + first_iterator = nil + second_iterator = nil + third_iterator = nil + + Feed::Item.asc(field_name).limit(2).scroll(cursor_type) do |_, iterator| + first_iterator = iterator + end + + Feed::Item.asc(field_name).limit(2).scroll(first_iterator.next_cursor) do |_, iterator| + second_iterator = iterator + end + + Feed::Item.asc(field_name).limit(2).scroll(second_iterator.next_cursor) do |_, iterator| + third_iterator = iterator + end + + records = Feed::Item.asc(field_name) + expect(Feed::Item.asc(field_name).limit(2).scroll(second_iterator.previous_cursor)).to eq(records.limit(2)) + expect(Feed::Item.asc(field_name).limit(2).scroll(third_iterator.previous_cursor)).to eq(records.skip(2).limit(2)) + end end end end @@ -190,16 +211,16 @@ ).asc(:a_time) records = [] cursor = nil - criteria.limit(2).scroll(cursor_type) do |record, next_cursor| + criteria.limit(2).scroll(cursor_type) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor end expect(records.size).to eq 2 expect(records.map(&:name)).to eq ['Feed Item 0', 'Feed Item 1'] records = [] - criteria.limit(2).scroll(cursor) do |record, next_cursor| + criteria.limit(2).scroll(cursor) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor end expect(records.size).to eq 1 expect(records.map(&:name)).to eq ['Feed Item 2'] @@ -213,7 +234,7 @@ it 'respects embedded queries' do records = [] criteria = @item.embedded_items.limit(2) - criteria.scroll(cursor_type) do |record, _next_cursor| + criteria.scroll(cursor_type) do |record, _iterator| records << record end expect(records.size).to eq 1 @@ -235,12 +256,12 @@ it "scrolls by #{sort_order}" do records = [] cursor = nil - Feed::Item.order_by(sort_order).limit(2).scroll(cursor_type) do |record, next_cursor| + Feed::Item.order_by(sort_order).limit(2).scroll(cursor_type) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor end expect(records.size).to eq 2 - Feed::Item.order_by(sort_order).scroll(cursor) do |record, _next_cursor| + Feed::Item.order_by(sort_order).scroll(cursor) do |record, _iterator| records << record end expect(records.size).to eq 3 @@ -268,9 +289,9 @@ it 'doesn\'t lose the precision when rebuilding the cursor' do records = [] cursor = nil - Feed::Item.order_by(a_datetime: 1).limit(2).scroll(cursor_type) do |record, next_cursor| + Feed::Item.order_by(a_datetime: 1).limit(2).scroll(cursor_type) do |record, iterator| records << record - cursor = next_cursor + cursor = iterator.next_cursor end expect(records).to eq [item_1, item_2] cursor = cursor_type.new(cursor.to_s, field: Feed::Item.fields['a_datetime']) diff --git a/spec/mongoid/cursor_spec.rb b/spec/mongoid/cursor_spec.rb index d50aaa7..4e59771 100644 --- a/spec/mongoid/cursor_spec.rb +++ b/spec/mongoid/cursor_spec.rb @@ -131,6 +131,14 @@ end.to raise_error ArgumentError end end + context 'an invalid type cursor' do + let(:feed_item) { Feed::Item.create!(a_string: 'astring') } + it 'raises Mongoid::Scroll::Errors::UnsupportedTypeError' do + expect do + Mongoid::Scroll::Cursor.new "#{feed_item.a_string}:#{feed_item.id}", field_name: 'a_string', field_type: String, include_current: true, type: :invalid + end.to raise_error Mongoid::Scroll::Errors::UnsupportedTypeError, /The type supplied in the cursor is not supported: invalid./ + end + end context 'a cursor with include_current set to true' do let(:feed_item) { Feed::Item.create!(a_string: 'astring') } subject do