integreat-io / integreat-transporter-mongodb Goto Github PK

Sorting should be specified by setting a sort object on an endpoint.

Example endpoint:

{
  scope: 'collection',
  type: 'entry',
  sort: {
    'attributes.updatedAt': -1
  }
}

The sort object will be passed directly to MongoDB's sort() method as is.

Must differentiate urls and connection options.

Right now, several items are set or deleted by calling the mongodb 'one' methods for each.

The sourceOptions object will not be available until Integreat 0.7, so wait a bit with this issue.

This is easy when fetching one page. With multiple pages, the total number of items will probably have to be fetched in a separate query.

In pagination, pageId is a filter that is applied to start at the right document. This filter also becomes part of the filter we use to count documents, and so affects the total count. We should only use the initial filter for the total count ...

When a GET request contains a pageSize param, paging kicks in, and the response should contain a paging object with params for getting the next page of results. The param object should be included with the params in the request for the next page.

For example, the action ...

{type: 'GET', payload: {type: 'entry', pageSize: 20}}

... will return a response looking like this:

{
  status: 'ok',
  data: [...],
  paging: {
    next: {
      query: {_id: {$gte: 'ent19'}},
      pageAfter: 'ent19',
      pageSize: 20
  }
}

What the adapter will do on this request, is stepping through the results until it reaches the document with id: 'ent19' (from the pageAfterprop), and return the next 20 items after that one. To make this a bit more efficient, the query object is added to the query from endpoint, so in this case, the pageAfter id will always be the first in the result set.

When the endpoint has a sort object, the sorting props will be use to build the query object for the next page, so that the pageAfter id will be included in the result, but not be further down than necessary.

For example, with descending sorting on attributes.updatedAt, the result will look like this:

{
  status: 'ok',
  data: [...],
  paging: {
    next: {
      query: {'attributes.updated': {$lte: new Date(...)}},
      pageAfter: 'ent19',
      pageSize: 20
    }
  }
}

As the sorting is ascending, by getting every entry that has an updatedAt date earlier than that of the last document on the previous page, we're sure to get the pageAfter document and we'll know where to start returning documents for the next page.

Find a proper way to inform of partial errors, and return only items that was actually written to the database.