Can you use this for pagination or how would you use this together with pagination like for example how would you paginate would you need to use the page number before like so

http://localhost:3000/api/tst/users/1/?_id=5d74d73ad8ba0e04f80ee346
the number 1 being the page number
or after the query parameters although doing that after causes an error
I'm a newbie I apologize for this question

const aqp = require('api-query-params').default; 
const test = aqp('timestamp>2016-01-01');
 console.log(test);
 const test2 = aqp('timestamp>2016-01-01',{
   whitelist : ['timestamp']
 });
 console.log(test2);

the code above gives :

{ filter: { timestamp: { '$gt': 2016-01-01T00:00:00.000Z } } }
{ filter: {} }

$ node -v
v6.5.0

I have the following Model:

const userSchema = new mongoose.Schema({
   name: String,
   by: { type: mongoose.Schema.ObjectId, ref: 'User' }
})
const UserModel = mongoose.model('User', userSchema, 'users')

and I am using api-query-params in one of my controllers:

const aqp = require('api-query-params')
app.get('/users', (req, res, next) => {
   const docs = await UserModel.find(aqp.filter).populate(aqp.population)
   res.status(200).json({  data: docs })
})

Let's say there's a user called A who created himself somehow.
I am requesting: http://localhost:4000/users?populate=by and I am getting the response as follows:

{ # 200 OK
   data: [
      {
          name: A,
          by: {
              name: A,
              by: 62e7960ebc9aa651523bf5b2 // Object Id
          }
      }
   ]
}

I am OK with this response, however, the problem comes when someone requests http://localhost:4000/users?populate=by.by.by.by

the response becomes:

{
    # 200 OK
    data: [{
        name: A,
        by: {
            name: A,
            by: {
                name: A,
                by: {
                    name: A,
                    by: 62e7960ebc9aa651523bf5b2 // Object ID
                }
            }
        }
    }]
}

Is there an option where I can only allow one level of populating in aqp (api-query-params)?

Hi, would be nice if you could transpile it with Babel so require and import can work in the same way. The import syntax is not supported in nodejs. Thanks.

It would be nice to have a few lines of documentation showing that you should use spaces when you want to populate multiple nested fields.

When you have a request that looks like:
/orders/${id}?populate=quotation.customer,quotation.quotlines,movements

you end up with a populate object that looks like:
population [ { path: 'quotation', populate: { path: 'quotlines' } }, { path: 'movements' } ]
Instead of :
population [ { path: 'quotation', populate: [{ path: 'quotlines' }, { path: 'customer' }] }, { path: 'movements' } ]

The solution to that is that your request should be made using spaces for consecutive fields from the same key, like so:
/orders/${id}?populate=quotation.customer quotlines,movements

That solution was provided by a veteran user from my team, but it would be nice for it to be documented in the readMe in the populate section.

Hey, I have had a strange bug with this, I dont know why but if you input this random id to the querystring like this:

let { filter} = aqp({ uid: '115524599024168443300' }

The filter gets this as output:

{ uid: 115524599024168440000 }

It somehow changes the last threes for zeros. Maybe its some type of overflow? I have solved it by setting that particular castParam to string, but I suppose you wanted to know it.

We are having an issues with the != operator. When we try to query where _id!=64514704b973cc48c724d563 instead of resolving this as $ne it instead is turning into $not which happens at this line

The reason this triggers is becasue we have a caster for this field:

import { ObjectId } from 'mongodb';
const aqpOptions = {
  casters: {
    mongoId: (val) => new ObjectId(val),
  },
  castParams: {
    _id: 'mongoId',
  },
};

What I would like to know is when is the $not operator intended to be added? In the documentation and examples I only see it next to regex values. Are there other cases where $not should be added as well, or would it be safe to also check value instanceof RegExp before using that condition?

• I have a "Users" collection with a user's password stored it "password" field.

• I have another collection "Articles" with the "_id" of the "User" collection as the foreign key to "user" field in "Articles" collection

• I can then populate the "user" field in "Articles" collection and get the password of the user by passing in "user.password"

This is a possible security flaw. It'd be great if you can provide a mechanism to overcome this.

Hey there, I would like to use regex with the $or operator.

aqp('filter={"$or":[{"name_en":/^value/},{"name_fr":/^value/}]}'); // string comes directly from the query string

Unfortunately this produces the following error:

Invalid JSON string: {"$or":[{"name_en":/^value/},{"name_fr":/^value/}]}

Regex is correctly parsed when used on a single field, but I need to use it on multiple using conditions. Is there a workaround for this?

Many thanks

Hello,

I am not able to import the library as per the documentation. When doing it like so:

import aqp from 'api-query-params'

I get the following error: TypeError: (0 , api_query_params_1.default) is not a function. I'm attempting to call the function as in the docs' example yet again:

const { filter, skip, limit, sort, projection, population } = aqp(query)

When doing an old-fashioned require statement though, (e.g. const aqp = require('api-query-params') ) behavior is as expected and I'm not getting errors when attempting to call the function.

I would like to use the type definitions if possible to avoid mixing all my other import statements with this require one.
By the way, doing this:

import { AqpQuery } from 'api-query-params' for that exported type works well. it is the function the one I just can't import with the help of TS.

Any guidance would be much appreciated. Thank you

Please add Types to the project

I am trying to send a E.164 formatted phone number in query string.

Sample HTTP request

GET /api/nonprofits/?members.phone=%2B919831042144 HTTP/1.1

After parsing with api-query-params, the + in the param is stripped and I get this as a log for filter object

{ 'members.phone': 919831042144 }

It seems to be auto casting it to a number and therefore removing the + from the value.

Should values starting with + be considered numbers or should the check for numbers be made stricter? I think I can specify casters per key to override this but just wanted to understand why it is done this way

I tried to populate in multiple levels but it seems it is not working.

Here example : https://medium.com/fredwong-it/mongoose-populate-multiple-levels-7e45be41d16b

I am passing the special characters but it replaced as special characters.
Exmaple:

const aqp = require('api-query-params');
const q = {mobile_no : "+919876543210"};
const result  = aqp(q);
console.log(result);

Current Output:
{ filter: { mobile_no: 919876543210 } }

Expected Output:

{ filter: { mobile_no: +919876543210 } }

import * as aqp from 'api-query-params';

const query = aqp('filter={"$or":[{"key1":"value1"},{"key2":"value2"}]}', {
    whitelist: ['key1']
});
console.log(JSON.stringify(query, null, 2));

Will print out:

{
  "filter": {
    "$or": [
      {
        "key1": "value1"
      },
      {
        "key2": "value2"
      }
    ]
  }
}

Expected result:

{
  "filter": {
    "$or": [
      {
        "key1": "value1"
      }
    ]
  }
}

Example:

aqp('cmp=string(1010)&cmp=string(1011)&cmp=string(1012)');

Result:

 { filter: { companyNumber: '1010),string(1011),string(1012' } }  

Hi, So I found a gap in the docs where if we attempt to use regex, it will basically just to match to a string

The final 2 rows in the table here show that if you try to search with regex, the mongoose object returned is going to be a string matcher and not a regex matcher.

Correct notation for the EQUALS would be {filter: {email : {$regex: /blahblah/i}}} or in the NOT case {filter: {email : { $not: {$regex: /blahblah/i}}}}

I have confirmed that this is what the code returns and when you attempt to search via regex, it doesnt work in the expected way. I copied the code locally and made these changes in my own way to serve this purpose. There was probably a time where this worked, perhaps with updated over time, this has now become defunct.

Hey

This is an awesome package. But the doc isn't clear. This will be better to have some real life axios or more example api request params usage

Not use 'fields' works - aqp('populate=a,b&fields=foo,bar,a.baz');

Thanks

I've forked this project to do some minor changes, which are specific for my project. I've recently updated a lot of project dependencies including api-query-params. With new version of npm I've run into problems with editing api-query-params code directly so I wanted some clean solution.

After forking api-query-params and adding fork url in package.json in my project the package installs fine, but the installed package looks a lot different - basically a copy of github repo. When i try to launch the whole app i just get Error: Cannot find module 'api-query-params'.

I figured out that the source needs to be built, but im totally lost what to do next to use my fork by npm in package.json to be installed as it should and to just work as before.

Can you please provide steps how to build package that can be used in package.json from github tarball?

[EDIT]
I had actually some time and make
the PR #99 with a POC.

Hello Loris!
Thanks for sharing this great lib.

It would be great to have a populate params to handle relation populations.
https://mongoosejs.com/docs/populate.html

Ex. Passing ?populate=users to the query, would get:

  const { filter, skip, limit, sort, projection, population } = aqp(req.query);
  User.find(filter)
    .skip(skip)
    .limit(limit)
    .sort(sort)
    .select(projection)
    .populate(population) // <---- here
    .exec((err, users) => {
      if (err) {
        return next(err);
      }
 
      res.send(users);
    });

... would turn that

{
  "_id": "5ce158dd5e60d70972ff97dc",
  "name": "admins",
  "users": [
    "5cded210825b89040cb81185",
    "5cdebeb003db96001fabd040",
  ]
}

in that:

{
  "_id": "5ce158dd5e60d70972ff97dc",
  "name": "admins",
  "users": [
    { 
      "_id": "5cded210825b89040cb81185",
      "firstname": "brad",
      "lastname": "pitt"
    },
    {
      "_id": "5cdebeb003db96001fabd040",
      "firstname": "matt",
      "lastname": "damon"
    }
  ]
}

Since this lib is already handling projection, it could handle population projection:
Ex. ?populate=users&field=name,users.lastname

TL;DR;

?populate=a,b,c

{ 
  population: [ 'a', 'b', 'c' ];
}

?populate=a,b,c?fields=a.f1,a.f2,b.f3.c.f4

{
  population: [
    { path: 'a', select: [ 'f1', 'f2' ]},
    { path: 'b', select: [ 'f3' ]},
    { path: 'c', select: [ 'f4' ]}
  ]
}

What do you think ?

Ok so the 1 in your URL (http://localhost:3000/api/tst/users/1) is not an URL query parameter but an URL path parameter, so you can use the library this way:

import express from 'express';
import aqp from 'api-query-params';
import User from './models/User';

const app = express();

app.get('/api/tst/users/:page', (req, res, next) => {
  const recordsPerPage = 10;
  const { filter, skip, limit, sort, projection, population } = aqp({ skip: req.params.page * recordsPerPage, ...req.query});
  ... // same as in the README.md example
});

In the example, I extract the page value from the req.params object, and build a virtual skip variable (which is page number * number of records per page, so if you're on a page 3, you will skip the first 30 records for instance), let me know if this is clear
@loris

Doing this does not work when I change the page number the same records still appear I'm using it like this http://localhost:3000/api/tst/users/1/?skip=1

since using it like this
http://localhost:3000/api/tst/users/1/

without the skip param gives me an empty array
when I go to the second page nothing changes
like this http://localhost:3000/api/tst/users/**2**/?skip=1
the same records still apear

Sorry sorry I've only just realized I have around 6 records and in your example you've used 10 records per page so THANKS it works as expected however how would I use this on the front end since I want to change the page number while still keeping the user's filters

like so how would I do this
http://localhost:3000/api/tst/users/3/?featured=true while changing the page number but still have the query params in place

Originally posted by @Chukstart in #101 (comment)

When running a query such as api/strore?owner=5996e60a6097712268bcfcb5,599999990e58541d229ef318

generates the query:
{ owner: { $in: [ "5996e60a6097712268bcfcb5", "599999990e58541d229ef318" ] } }

It should generate the query as follows:
{ owner: { $in: [ ObjectId("5996e60a6097712268bcfcb5"), ObjectId("599999990e58541d229ef318") ] } }

I am passing the following to the method:

var agpOptions = {
casters: {
mongoId: val => mongo.ObjectId(val)
},
castParams: {
_id: 'mongoId',
owner: 'mongoId'
}
};

var parsedQuery = aqp(queryString, agpOptions);

Firstly, thanks for a really useful library, does 99% of what I need to do.

I'm looking for a way to coerce an specific field in the filter result to a mongodb.ObjectId before I run the query against the db, is there currently a way to do this in the library?

So for example a way to do it manually at the moment would be:

'use strict';
var mongo = require('mongodb');
var aqp = require('api-query-params').default;

var queryString = '?ownderId:5829ba2b7a3a7a08307483a9';
var parsedQueryString = aqp(queryString);
parsedQueryString.ownderId = mongo.ObjectId(parsedQueryString.ownderId);
mongo.db.collection('items').findOne(parsedQueryString.filter, dataRetrieved);

In the above example it's fairly trivial, but it would be nice to have an extensible way of doing coercion before the query goes into the db. The way I see it there are a few options if this functionality doesn't already exist:

Custom functions

Allow setting up of custom "functions" in the query string before parsing, in a similar manner to:

aqp('key1=string(10)&key2=date(2016)&key3=string(null)');

For example

aqp.addFunction('mongoId', function(stringValue, keyName, fullQueryString){
    return mongo.ObjectId(stringValue);
});
aqp('?ownderId:mongoId(5829ba2b7a3a7a08307483a9)');

Key Name Lookup

Use the key name in the query string to do type casting

aqp.coerce(['ownderId'], function(stringValue, keyName, fullQueryString){
    return mongo.ObjectId(stringValue);
});
aqp('?ownderId:5829ba2b7a3a7a08307483a9');

Apologies if there is already a way to do this.

Hi Loris,
I'm using your lib for a long time and it's amazing: thanks!

I found some limits in sorting and filtering in populated content.

If I've understand well, it's impossible to sort by populated fields:
?populate=a,b,c&sort=a.f1,b.f2,c.f1

or to filter by populated fields:
?populate=a,b,c&a.f1=value&b.f2=value2

Is it right?
Do you have any suggestion about this need?

At the moment I tried to use "advanced filter" like
filter={"$or":[{"key1":"value1"},{"key2":"value2"}]}
but it's not so comfortable.

Thanks for any help ✌️

• Features list (es6, dependency-free, operators, well-tested, etc.)
• Full example usage with Express/Mongoose
• Detailed usage (operators, options, etc.)

Create ability to customize filter keys. May be useful for other orm.
For example:
count>5
{filter: {count: {gt: 5}}}
Key gt without $

Is there a built in way to handle querying for strings that include commas inside them? For example:

thing=a,b,c

In this case it will search the field thing for a value of a, b or c.

Instead, we need it to search the field thing for a value of a,b,c.

Just as a note, I've also attempted forcing a string for the whole value but that hasn't worked for me either:

thing=string(a,b,c)

Really nice lib!

The only thing I'm having trouble figure out is how to exclude "" values:

{
 id: 23423,
 key1 : "test",
 key2 : "" 
}

I tried &key2!="" with no effect.

Thanks!

Short description
When passing a different order of query parameters, the output will be different. I assume, query params are sequence order insensitive. If you disagree about my assumption, this issue is meaningless to you.

Example

// variant 1: NE before EQ
const query = aqp('type!=10a&type=20b');

//  {
//    filter: {
//      type: '10a'
//    },
//  }

// variant 2: EQ before NE
const query = aqp('type=10a&type!=20b');

//  error: 'Cannot create property '$ne' on string '10a'

Code debugging
When you change the order of "equals" = and "not equals to" !=, the query parser will fail due to internal parser logic:
'Cannot create property '$ne' on string '10a''. Both variants are kinda wrong. So far, my debugging resolves to the method getFilter.

Variant 1: The first iteration will lead to an operator === '$ne' and assign the value like result[key][op] = value. The second iteration will lead to operator === '$eq' and will reassign like result[key] = value`. Therefore the later iteration will override the first one. Is this intentional?

Variant 2: The first iteration will lead to an operator === '$eq' and it will assign the value like result[key] = value. The second iteration, will lead to operator === '$ne' and therefore reaches the default/else case result[key][op] = value. The last statement will create the previously announced error message, because result[key] is already a sting value from the first iteration.

Upcoming
A quick solution from our team was to use always the specific operator and create always the operator specific filter:

...
} else if (op === '$eq') {
      result[key][op] = value;
} ...

Please give feedback about this topic and provide a roadmap/fix to satisfy this requirement.

Can i get populate query parameter like this { path: 'array.objectProperty' } but when i send populate param ?populate=array.objectProperty. The populate that I received was multi-levels populate object.

I'm using express, as shown in your example in the README.

For this use case /users?age>18 for example, express fails to parse the query correctly, and req.query is an empty object. The same problem occurs whenever the operator is not =.

This problem is not directly related to this package, but I don't see the point of including as an example in the README if it does not work for many operators. And if it doesn't work with Express, the use cases are very limited.

As an alternative, I tried grabbing req.originalUrl, but the age<18 part is already filtered out.

And I know that I can use the filter key directly, but it is less convenient.

Is there something I'm missing?

Can be tested with the following Jest test

const aqp = require('api-query-params')
describe('AQP test', function () {
  it('should test our aqp guid parsed as date error', done => {
    const notWorking = 'uuid=c0974f20-1071-11e7-81c3-a593aec7037f'
    const working = 'uuid=1e78b83e-d1d7-4b62-9472-d0ebf8531091'

    let aqpResult = aqp(working)
    expect(aqpResult.filter.uuid).toBe('1e78b83e-d1d7-4b62-9472-d0ebf8531091')

    aqpResult = aqp(notWorking)
    expect(aqpResult.filter.uuid).toBe('c0974f20-1071-11e7-81c3-a593aec7037f')
    done()
  })
})

RESULT:

Expected: "c0974f20-1071-11e7-81c3-a593aec7037f"
    Received: Date { NaN }

    Difference:

      Comparing two different types of values. Expected string but received date.

Does the library support this projection?
https://docs.mongodb.com/manual/tutorial/project-fields-from-query-results/#project-specific-array-elements-in-the-returned-array

Such as slicing the last element of an array field:

db.inventory.find( { status: "A" }, { item: 1, status: 1, instock: { $slice: -1 } } )

If not, it would be amazing if it could :)

Thank you!

If I do this will it work

 const desiredBook= await Book
    .find(filter)
    .skip(skip)
    .limit(recordsPerPage)
    .sort(sort)
    .select(projection)
    .populate(population)
      .or(["audiobook.book.bookname":searchQuery,
      "pdf.author.authorname":searchQuery,
      "book.booktitle.bookTitleName":searchQuery ,        
     "location.location.location":searchQuery 
      ])
    .sort(sort);

or is there a way I could do this

const { filter, skip, limit, sort, projection, population } =
    aqp(`filter={ "$or": [
      {"audiobook.book.bookname": "${searchQuery}" },
      {  "pdf.author.authorname": "${searchQuery}" },
      { "book.booktitle.bookTitleName": "${searchQuery}" },
      { "location.location.location": "${searchQuery}" },
 ]}`
 , { skip: ((req.params.page * recordsPerPage) - recordsPerPage), ...req.query });

Hi, I don't like the unpredictability that automatic casting adds to the application. I may have dozens of string fields and in some of them the user might decide to use only numbers. I don't want to have to declare all my string fields explicitly (castParams)

Might you include an option to disable automatic casters?

I believe the following sort and limit variables are undefined and it should be read from the query.

app.get('/users', (req, res, next) => {
  const query = aqp(req.query);
  User
    .find(query.filter)
    .skip(skip)
    .limit(limit)
    .sort(sort)
    .exec((err, users) => {
      if (err) {
        return next(err);
      }

      res.send(users);
    });
});```

const aqp = require('api-query-params')

Just calling aqp from any method I got this error:

TypeError: apq is not a function

Tested on node 8.9.4 and 10.10.0. And know the I don't give any details, but I don't get more information on this.

I am trying to filter by a string that contains one of more / characters.

Query: {key: 'This key/has special chars!'}

Where the console.logs are:

console.log(ctx.query)
const { filter, skip, limit, sort, projection } = aqp(ctx.query)
console.log(filter)

What that console.log looks like:

{ key: 'This key/has special chars!' }
{ key: { '$in': [ 'This key', 'has special chars!' ] } }

Expected result: the string for key: is preserved as a single string for the mongoose query.

Hi and sorry for my poor english.
I've a problem with ')' char when i try to force cast with string() or use casters with a custom function.
To reproduce the issue:

  const val = aqp("key1=string(abc))");
  console.log(val);

  const val2 = aqp("key1=stringPers(abc))", {
    casters: {
      stringPers: (val: string) => val,
    },
  });
  console.log(val2);

with '(' char works very well.

A workaround seems to use castParams that works with ')' but fails with ',' (the key is converted in array with $in operator)

To reproduce:

const val3 = aqp("key1=a,b,c,)&key2=abc)", {
    castParams: {
      key1: "string",
      key2: "string",
    },
  });
  console.log(val3);

So i haven't still find a way to cast a key with ')' and ',' to simple string

Thanks for help and compliments for your package, is very useful.

Filippo