Parse query string to Postgres SQL operators. This library does not generate the SQL statements, but it can be paired with other ORMs to generate dynamic SQL.
Requires go 1.18+
.
for strongly-type approach, see rough implementation here.
- customizable field names, field ops, query string fields, as well as struct tags
- supports many operators used by Postgres
- handles conjunction (and/or)
- handles type conversion from query string to designated struct field's type
- handles limit/offset
- handles sorting
- handles parsing slices for array operators like
IN
,NOT IN
,LIKE
,ILIKE
go get github.com/alextanhongpin/goql
package main
import (
"fmt"
"net/url"
"github.com/alextanhongpin/goql"
)
type Book struct {
Author string
Title string
PublishYear int `q:"publish_year" sort:"true"`
}
func main() {
// Register a new decoder for the type Book.
// Information are extracted from the individual struct
// fields, as well as the struct tag.
dec := goql.NewDecoder[Book]()
/*
Query: Find books that are published by 'Robert
Greene' which has the keyword 'law' and 'master'
published after 2010. The results should be ordered
descending nulls last, limited to 10.
SELECT *
FROM books
WHERE author = 'Robert Greene'
AND publish_year > 2010
AND title ilike any(array['law', 'master'])
ORDER BY publish_year desc nullslast
LIMIT 10
*/
v := make(url.Values)
v.Set("author.eq", "Robert Greene")
v.Set("publish_year.gt", "2010")
v.Add("title.ilike", "law")
v.Add("title.ilike", "master")
v.Add("sort_by", "publish_year.desc.nullslast")
v.Add("limit", "10")
f, err := dec.Decode(v)
if err != nil {
panic(err)
}
for _, and := range f.And {
fmt.Println("and:", and)
}
fmt.Println("limit:", *f.Limit)
for _, sort := range f.Sort {
fmt.Println("sort:", sort)
}
}
Output:
and: author eq "Robert Greene"
and: publish_year gt 2010
and: title ilike []interface {}{"law", "master"}
limit: 10
sort: publish_year desc nullslast
Basic datatypes (int, float, string, bool, time):
op | querystring | sql |
---|---|---|
eq | name.eq=john appleseed |
name = 'john appleseed' |
neq | name.neq=john appleseed |
name <> 'john appleseed' |
lt | age.lt=17 |
age < 17 |
lte | age.lte=17 |
age <= 17 |
gt | age.gt=17 |
age > 17 |
gte | age.gte=17 |
age >= 17 |
like | title.like=programming% |
title like 'programming%' |
ilike | title.ilike=programming% |
title ilike 'programming%' |
notlike | title.notlike=programming% |
title not like 'programming%' |
notilike | title.notilike=programming% |
title not ilike 'programming%' |
in | hobbies.in=programming&hobbies.in=music |
hobbies in ('programming', 'music') |
notin | hobbies.notin=programming&hobbies.notin=music |
hobbies not in ('programming', 'music') |
is | married.is=true |
married is true |
isnot | married_at.isnot=null |
married_at is not null |
Some operators such as IN
, LIKE
, ILIKE
and their negation NOT
supports multiple values:
op | querystring | sql |
---|---|---|
like | title.like=programming%&title.like=music% |
title like any(array['programming%', 'music%']) |
ilike | title.ilike=programming%&title.ilike=music% |
title ilike any(array['programming%', 'music%']) |
notlike | title.notlike=programming%&title.notlike=music% |
title not like all(array['programming%', 'music%']) |
notilike | title.notilike=programming%&title.notilike=music% |
title not ilike all(array['programming%', 'music%']) |
in | hobbies.in=programming&hobbies.in=music |
hobbies in ('programming', 'music') |
notin | hobbies.notin=programming&hobbies.notin=music |
hobbies not in ('programming', 'music') |
If the target type is an array
1, then multiple values are accepted too:
op | querystring | sql |
---|---|---|
eq | hobbies.eq=swimming&hobbies.eq=dancing |
hobbies = array['swimming', 'dancing'] |
neq | hobbies.neq=swimming&hobbies.neq=dancing |
hobbies <> array['swimming', 'dancing'] |
lt | scores.lt=50&scores.lt=100 |
scores < array[10, 100] |
lte | scores.lte=50&scores.lte=100 |
scores <= array[10, 100] |
gt | scores.gt=50&scores.gt=100 |
scores >= array[10, 100] |
gte | scores.gte=50&scores.gte=100 |
scores >= array[10, 100] |
AND/OR
can be chained and nested.
op | querystring | sql |
---|---|---|
and | and=age.gt:13&and=age.lt:30&or=and.(name.ilike:alice%,name.notilike:bob%) |
AND age > 13 AND age < 30 OR (name ilike 'alice%' AND name not ilike 'bob%')) |
or | or=and.(height.isnot:null,height.gte:170) |
OR (height is not null AND height >= 170) |
or | or=height.isnot:null&or=height.gte:170 |
OR height is not null OR height >= 170 |
The default naming for the limit/offset in querystring is limit
and offset
. The query string name can changed by calling:
dec.SetQueryLimitName("_limit")
dec.SetQueryOffsetName("_offset")
The default min/max for the limit is 1
and 20
respectively. To change the limit:
dec.SetLimitRange(5, 100)
op | querystring | sql |
---|---|---|
limit | limit=20 |
LIMIT 20 |
offset | offset=42 |
OFFSET 42 |
limit/offset | limit=20&offset=42 |
LIMIT 20, OFFSET 42 |
To enable sorting for a field, set the struct tag sort:"true"
. E.g.
type User struct {
Age int `sort:"true"`
}
The sort
struct name can be changed:
dec.SetSortTag("sortable")
And the example above will then be:
type User struct {
Age int `sortable:"true"`
}
op | querystring | sql |
---|---|---|
sort | sort=age |
ORDER BY AGE ASC NULLSLAST |
sort=age.asc |
ORDER BY age ASC NULLSLAST |
|
sort=age.desc |
ORDER BY age DESC NULLSFIRST |
|
sort=age.asc.nullsfirst |
ORDER BY age ASC NULLSFIRST |
|
sort=age.desc.nullslast |
ORDER BY age DESC NULLSLAST |
|
sort=id.desc&sort=age |
ORDER BY id DESC NULLSFIRST, age ASC NULLSLAST |
type User struct {
Age int `q:"age"`
}
The default tag name is q
. To change it:
dec.SetFilterTag("filter")
And the example above will be:
type User struct {
Age int `filter:"age"`
}
field | tag | description |
---|---|---|
name string q:"name" |
name | modifies the query string name |
MarriedAt time.Time q:"," |
name | if no name is specified, it defaults to lower common initialism of the name |
ID uuid.UUID q:",type:uuid" |
type: | specifies the type that would be used by the parser |
IDs []string q:",type:[]uuid" |
type:[] | specifies an array type. array types have special operators |
ID *string q:",type:*uuid" |
type:* | specifies a null type. null types have special operators |
ID string q:",null" |
null | another approach of specifying null types |
ID string q:",ops:eq,neq" |
ops | specifies the list of supported ops. In this example, only id.eq=v and id.neq=v is valid. This can be further overwritten by dec.SetOps . |
Examples of valid tags:
q:",null"
q:"custom_name,null"
q:"custom_name,type:uuid"
q:"custom_name,ops:eq,neq"
q:"custom_name,type:uuid,ops:eq,neq"
q:"custom_name,type:[]uuid"
q:"custom_name,type:[]*uuid"
q:"custom_name,type:[]*uuid,ops:eq,neq,in,notin"
To customize ops
for a specific field, either set the struct tag ops:<comma-separate-list-of-ops>
, or set it through the method SetOps
:
// To allow only `eq` and `neq` for the field `name`:
dec.SetOps("name", goql.OpEq | goql.OpNeq)
Query string parameters are string (or list of string). Parsers are responsible for parsing the string to the desired types that are either
- inferred through the struct field's type through reflection
- set at the struct tag through
type:"yourtype"
Custom parsers can be registered as follow:
type Book struct {
ID uuid.UUID `q:"id,type:uuid"` // Register a new type `uuid`.
}
id := uuid.New()
v := make(url.Values)
v.Set("id.eq", id.String())
v.Add("id.in", id.String()) // Automatically handles conversion for a list of values.
v.Add("id.in", id.String())
dec := goql.NewDecoder[Book]()
dec.SetParser("uuid", parseUUID)
f, err := dec.Decode(v)
if err != nil {
t.Fatal(err)
}
where parseUUID
fulfills the goql.ParseFn
method definition:
func parseUUID(in string) (any, error) {
return uuid.Parse(in)
}
What if I need to filter some fields from
url.Values
?
Filter it manually before passing to Decode(v url.Values)
.
What if I need to add validation to the values?
Create a new type (aka `value object), and add a parser for that type which includes validation. Parsers are type-specific.
// You can edit this code!
// Click here and start typing.
package main
import (
"errors"
"fmt"
"net/url"
"strings"
"github.com/alextanhongpin/goql"
)
var ErrInvalidEmail = errors.New("bad email format")
type Email string
func (e Email) Validate() error {
if !strings.Contains(string(e), "@") {
return ErrInvalidEmail
}
return nil
}
type User struct {
Email Email `q:",type:email"` // Register a new type "email"
}
func main() {
dec := goql.NewDecoder[User]()
dec.SetParser("email", parseEmail)
v := make(url.Values)
v.Set("email.eq", "bad email") // Register a parser for type "email"
_, err := dec.Decode(v)
fmt.Println(err)
fmt.Println(errors.Is(err, ErrInvalidEmail))
}
func parseEmail(in string) (any, error) {
email := Email(in)
return email, email.Validate()
}
Why is there no support for keyset/cursor pagination, only limit/offset?
Because you can construct the cursor pagination directly.