Giter VIP home page Giter VIP logo

postgrex's Introduction

Postgrex

Build Status

PostgreSQL driver for Elixir.

Documentation: http://hexdocs.pm/postgrex/

Examples

iex> {:ok, pid} = Postgrex.start_link(hostname: "localhost", username: "postgres", password: "postgres", database: "postgres")
{:ok, #PID<0.69.0>}

iex> Postgrex.query!(pid, "SELECT user_id, text FROM comments", [])
%Postgrex.Result{command: :select, empty?: false, columns: ["user_id", "text"], rows: [[3,"hey"],[4,"there"]], size: 2}}

iex> Postgrex.query!(pid, "INSERT INTO comments (user_id, text) VALUES (10, 'heya')", [])
%Postgrex.Result{command: :insert, columns: nil, rows: nil, num_rows: 1}}

Features

  • Automatic decoding and encoding of Elixir values to and from PostgreSQL's binary format
  • User defined extensions for encoding and decoding any PostgreSQL type
  • Supports transactions, prepared queries and multiple pools via DBConnection
  • Supports PostgreSQL 8.4, 9.0-9.6, and later (hstore is not supported on 8.4)

Data representation

PostgreSQL      Elixir
----------      ------
NULL            nil
bool            true | false
char            "é"
int             42
float           42.0
text            "eric"
bytea           <<42>>
numeric         #Decimal<42.0> *
date            %Date{year: 2013, month: 10, day: 12}
time(tz)        %Time{hour: 0, minute: 37, second: 14} **
timestamp       %NaiveDateTime{year: 2013, month: 10, day: 12, hour: 0, minute: 37, second: 14}
timestamptz     %DateTime{year: 2013, month: 10, day: 12, hour: 0, minute: 37, second: 14, time_zone: "Etc/UTC"} **
interval        %Postgrex.Interval{months: 14, days: 40, secs: 10920, microsecs: 315}
array           [1, 2, 3]
composite type  {42, "title", "content"}
range           %Postgrex.Range{lower: 1, upper: 5}
multirange      %Postgrex.Multirange{ranges: [%Postgrex.Range{lower: 1, upper: 5}, %Postgrex.Range{lower: 20, upper: 23}]}
uuid            <<160,238,188,153,156,11,78,248,187,109,107,185,189,56,10,17>>
hstore          %{"foo" => "bar"}
oid types       42
enum            "ok" ***
bit             << 1::1, 0::1 >>
varbit          << 1::1, 0::1 >>
tsvector        [%Postgrex.Lexeme{positions: [{1, :A}], word: "a"}]

* Decimal

** Timezones will always be normalized to UTC or assumed to be UTC when no information is available, either by PostgreSQL or Postgrex

*** Enumerated types (enum) are custom named database types with strings as values.

**** Anonymous composite types are decoded (read) as tuples but they cannot be encoded (written) to the database

Postgrex does not automatically cast between types. For example, you can't pass a string where a date is expected. To add type casting, support new types, or change how any of the types above are encoded/decoded, you can use extensions.

JSON support

Postgrex comes with JSON support out of the box via the Jason library. To use it, add :jason to your dependencies:

{:jason, "~> 1.0"}

You can customize it to use another library via the :json_library configuration:

config :postgrex, :json_library, SomeOtherLib

Once you change the value, you have to recompile Postgrex, which can be done by cleaning its current build:

mix deps.clean postgrex --build

Extensions

Extensions are used to extend Postgrex' built-in type encoding/decoding.

The extensions directory in this project provides implementation for many Postgres' built-in data types. It is also a great example of how to implement your own extensions. For example, you can look at the Date extension as a starting point.

Once you defined your extensions, you should build custom type modules, passing all of your extensions as arguments:

Postgrex.Types.define(MyApp.PostgrexTypes, [MyApp.Postgis.Extensions], [])

Postgrex.Types.define/3 must be called on its own file, outside of any module and function, as it only needs to be defined once during compilation.

Once a type module is defined, you must specify it on start_link:

Postgrex.start_link(types: MyApp.PostgrexTypes)

OID type encoding

PostgreSQL's wire protocol supports encoding types either as text or as binary. Unlike most client libraries Postgrex uses the binary protocol, not the text protocol. This allows for efficient encoding of types (e.g. 4-byte integers are encoded as 4 bytes, not as a string of digits) and automatic support for arrays and composite types.

Unfortunately the PostgreSQL binary protocol transports OID types as integers while the text protocol transports them as string of their name, if one exists, and otherwise as integer.

This means you either need to supply oid types as integers or perform an explicit cast (which would be automatic when using the text protocol) in the query.

# Fails since $1 is regclass not text.
query("select nextval($1)", ["some_sequence"])

# Perform an explicit cast, this would happen automatically when using a
# client library that uses the text protocol.
query("select nextval($1::text::regclass)", ["some_sequence"])

# Determine the oid once and store it for later usage. This is the most
# efficient way, since PostgreSQL only has to perform the lookup once. Client
# libraries using the text protocol do not support this.
%{rows: [{sequence_oid}]} = query("select $1::text::regclass", ["some_sequence"])
query("select nextval($1)", [sequence_oid])

PgBouncer

When using PgBouncer with transaction or statement pooling named prepared queries can not be used because the bouncer may route requests from the same postgrex connection to different PostgreSQL backend processes and discards named queries after the transactions closes. To force unnamed prepared queries:

Postgrex.start_link(prepare: :unnamed)

Contributing

To contribute you need to compile Postgrex from source and test it:

$ git clone https://github.com/elixir-ecto/postgrex.git
$ cd postgrex
$ mix test

The tests requires some modifications to your hba file. The path to it can be found by running $ psql -U postgres -c "SHOW hba_file" in your shell. Put the following above all other configurations (so that they override):

local   all             all                     trust
host    all             postgrex_md5_pw         127.0.0.1/32    md5
host    all             postgrex_cleartext_pw   127.0.0.1/32    password
host    all             postgrex_scram_pw       127.0.0.1/32    scram-sha-256

The server needs to be restarted for the changes to take effect. Additionally you need to setup a PostgreSQL user with the same username as the local user and give it trust or ident in your hba file. Or you can export $PGUSER and $PGPASSWORD before running tests.

Testing hstore on 9.0

PostgreSQL versions 9.0 does not have the CREATE EXTENSION commands. This means we have to locate the postgres installation and run the hstore.sql in contrib to install hstore. Below is an example command to test 9.0 on OS X with homebrew installed postgres:

$ PGVERSION=9.0 PGPATH=/usr/local/share/postgresql9/ mix test

License

Copyright 2013 Eric Meadows-Jönsson

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

postgrex's People

Contributors

asonge avatar chaodhib avatar chulkilee avatar dantswain avatar enilsen16 avatar ericmj avatar fishcakez avatar gjaldon avatar greg-rychlewski avatar hauleth avatar jonatanklosko avatar josevalim avatar keichan34 avatar kianmeng avatar kipcole9 avatar lexmag avatar lukaszsamson avatar maartenvanvliet avatar michalmuskala avatar msch avatar pma avatar richardkmichael avatar romul avatar scrogson avatar troyk avatar tt avatar tyre avatar v0idpwn avatar whatyouhide avatar wojtekmach avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

postgrex's Issues

Cannot connect to Amazon Redshift

I don't know which versions of PostgreSQL the postgrex library is intended to support but I cannot get it to work with Amazon Redshift which is a modified version of PostgreSQL 8.0.2

http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.html

I get a number of different errors trying to connect to and query Redshift. See my console session below. I tried patching a few of those errors, such as patching Connection.query to deal with a response of 🆗

def query(pid, statement, params, opts \ []) do
message = {:query, statement, params}
timeout = opts[:timeout] || @timeout
case GenServer.call(pid, message, timeout) do
:ok ->
{:ok, []}
%Postgrex.Result{} = res ->
{:ok, res}
%Postgrex.Error{} = err ->
{:error, err}
{:error, kind, reason, stack} ->
:erlang.raise(kind, reason, stack)
end
end

However that just triggered other problems down the line related to types and I don't know enough about Postgrex to fix them and I don't even know if Redshift should be supported. Should it? Is there any other way to connect to Redshift from Erlang/Elixir?

iex(1)> {:ok, pid} = Postgrex.Connection.start_link(hostname: "innovation-dw.cjer9kp9xlsu.eu-west-1.redshift.amazonaws.com", username: "", password: "", database: "", port: 5439)
{:ok, #PID<0.449.0>}
iex(2)> [warn] Unhandled Postgres error: ERROR (syntax_error): syntax error at or near "]"

nil

iex(3)> Postgrex.Connection.query!(pid, "select * from foo", [])
** (EXIT from #PID<0.447.0>) an exception was raised:
** (MatchError) no match of right hand side value: "bla" lib/postgrex/types.ex:84: anonymous fn/1 in Postgrex.Types.build_types/1
(elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
(elixir) lib/enum.ex:1387: Enum."-reduce/3-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:1043: Enum.map/2
(postgrex) lib/postgrex/protocol.ex:159: Postgrex.Protocol.message/3
(postgrex) lib/postgrex/connection.ex:417: Postgrex.Connection.new_data/2
(postgrex) lib/postgrex/connection.ex:292: Postgrex.Connection.handle_info/2
(stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:681: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Interactive Elixir (1.1.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> [error] GenServer #PID<0.449.0> terminating
** (MatchError) no match of right hand side value: "bla" lib/postgrex/types.ex:84: anonymous fn/1 in Postgrex.Types.build_types/1
(elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
(elixir) lib/enum.ex:1387: Enum."-reduce/3-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:1043: Enum.map/2
(postgrex) lib/postgrex/protocol.ex:159: Postgrex.Protocol.message/3
(postgrex) lib/postgrex/connection.ex:417: Postgrex.Connection.new_data/2
(postgrex) lib/postgrex/connection.ex:292: Postgrex.Connection.handle_info/2
(stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:681: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:tcp, #Port<0.13922>, <<50, 0, 0, 0, 4, 68, 0, 0, 0, 13, 0, 1, 0, 0, 0, 3, 98, 108, 97, 67, 0, 0, 0, 11, 83, 69, 76, 69, 67, 84, 0, 90, 0, 0, 0, 5, 73>>}
State: %{backend_key: {1354, 405653114}, bootstrap: {#Reference<0.0.2.1879>, 1384548, [Postgrex.Extensions.Binary, Postgrex.Extensions.Text], #HashDict<[{Postgrex.Extensions.Text, {8, 0, 2}}, {Postgrex.Extensions.Binary, {8, 0, 2}}]>}, extensions: [{Postgrex.Extensions.Binary, nil}, {Postgrex.Extensions.Text, nil}], listener_channels: #HashDict<[]>, listeners: #HashDict<[]>, opts: [password: :REDACTED, hostname: "innovation-dw.cjer9kp9xlsu.eu-west-1.redshift.amazonaws.com", username: "inno_dw", database: "dev", port: 5439], parameters: %{"TimeZone" => "UTC", "client_encoding" => "UNICODE", "datestyle" => "ISO, MDY", "gconf_case_sensitive" => "on", "integer_datetimes" => "on", "is_superuser" => "on", "max_numeric_precision" => "38", "max_varchar_size" => "65535", "padb_revision" => "V1-Patch61-44-g04ecee3f46", "padb_version" => "", "server_encoding" => "UNICODE", "server_version" => "8.0.2", "session_authorization" => "inno_dw", "timezone_abbreviations" => "Default"}, portal: [], queue: {[%{command: {:query, "select * from foo", []}, from: {#PID<0.447.0>, #Reference<0.0.1.3732>}, reply: :no_reply}], []}, rows: [], sock: {:gen_tcp, #Port<0.13922>}, state: :binding, statement: nil, tail: "", types: :types_removed, types_key: {'innovation-dw.cjer9kp9xlsu.eu-west-1.redshift.amazonaws.com', 5439, "dev", []}}

iex(10)> Postgrex.Connection.query!(pid, "select true", [])
** (EXIT from #PID<0.454.0>) an exception was raised:
** (MatchError) no match of right hand side value: "t" lib/postgrex/types.ex:84: anonymous fn/1 in Postgrex.Types.build_types/1
(elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
(elixir) lib/enum.ex:1387: Enum."-reduce/3-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:1043: Enum.map/2
(postgrex) lib/postgrex/protocol.ex:159: Postgrex.Protocol.message/3
(postgrex) lib/postgrex/connection.ex:417: Postgrex.Connection.new_data/2
(postgrex) lib/postgrex/connection.ex:292: Postgrex.Connection.handle_info/2
(stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:681: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Interactive Elixir (1.1.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> [error] GenServer #PID<0.461.0> terminating
** (MatchError) no match of right hand side value: "t" lib/postgrex/types.ex:84: anonymous fn/1 in Postgrex.Types.build_types/1
(elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
(elixir) lib/enum.ex:1387: Enum."-reduce/3-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:1043: Enum.map/2
(postgrex) lib/postgrex/protocol.ex:159: Postgrex.Protocol.message/3
(postgrex) lib/postgrex/connection.ex:417: Postgrex.Connection.new_data/2
(postgrex) lib/postgrex/connection.ex:292: Postgrex.Connection.handle_info/2
(stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:681: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:tcp, #Port<0.14244>, <<50, 0, 0, 0, 4, 68, 0, 0, 0, 11, 0, 1, 0, 0, 0, 1, 116, 67, 0, 0, 0, 11, 83, 69, 76, 69, 67, 84, 0, 90, 0, 0, 0, 5, 73>>}
State: %{backend_key: {1439, 1808437079}, bootstrap: {#Reference<0.0.1.3793>, 1388644, [Postgrex.Extensions.Binary, Postgrex.Extensions.Text], #HashDict<[{Postgrex.Extensions.Text, {8, 0, 2}}, {Postgrex.Extensions.Binary, {8, 0, 2}}]>}, extensions: [{Postgrex.Extensions.Binary, nil}, {Postgrex.Extensions.Text, nil}], listener_channels: #HashDict<[]>, listeners: #HashDict<[]>, opts: [password: :REDACTED, hostname: "innovation-dw.cjer9kp9xlsu.eu-west-1.redshift.amazonaws.com", username: "inno_dw", database: "dev", port: 5439], parameters: %{"TimeZone" => "UTC", "client_encoding" => "UNICODE", "datestyle" => "ISO, MDY", "gconf_case_sensitive" => "on", "integer_datetimes" => "on", "is_superuser" => "on", "max_numeric_precision" => "38", "max_varchar_size" => "65535", "padb_revision" => "V1-Patch61-44-g04ecee3f46", "padb_version" => "", "server_encoding" => "UNICODE", "server_version" => "8.0.2", "session_authorization" => "inno_dw", "timezone_abbreviations" => "Default"}, portal: [], queue: {[%{command: {:query, "select true", []}, from: {#PID<0.454.0>, #Reference<0.0.1.3810>}, reply: :no_reply}], []}, rows: [], sock: {:gen_tcp, #Port<0.14244>}, state: :binding, statement: nil, tail: "", types: :types_removed, types_key: {'innovation-dw.cjer9kp9xlsu.eu-west-1.redshift.amazonaws.com', 5439, "dev", []}}

nil

Run mix do deps.get error

I run mix do deps.get with this error:

** (UndefinedFunctionError) undefined function: Access.get/2
    (elixir) Access.get([hex_app: :decimal, build: "/Users/wei/hobbi_source/story_dynamo/deps/postgrex/_build/shared/lib/decimal", dest: "/Users/wei/hobbi_source/story_dynamo/deps/postgrex/deps/decimal"], :dest)
    lib/hex/scm.ex:30: Hex.SCM.checked_out?/1
    (mix) lib/mix/dep/loader.ex:154: Mix.Dep.Loader.scm_status/2
    (mix) lib/mix/dep/loader.ex:139: Mix.Dep.Loader.with_scm_and_app/1
    (mix) lib/mix/dep/loader.ex:98: Mix.Dep.Loader.to_dep/3
    (elixir) lib/enum.ex:967: Enum."-map/2-lc$^0/1-0-"/2
    (elixir) lib/enum.ex:967: Enum."-map/2-lc$^0/1-0-"/2
    (mix) lib/mix/dep/loader.ex:21: Mix.Dep.Loader.children/1

ErlangError while trying README example

if I say

iex> {:ok, pid} = Postgrex.Connection.start_link(hostname: "localhost", username: "foo", password: "bar", database: "api_development")

I get

erlang error: {:noproc, {GenServer, :call, [Postgrex.TypeServer, {:fetch, {'localhost', 5432, "api_development", []}}, 60000]}}
(elixir) lib/gen_server.ex:356: GenServer.call/3
        lib/postgrex/protocol.ex:34: Postgrex.Protocol.bootstrap/1
        lib/postgrex/connection.ex:417: Postgrex.Connection.new_data/2
        lib/postgrex/connection.ex:292: Postgrex.Connection.handle_info/2
        (stdlib) gen_server.erl:593: :gen_server.try_dispatch/4
        (stdlib) gen_server.erl:659: :gen_server.handle_msg/5
        (stdlib) proc_lib.erl:237: :proc_lib.init_p_do_apply/3

What am I doing wrong?

Thanks in advance

Cannot encode date 1999-12-31 properly

With 0.7.0, {1999,12,31} is encoded as -1 which is later treated as null.

It seems to be fixed in master, but from the looks of commit logs it might have been fixed by coincidence.

I opened this issue as just a simple heads up in case this wasn't known.

could not compile dependency decimal

When trying to run tests with mix test, I get the following error:

== Compilation error on file lib/decimal.ex ==
** (CompileError) lib/decimal.ex:66: undefined function ::/2
    (stdlib) lists.erl:1352: :lists.mapfoldl/3
    (stdlib) lists.erl:1353: :lists.mapfoldl/3

==> decimal
could not compile dependency decimal, mix compile failed. You can recompile this dependency with `mix deps.compile decimal` or update it with `mix deps.update decimal`

mix deps.compile decimal and mix deps.update decimal can not fix it.

I'm using CentOS with Elixir(0.14.3-dev):

Erlang/OTP 17 [erts-6.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Interactive Elixir (0.14.3-dev) - press Ctrl+C to exit (type h() ENTER for help) 
iex(1)> 

How to execute this query?

I am trying to do a migration with a semi-complex query and I am getting a prosgrex error due to the dollar sign on the $do$ line

def change do
    execute(~S"""
      DO
      $do$
      DECLARE
          v vistas%rowtype;
      BEGIN
        FOR v IN 
            SELECT * FROM vistas ORDER BY id
        LOOP

           INSERT INTO experiencias (nombre, inserted_at, updated_at)
           VALUES (v.nombre, now(), now());

           UPDATE vistas
           SET experiencia_id = currval(pg_get_serial_sequence('experiencias', 'id'))
           WHERE id = v.id;
        END LOOP;
      END
    """)
  end

No extension found for OID xxxxxx when using citext

I've struggled with this for a long time and it seems like postgrex is very happy if I manually run create extension citext in psql; but, when it's part of my migrations I get this error:

Possibly related #60?

** (ArgumentError) no extension found for oid `62022`
    (postgrex) lib/postgrex/types.ex:280: Postgrex.Types.fetch!/2
    (postgrex) lib/postgrex/types.ex:213: Postgrex.Types.format/2
    (elixir) lib/enum.ex:977: anonymous fn/3 in Enum.map/2
    (elixir) lib/enum.ex:1261: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir) lib/enum.ex:977: Enum.map/2
    (postgrex) lib/postgrex/protocol.ex:134: Postgrex.Protocol.message/3
    (postgrex) lib/postgrex/connection.ex:417: Postgrex.Connection.new_data/2
    (postgrex) lib/postgrex/connection.ex:292: Postgrex.Connection.handle_info/2

First Migration to run:

defmodule Charlie.Repo.Migrations.EnableCitext do
  use Ecto.Migration

  # def change do
  #   # TODO - use citext for email
  #   execute("CREATE EXTENSION IF NOT EXISTS citext WITH SCHEMA public;")
  # end
  def up do
    execute "CREATE EXTENSION IF NOT EXISTS citext"
  end

  def down do
    execute "DROP EXTENSION IF EXISTS citext"
  end
end

Used in:

defmodule App.Repo.Migrations.CreateUser do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :email, :citext
      add :hashed_password, :string

      timestamps
    end
    create index(:users, [:email], unique: true)
  end
end
Versions
Name        : postgresql
Version     : 9.4.4

Elixir 1.0.3
  "phoenix": {:hex, :phoenix, "1.0.2"},
  "ecto": {:hex, :ecto, "1.0.4"},
  "postgrex": {:hex, :postgrex, "0.9.1"},

unable to encode value `nil` as type text

Hi,

I'm using ecto in an elixir project. Recently, I upgraded the postgrex version and now I've got a regression.

Basically I've got a model with a property as an array of text

 CREATE TABLE correlations(
      id bigserial primary key,
      correlations text[]
    )

the values in the array can be null. Is used to work properly but now postgrex returns an error

unable to encode value nil as type text

Cheers,

David

LISTEN / NOTIFY support?

Any work in progress for this? If not, I may try my hand at writing something and submitting a PR. If so, I'd be happy to try to help out (if help is needed).

Support for Postgres Range types?

Are you interested in a PR for Postgres' built-in Range types? I saw the discussions around HStore and custom encoders/decoders, but this is potentially a different case since the range type isn't an extension.

Driver does not seem to support binding a list to the parameters

so something like this fails:

iex(4)> Postgrex.Connection.query(conn, """
...(4)> select *
...(4)> from audit
...(4)> where user_id in ($1)
...(4)> """, [[22,94]])
{:error,
%Postgrex.Error{message: "unable to encode value [22, 94] as type int8",
postgres: nil}}

I also tried this:

iex(11)> Postgrex.Connection.query(conn, "select * from audit where user_id in ($1)", [22,94], param_types: ["array"])

this returns data only for user with id of 22

when I do this
iex(12)> Postgrex.Connection.query(conn, "select * from audit where user_id in ($1)", [[22,94]], param_types: ["array"])

I get the same error as in case 1

No extension found for oid on column with function in SELECT

Hi,

I’ve been trying to implement a full text search using postgresql. But when I try tu run a simple query with the to_tsvector function, the query raises on error. I call the postgrex lib from ecto. Here is the simplest example to reproduce the behavior.

query = """
select to_tsvector('simple', translations.key) from translations
"""

Ecto.Adapters.SQL.query(MyApp.Repo, query, [])

And the stacktrace:

** (exit) exited in: Postgrex.Connection.query(#PID<0.1096.0>, "select to_tsvector('simple', translations.key) from translations\n", [], [timeout: 5000])
    ** (EXIT) an exception was raised:
        ** (ArgumentError) no extension found for oid `3614`
            (postgrex) lib/postgrex/types.ex:284: Postgrex.Types.fetch!/2
            (postgrex) lib/postgrex/types.ex:213: Postgrex.Types.format/2
            (elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
            (elixir) lib/enum.ex:1385: Enum."-reduce/3-lists^foldl/2-0-"/3
            (elixir) lib/enum.ex:1043: Enum.map/2
            (postgrex) lib/postgrex/protocol.ex:72: Postgrex.Protocol.message/3
            (postgrex) lib/postgrex/connection.ex:361: Postgrex.Connection.new_data/2
            (postgrex) lib/postgrex/connection.ex:265: Postgrex.Connection.handle_info/2

postgrex seems to build a table with a mapping and cast of each column. So when it tries to find to mapping of the to_tsvector’s result, it fails here lib/postgrex/types.ex#L280

Thanks for your help!

Postgrex.Utils.version_to_int does not support beta version numbers

because I've yet to upgrade my dev box pg from 9.4beta2, but I'm off to do so now.

** (EXIT) an exception was raised:
        ** (ArgumentError) argument error
:erlang.binary_to_integer("4beta2")
            (elixir) lib/enum.ex:977: Enum."-map/2-lc$^0/1-0-"/2
            (elixir) lib/enum.ex:977: Enum."-map/2-lc$^0/1-0-"/2
            (postgrex) lib/postgrex/utils.ex:31: Postgrex.Utils.version_to_int/1

Discuss caching of type data

Today each postgrex connections needs to query the db when it is started. It would be nice to cache this information and have each connection retrieving it locally instead of hitting the db on load.

Support hstore type

Would be cool if there was encoding/decoding for maps as hstores.

Can open a PR if you'd like.

Postgrex doesn't give error on delete with linked foreign key

I was having similar issue as mentioned in this post
http://stackoverflow.com/questions/14182079/delete-rows-with-foreign-key-in-postgresql

The problem is that if I run the similar query, I get error from PostgreSQL. But when I call the delete query from Postgrex.Connection.query, I get a successful result value. It took a while to figure out the source of the error, as Postgrex didn't gave any error message. Postgrex should give something similar error mesage as returned by PostgreSQL, so that the error and source of the error is clearly identified.

Missing encoder for float8

Hello, I'm using earthdistance module for querying users by coordinates, and I'm getting this error when passing String. If I convert input to float or integer myself and query, then all is good. Here is the log:

Request: GET /users/nearby?longitude=42.6&latitude=12.3
** (exit) an exception was raised:
    ** (FunctionClauseError) no function clause matching in Postgrex.Extensions.Binary.encode/4
        (postgrex) lib/postgrex/extensions/binary.ex:59: Postgrex.Extensions.Binary.encode(%Postgrex.TypeInfo{array_elem: 0, base_type: 0, comp_elems: [], input: "float8in", oid: 701, output: "float8out", receive: "float8recv", send: "float8send", type: "float8"}, "12.3", 901179, {9, 4, 5})

I wanted to fix this myself, and send PR, but not that good in binary << :: :D >> stuff yet, sorry 😄

Error connecting to test database

I see this error off and on when running tests. Sometimes they all run fine and other times I see:

1) test decode basic types (QueryTest)
     test/query_test.exs:16
     ** (MatchError) no match of right hand side value: {:error, %Postgrex.Error{message: "tcp connect: nxdomain", postgres: nil}}
     stacktrace:
       test/query_test.exs:8: QueryTest.__ex_unit_setup_0/1
       test/query_test.exs:1: QueryTest.__ex_unit__/2

Google shows a few threads related to dns settings, but I'm not sure why those would change from time to time.

Add acking of notification messages

The postgrex connection can create a big message box by sending messages faster than the receiving process can churn them. ACKing every message will ensure that this does not happen by queueing messages in the connection. If the queue gets too big messages should be discarded.

Add SSL support

An :ssl=true option could be added and optionally :ssl_options to support connections over untrusted networks (eg. to Heroku Postgres).

Postgrex.Connection kills application on connection refused

Hi Eric,

I am trying to integrate a Postgrex.Connection into my gen_fsm module. The init/1 function looks like this:

#### gen_fsm implementation
def init(params) do
    try_connect_backend(params)
end

defp try_connect_backend(params) do
    case Postgrex.Connection.start_link(hostname: params.host, username: params.user, password: params.pass, database: "spoold") do
        {:ok, conn} ->
            Logger.info("connected to database backend")
            {:ok, :connected, %{:conn => conn, :params => params}, @timeouts.connected}
        {:error, err} ->
            Logger.warn("failed to connect to database backend: #{err}")
            {:ok, :errored, %{:params => params}, @timeouts.errored}
    end
end

In laymens term i'd like to have a connected and an errored state for my fsm representing whether i have a db connection or not. However it's not working as expected. When i run the application, i got the following (i am testing with a database server offline, to see whether the fsm entering errored state):

Compiled lib/spoold_worker.ex
Generated spoold app

14:18:12.404 [info] connected to database backend
Interactive Elixir (1.1.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
14:18:12.437 [info] connected to database backend
14:18:12.445 [info] connected to database backend
14:18:12.450 [info] connected to database backend
14:18:12.450 [error] GenServer #PID<0.143.0> terminating
** (stop) %Postgrex.Error{message: "tcp connect: connection refused - :econnrefused", postgres: nil}
Last message: nil
State: [password: :REDACTED, hostname: "devserver", username: "spoold", database: "spoold"]

14:18:12.454 [error] GenServer #PID<0.149.0> terminating
** (stop) %Postgrex.Error{message: "tcp connect: connection refused - :econnrefused", postgres: nil}
Last message: nil
State: [password: :REDACTED, hostname: "devserver", username: "spoold", database: "spoold"]

14:18:12.458 [error] GenServer #PID<0.151.0> terminating
** (stop) %Postgrex.Error{message: "tcp connect: connection refused - :econnrefused", postgres: nil}
Last message: nil
State: [password: :REDACTED, hostname: "devserver", username: "spoold", database: "spoold"]

14:18:12.461 [error] GenServer #PID<0.153.0> terminating
** (stop) %Postgrex.Error{message: "tcp connect: connection refused - :econnrefused", postgres: nil}
Last message: nil
State: [password: :REDACTED, hostname: "devserver", username: "spoold", database: "spoold"]

14:18:12.461 [info] Application spoold exited: shutdown

So somehow i get back {:ok, pid} from Postgrex.Connection.start_link, but after that Postgrex blows up several times (with the expected error) and my application shuts down.

I am new to elixir, so most likely i am doing something wrong, but i am out of ideas here. Could you tell me how could i catch the error?

Thanks
Andras

Make it easier to install?

I'm new to elixir and I'm having some trouble getting postgrex & ecto installed.

I brew installed elixir 0.13.0. It looks to me like the latest postgrex depends on a dev version of elixir. I found an older revision (7bb042d) and put that in my mix.exs and then I'm able to install it with "mix deps.update postgrex", but I can't compile because ecto wants a newer version of postgrex. I browsed through the ecto history a bit but it wasn't clear to me what revision to use.

Is there any easy way to do this without installing a dev version of elixir?

Thanks very much!

no pg_hba.conf entry for host "xxx.xxx.xxx.xxx"

I have a Postgres Database hosted separately which I'm trying to use with my Phoenix Application. My prod config is:

config :my_app, MyApp.Repo,
  adapter: Ecto.Adapters.Postgres,
  url: "postgres://username:password@myhost:5432/my_database",
  size: 20 

This worked fine while my application was hosted on Heroku, but since I've moved it to a VPS, I keep getting this error:

17:07:13.665 [info] GET /subjects
17:07:13.707 [info] Processing by MyApp.SubjectController.index/2
  Parameters: %{"format" => "html"}
  Pipelines: [:browser, :authorize]
17:07:13.951 [error] GenServer #PID<0.515.0> terminating
** (exit) %Postgrex.Error{message: nil, postgres: %{code: :invalid_authorization_specification, file: "auth.c", line: "474", message: "no pg_hba.conf entry for host \"xxx.xxx.xxx.xxx\", user \"username\", database \"my_database\", SSL off", pg_code: "28000", routine: "ClientAuthentication", severity: "FATAL"}}
17:07:13.970 [info] Sent 500 in 305ms
17:07:13.972 [error] #PID<0.513.0> running MyApp.Endpoint terminated
Server: xxx.xxx.xxx.xxx:80 (http)
Request: GET /subjects
** (exit) exited in: GenServer.call(#PID<0.515.0>, {:query, "SELECT s0.\"id\", s0.\"name\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"subjects\" AS s0", []}, 5000)
    ** (EXIT) %Postgrex.Error{message: nil, postgres: %{code: :invalid_authorization_specification, file: "auth.c", line: "474", message: "no pg_hba.conf entry for host \"xxx.xxx.xxx.xxx\", user \"username\", database \"my_database\", SSL off", pg_code: "28000", routine: "ClientAuthentication", severity: "FATAL"}}

One thing that I noted was that it states my VPS IP as the host instead of the host specified in the url.

HELP!

Postgres.Messages.parse breaking on very large query

I have a very large query I'm running that is causing the following error:

** (FunctionClauseError) no function clause matching in Postgrex.Messages.parse/3
    (postgrex) lib/postgrex/messages.ex:52: Postgrex.Messages.parse(<<129, 230, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, ...>>, 116, 133018)
    (postgrex) lib/postgrex/connection.ex:416: Postgrex.Connection.new_data/2
    (postgrex) lib/postgrex/connection.ex:292: Postgrex.Connection.handle_info/2
    (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
    (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:tcp, #Port<0.109833>, <<49, 0, 0, 0, 4, 116, 0, 2, 7, 158, 129, 230, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, 0, 0, 0, 23, ...>>}

The function clause that should be matching is this:

  def parse(<<len :: int16, rest :: binary(len, 32)>>, ?t, _size) do
    oids = for <<oid :: size(32) <- rest>>, do: oid
    msg_parameter_desc(type_oids: oids)
  end

However, parsing <<129,230>> as int16 actual overflows and returns a negative number (-32,282). If I change the function definition to use uint16 instead, it makes len 33,254, which makes the function clause match, as rest properly matches on 33,254 * 4 bytes (133,016), which then matches the _size parameter of 133,018 (133,016 plus 2 bytes for len).

Note that I'm running this from Ecto 1.0.4, Elixir 1.1.1, and Postgres 9.4.4

Postgrex is breaking the connection when a query fails

Maybe all queries fail like that but I was reproducing the bug by following all steps except step 6 here: elixir-ecto/ecto#781

The root of the bug in that issue is that the database does not exist when making the query. I am assuming that this particular kind of query fails early on, trigger some bits in Postgrex internals it was not supposed to.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.