microo8 / plgo Goto Github PK
View Code? Open in Web Editor NEWeasily create postgresql extensions in golang; moved to gitlab.com/microo8/plgo
easily create postgresql extensions in golang; moved to gitlab.com/microo8/plgo
Hi,
in my function I need to get as input parameter jsonb type. Is this supported ? I have tried with interface{} and it is not ok. Do you have any suggestion how I should resolve this?
Changed to plgo.go --
/*
#cgo CFLAGS: -I"/usr/include/postgresql/11/server" -fpic
#cgo LDFLAGS: -shared
//{windowsCFLAGS}
So I could build go module on deb latest.
Just an fyi --
Getting this error while trying to run plgo example/
2020/04/21 19:00:25 /var/folders/fx/2b_wyk0s123gqqlhvgk3g98svm9dbl/T/plgo381626588
runtime/cgo
command-line-arguments
# command-line-arguments
Undefined symbols for architecture x86_64:
"_DirectFunctionCall1Coll", referenced from:
_jsonb_to_datum in _x003.o
__cgo_9759c58d933a_Cfunc_jsonb_to_datum in _x003.o
"_JsonbToCString", referenced from:
_datum_to_jsonb_cstring in _x003.o
__cgo_9759c58d933a_Cfunc_datum_to_jsonb_cstring in _x003.o
"_SPI_connect", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_connect in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_connect)
"_SPI_execute_plan", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_execute_plan in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_execute_plan)
"_SPI_finish", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_finish in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_finish)
"_SPI_fname", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_fname in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_fname)
"_SPI_getbinval", referenced from:
_get_col_as_datum in _x003.o
__cgo_9759c58d933a_Cfunc_get_col_as_datum in _x003.o
"_SPI_gettype", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_gettype in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_gettypeid, __cgo_9759c58d933a_Cfunc_SPI_gettype )
"_SPI_gettypeid", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_gettypeid in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_gettypeid)
"_SPI_prepare", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_prepare in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_prepare)
"_SPI_processed", referenced from:
__cgohack_SPI_processed in _cgo_main.o
(maybe you meant: __cgohack_SPI_processed)
"_SPI_result", referenced from:
__cgohack_SPI_result in _cgo_main.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_result_code_string, __cgohack_SPI_result )
"_SPI_result_code_string", referenced from:
__cgo_9759c58d933a_Cfunc_SPI_result_code_string in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_SPI_result_code_string)
"_SPI_tuptable", referenced from:
__cgohack_SPI_tuptable in _cgo_main.o
(maybe you meant: __cgohack_SPI_tuptable)
"_construct_md_array", referenced from:
_array_to_datum in _x003.o
__cgo_9759c58d933a_Cfunc_array_to_datum in _x003.o
"_cstring_to_text", referenced from:
_cstring_to_datum in _x003.o
__cgo_9759c58d933a_Cfunc_cstring_to_datum in _x003.o
"_deconstruct_array", referenced from:
_datum_to_array in _x003.o
__cgo_9759c58d933a_Cfunc_datum_to_array in _x003.o
"_elog_finish", referenced from:
_elog_notice in _x003.o
__cgo_9759c58d933a_Cfunc_elog_notice in _x003.o
_elog_error.cold.1 in _x003.o
"_elog_start", referenced from:
_elog_notice in _x003.o
__cgo_9759c58d933a_Cfunc_elog_notice in _x003.o
_elog_error.cold.1 in _x003.o
"_get_call_expr_argtype", referenced from:
__cgo_9759c58d933a_Cfunc_get_call_expr_argtype in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_get_call_expr_argtype)
"_get_typlenbyvalalign", referenced from:
_array_to_datum in _x003.o
_datum_to_array in _x003.o
__cgo_9759c58d933a_Cfunc_array_to_datum in _x003.o
__cgo_9759c58d933a_Cfunc_datum_to_array in _x003.o
"_getmissingattr", referenced from:
_get_heap_getattr in _x003.o
"_heap_form_tuple", referenced from:
__cgo_9759c58d933a_Cfunc_heap_form_tuple in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_heap_form_tuple)
"_heap_getsysattr", referenced from:
_get_heap_getattr in _x003.o
"_jsonb_in", referenced from:
_jsonb_to_datum in _x003.o
__cgo_9759c58d933a_Cfunc_jsonb_to_datum in _x003.o
"_nocachegetattr", referenced from:
_get_heap_getattr in _x003.o
"_palloc", referenced from:
_bytes_to_datum in _x003.o
__cgo_9759c58d933a_Cfunc_bytes_to_datum in _x003.o
"_palloc0", referenced from:
_array_to_datum in _x003.o
__cgo_9759c58d933a_Cfunc_array_to_datum in _x003.o
"_parseTypeString", referenced from:
__cgo_9759c58d933a_Cfunc_parseTypeString in _x003.o
(maybe you meant: __cgo_9759c58d933a_Cfunc_parseTypeString)
"_pg_detoast_datum", referenced from:
_jsonb_to_datum in _x003.o
_datum_to_array in _x003.o
_datum_to_jsonb_cstring in _x003.o
__cgo_9759c58d933a_Cfunc_datum_to_array in _x003.o
__cgo_9759c58d933a_Cfunc_datum_to_jsonb_cstring in _x003.o
__cgo_9759c58d933a_Cfunc_jsonb_to_datum in _x003.o
"_pg_detoast_datum_packed", referenced from:
_datum_to_byteap in _x003.o
__cgo_9759c58d933a_Cfunc_datum_to_byteap in _x003.o
"_text_to_cstring", referenced from:
_datum_to_cstring in _x003.o
__cgo_9759c58d933a_Cfunc_datum_to_cstring in _x003.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Cannot build package: exit status 2
Writing a go function that returns []byte will generate a sql header like:
CREATE OR REPLACE FUNCTION CompressGzip(data text,level bigint)
RETURNS [] AS
'$libdir/vl', 'CompressGzip'
LANGUAGE c VOLATILE STRICT;
From this go function:
func CompressGzip(data string, level int) []byte {
return cGzip([]byte(data), level);
}
Amending that sql header and installing the function will result in a server crash on use of the function.
My extension work completely well on normal server under different PG version.
But I can't make it work in docker container, it's very strange. :(
goroutine 17 [running, locked to thread]: ???!!
`panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x7f5ed3227d5d]
goroutine 17 [running, locked to thread]:
main.crimeTrg(0x0, 0x0)
/tmp/plgo140856321/package.go:48 +0x9d
main.CrimeTrg(0x7ffc9c66eeb0, 0xc400000008)
/tmp/plgo140856321/methods.go:11 +0x3b
main._cgoexpwrap_b36115c3fa71_CrimeTrg(0x7ffc9c66eeb0, 0x1a1d0ac2cc66ca00)
command-line-arguments/_obj/_cgo_gotypes.go:1374 +0x2d
`
All blocks on:
td.NewRow.Scan(&id)
td.FiredAfter() ....
Hi,
Im getting this when trying to install it:
# github.com/microo8/plgo
/root/go/src/github.com/microo8/plgo/pl.go:7:22: fatal error: postgres.h: No such file or directory
#include "postgres.h"
^
compilation terminated.
I have installed all devel packages etc. Im on Pg9.6, CentOS 7
go version:
go version go1.10.3 linux/amd64
os:
Linux nil 4.17.3-1-ARCH #1 SMP PREEMPT Tue Jun 26 04:42:36 UTC 2018 x86_64 GNU/Linux
I was unable to compile using plgo.
I will provide the code if required.
Cannot format package format.Node internal error (127:1: expected declaration, found 'import' (and 5 more errors))
Thank you.
I try figure out how get new data from trigger.
I tried:
var res interface{}
td.NewRow.Set(&res)`
var res string
td.NewRow.Set(0,&res)
log.Println("res",res)
Don't work for me.??
Hello! There are situations when expansion stops working with panic,
psql:
server unexpectedly closed connection
Most likely the server has stopped working due to a failure
before or during the request.
The connection to the server was lost. Attempting reset: Failed.
How to make detailed error information displayed as if it were a regular program?
or jdbc_driver: java.io.EOFException
Привет! Бывают ситуации когда расширение прекращает работу с паникой,
psql :
сервер неожиданно закрыл соединение
Скорее всего сервер прекратил работу из-за сбоя
до или в процессе выполнения запроса.
The connection to the server was lost. Attempting reset: Failed.
как сделать чтобы выводилась подробная информация об ошибке как если бы это была обычная программа?
jdbc_driver пишет : java.io.EOFException
I installed postgresql-server-dev-all
Then I run go get -u github.com/microo8/plgo/plgo
.
After that, I downloaded the example from github https://github.com/microo8/plgo/blob/master/example/example_methods.go
When I run ~/go/bin/plgo .
I got the following result:
Package github.com/microo8/plgo not installed
please install it with: go get -u github.com/microo8/plgo/...
I follow the instructions and run go get -u github.com/microo8/plgo/...
$ go get -u github.com/microo8/plgo/...
# github.com/microo8/plgo
/root/go/src/github.com/microo8/plgo/pl.go:7:10: fatal error: postgres.h: No such file or directory
#include "postgres.h"
^~~~~~~~~~~~
compilation terminated.
# github.com/microo8/plgo/test/types
/root/go/src/github.com/microo8/plgo/test/types/pl.go:7:10: fatal error: postgres.h: No such file or directory
#include "postgres.h"
^~~~~~~~~~~~
compilation terminated.
# github.com/microo8/plgo/test/bgw
/root/go/src/github.com/microo8/plgo/test/bgw/background_worker.go:7:10: fatal error: postgres.h: No such file or directory
#include "postgres.h"
^~~~~~~~~~~~
compilation terminated.
Am I missing something obvious?
It seems that adding Windows support updated the CFLAGS line to #cgo CFLAGS: -I"/usr/include/postgresql/server" -fpic
has added quotes which were not present previously, causing what was the end of the line to be the middle. The path here is provided from the call to pg_config --includedir-server
which includes a newline character at the end, since this newline is no longer the final character in the declaration this produces a syntax error. This is fixed by getcorrectpath
for Windows only, leaving the Linux version failing.
Here is a Dockerfile for reproducing this issue:
FROM golang:1.13
RUN apt-get update && \
apt-get -y install postgresql-server-dev-11 && \
go get -u github.com/microo8/plgo/plgo && \
plgo /go/src/github.com/microo8/plgo/example/.
Running this produces the following error:
can't load package: package main: /go/plgo072699724/pl.go: invalid #cgo line: #cgo CFLAGS: -I"/usr/include/postgresql/11/server
By trimming the newline character from the end of the path for the non-Windows version of getcorrectpath
I am able to build successfully. (#33) Not sure if I am missing something here since based on my understanding I'd expect any Linux build to hit the same problem and I haven't been able to find any other reference to this error.
Hi,
We did some experiments, look at: https://github.com/prest/bgworker
go get -u github.com/microo8/plgo/...
give me:
microo8/plgo/pl.go:579: cannot use *_Cvar_SPI_processed (type C.uint32) as type C.uint64 in argument to newRows
There is no rule for building the target “sample.bc” required for “all”. Stop
Pls help
$ cd build
$ sudo make install
make: *** There is no rule for building the target “sample.bc” required for “all”. Stop.
PostgreSQL:
PostgreSQL 11.2 (Ubuntu 11.2-1.pgdg18.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0, 64-bit
Go:
go1.12.2 linux/amd64
Linux:
Ubuntu 18.04.2 LTS
GCC:
gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
G++:
g++ (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
Makefile:
EXTENSION = sample
DATA = sample--0.1.sql # script files to install
# REGRESS = sample_test # our test script file (without extension)
MODULES = sample # our c module file to build
# postgres build stuff
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
Line 129 in a744149
Hi @microo8 sorry to bothering you again could you add support for multiple return types. I thought I could make a PR but my go skills still need some work lol. BUT I've made my homework here is the AST on my package :
33 . . . Type: *ast.FuncType {
34 . . . . Func: returned-values.go:5:1
35 . . . . Params: *ast.FieldList {
36 . . . . . Opening: returned-values.go:5:7
37 . . . . . Closing: returned-values.go:5:8
38 . . . . }
39 . . . . Results: *ast.FieldList {
40 . . . . . Opening: returned-values.go:5:10
41 . . . . . List: []*ast.Field (len = 3) {
42 . . . . . . 0: *ast.Field {
43 . . . . . . . Type: *ast.ArrayType {
44 . . . . . . . . Lbrack: returned-values.go:5:11
45 . . . . . . . . Elt: *ast.Ident {
46 . . . . . . . . . NamePos: returned-values.go:5:13
47 . . . . . . . . . Name: "float64"
48 . . . . . . . . }
49 . . . . . . . }
50 . . . . . . }
51 . . . . . . 1: *ast.Field {
52 . . . . . . . Type: *ast.ArrayType {
53 . . . . . . . . Lbrack: returned-values.go:5:22
54 . . . . . . . . Elt: *ast.Ident {
55 . . . . . . . . . NamePos: returned-values.go:5:24
56 . . . . . . . . . Name: "float64"
57 . . . . . . . . }
58 . . . . . . . }
59 . . . . . . }
60 . . . . . . 2: *ast.Field {
61 . . . . . . . Type: *ast.ArrayType {
62 . . . . . . . . Lbrack: returned-values.go:5:33
63 . . . . . . . . Elt: *ast.Ident {
64 . . . . . . . . . NamePos: returned-values.go:5:35
65 . . . . . . . . . Name: "float64"
66 . . . . . . . . }
67 . . . . . . . }
68 . . . . . . }
69 . . . . . }
Is possible to get SETOF support
Planning to migrate to PG13.
Can you tell me if plgo is compatible with PG13?
Hi! Vlado, I'm sorry. When i do
$ cd build
$ sudo make install,
then a message appears:
make: *** There is no rule for building the target “main.bc” required for “all”. Stop.
$ more Makefile
EXTENSION = main
DATA = main--0.1.sql # script files to install
# REGRESS = main_test # our test script file (without extension)
MODULES = main # our c module file to build
# postgres build stuff
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
How to fix it tell me?
the same as #19
i hava use go get -u github.com/microo8/plgo/...
but can't works
golang version
go version
go version go1.10.4 linux/amd64
postgresql version
with pg_config
VERSION = PostgreSQL 10.7 (Ubuntu 10.7-0ubuntu0.18.04.1)
Hi there,
I'm quite new to Go...
I can't figure out how to get my .so file to build.
First problem is that Go won't let me have my go file (main package) alongside the pl.go file (plgo package).
I tried to turn my package into plgo's.
Second problem is that I get an error:
# command-line-arguments
/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/../../../../x86_64-pc-linux-gnu/bin/ld: -r and -shared may not be used together
collect2: error: ld returned 1 exit status
Any help would be really appreciated,
Regards!
Hi @microo8 I was searching how to apply my functions on an entire column and not on an array I found useful articles about custom aggregations:
Do you think it could be tricky to create a type Aggregate and automate the rewrite of the functions in the sql file ?
Hi!
Have a truble:
$ plgo .
/tmp/plgo272805124/pl.go:366:15: could not determine kind of name for C.FunctionCallInfoBaseData
And there's no such mistake on the other computer (Ubuntu 19.10 + go1.13.4).
I cannot understand what the error is.
Help please.
go version go1.12.2 linux/amd64
gcc (Ubuntu 7.4.0-1ubuntu1 18.04.1) 7.4.0
g++ (Ubuntu 7.4.0-1ubuntu1 18.04.1) 7.4.0
Ubuntu 18.04.3 LTS
/tmp/plgo197184652/pl.go: In function ‘jsonb_to_datum’:
/tmp/plgo197184652/pl.go:148:17: warning: implicit declaration of function ‘DatumGetJsonb’; did you mean ‘DatumGetJsonbP’? [-Wimplicit-function-declaration]
return (Datum) DatumGetJsonb(DirectFunctionCall1(jsonb_in, (Datum) (char *) val));
^~~~~~~~~~~~~
DatumGetJsonbP
/tmp/plgo197184652/pl.go: In function ‘datum_to_jsonb_cstring’:
/tmp/plgo197184652/pl.go:226:17: warning: initialization of ‘Jsonb *’ {aka ‘struct *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
Jsonb *jsonb = DatumGetJsonb(val);
^~~~~~~~~~~~~
/tmp/plgo197184652/pl.go:347:171: cannot convert _Ciconst_true (type untyped number) to type _Ctype__Bool
/tmp/plgo197184652/pl.go:347:171: invalid operation: (func literal)() == _Ciconst_true (mismatched types _Ctype__Bool and int)
/tmp/plgo197184652/pl.go:400:51: cannot convert _Ciconst_true (type untyped number) to type _Ctype__Bool
/tmp/plgo197184652/pl.go:400:51: invalid operation: _Cfunc_trigger_fired_before(td.tgEvent) == _Ciconst_true (mismatched types _Ctype__Bool and int)
/tmp/plgo197184652/pl.go:405:50: cannot convert _Ciconst_true (type untyped number) to type _Ctype__Bool
/tmp/plgo197184652/pl.go:405:50: invalid operation: _Cfunc_trigger_fired_after(td.tgEvent) == _Ciconst_true (mismatched types _Ctype__Bool and int)
/tmp/plgo197184652/pl.go:410:52: cannot convert _Ciconst_true (type untyped number) to type _Ctype__Bool
/tmp/plgo197184652/pl.go:410:52: invalid operation: _Cfunc_trigger_fired_instead(td.tgEvent) == _Ciconst_true (mismatched types _Ctype__Bool and int)
/tmp/plgo197184652/pl.go:415:52: cannot convert _Ciconst_true (type untyped number) to type _Ctype__Bool
/tmp/plgo197184652/pl.go:415:52: invalid operation: _Cfunc_trigger_fired_for_row(td.tgEvent) == _Ciconst_true (mismatched types _Ctype__Bool and int)
/tmp/plgo197184652/pl.go:415:52: too many errors
Cannot build package: exit status 2
# command-line-arguments
/tmp/plgo748230890/pl.go:35:5: error: conflicting types for 'varsize_any'
int varsize_any(void *var) {
^
In file included from /tmp/plgo748230890/pl.go:13:0:
/usr/pgsql-11/include/server/access/htup_details.h:835:15: note: previous declaration of 'varsize_any' was here
extern size_t varsize_any(void *p);
^
Cannot build package: exit status 2
Hi Vlado!
yesterday everything was fine, today there was an error
I upload files
'/usr/lib/postgresql/11/lib/main.h'
'/usr/lib/postgresql/11/lib/main.so'
'/usr/share/postgresql/11/extension/main.control'
'/usr/share/postgresql/11/extension/main--0.1.sql'
create extension main;
[XX000] ERROR: load the library "/usr/lib/postgresql/11/lib/main.so" failed: /usr/lib/postgresql/11/lib/main.so: undefined symbol: getmissingattr
What could it be ?
and how to set another package name not main?
I'm trying to build the example using plgo
but it fails with a long
Undefined symbols for architecture x86_64:
"_DirectFunctionCall1Coll", referenced from:
_jsonb_to_datum in _x003.o
__cgo_2dee455c7a62_Cfunc_jsonb_to_datum in _x003.o
"_JsonbToCString", referenced from:
_datum_to_jsonb_cstring in _x003.o
__cgo_2dee455c7a62_Cfunc_datum_to_jsonb_cstring in _x003.o
...
My guess is that my version of postgres (10) is incompatible with plgo perhaps. Which versions are supported?
Regarding your example concatll, when accessing a field with a text type, it gives an error "I / O error when sending backend", does not work with a field of type varchar,
I added in pl.go to the func scanVal in
case * string
switch oid {
case C.VARCHAROID:
and it began to work correctly with a field of type varchar, but with field of type text does not work: (
Hi @microo8 I have this issue when I run plgo in my package https://github.com/Sach97/vector-operation
However I've seen in your code that you have implemented the type conversion []float64 to double precision[]. I don't know what I did wrong it works fine when I run "go run main.go". Do you have an idea of what I could change in my code to run plgo ?
I try use yours example on
VERSION = PostgreSQL 9.6.2
All function work correctly a part of CreatedTimeTrigger()
`CREATE TRIGGER gettrigger BEFORE INSERT OR UPDATE OR DELETE ON urls FOR EACH ROW EXECUTE PROCEDURE CreatedTimeTrigger();
CREATE TRIGGER so looks OK
`
`
func CreatedTimeTrigger(td *plgo.TriggerData) *plgo.TriggerRow {
// td.NewRow.Set(4, time.Now()) //set the 4th column to now()
// td.NewRow.Set(1, "trig")
// log.Println("trigget start")
return td.NewRow //return the new modified row
}
But after
INSERT INTO urls VALUES ('333llssl');`
server closed the connection unexpectedly
Hello!
Help me pls, when I try to insert or update, postgres gives an error
ERROR: INSERT cannot be used in non-volatile function
Although it is indicated in the generated sql file that the volatility function
CREATE OR REPLACE FUNCTION SaveData()
RETURNS text AS
'$libdir/main', 'SaveData'
LANGUAGE c VOLATILE STRICT;
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.