xataio / pgzx Goto Github PK
View Code? Open in Web Editor NEWCreate PostgreSQL extensions using Zig.
Home Page: https://xata.io
License: Apache License 2.0
Create PostgreSQL extensions using Zig.
Home Page: https://xata.io
License: Apache License 2.0
When using zig build
it automatically defaults to zig build install
. Out of the box one has an install
and uninstall
target.
When running zig build install
our builds do compile the extension and copy all artifacts into the prefix path, which defaults to zig-out
. The zig toolchain "jails" the install path so to say.
If you want to install the extension on your host postgres installation you might need to run the command via sudo
. Unfortunately if the artifacts do not exist yet sudo
would also affect the compilation (and ownership of the build artifacts).
The current workaround is to install the extension twice via:
$ zig build
$ sudo zig build -p $PG_HOME
The first call will install the extension into zig-out
and the second call will install the extension on the host.
Internally we use $PG_HOME
to determine the location of pg_config
. If set correctly in step 1 nothing would be rebuild.
Using $PG_HOME
on the command line feels redundant and might be confusing to our users.
As a first step I would like to update our build support to separate compilation from install. We already introduce custom steps in our build scripts like zig build pg_regress
and zig build unit
for testing. I would like to introduce another build step named compile
that will only compile the artifacts, but not install them yet. CLI usage will become:
$ zig build compile
$ sudo zig build [install] -p $PG_HOME
Discussion on Discuss also proposed to introduce a deploy
step. That would work similar to zig build install
but would 'escape' the jail by copying the artifacts directly to the folders given by pg_config
. CLI usage would become:
$ zig build compile
$ sudo zig build deploy
Note: When using zig build install
we already used pg_config --<dir name>
to determine the target folders. But encourage by how the zig build system installs files we use the configured install-prefix path instead of the absolute paths. In order to implement support for deploy
we would either need to implement our own file/directory copy steps to have better control over the placement or take the easy route and have the deploy
step recursively call zig build install -p $(dirname $(pg_config --bindir))
.
The makeNode
and IsA
macros do not translate on to Zig as they rely on string manipulation to produce the proper C code. The macros use node tags generated at build time and stored in nodes/nodetags.h
. Finally the NodeTag
enum contains all available tags.
One can implement a makeNode
function in Zig like so:
pub fn makeNode(comptime T: type, comptime tag: pg.NodeTag) *T {
const node: *pg.Node = @ptrCast(@alignCast(pg.palloc0fast(@sizeOf(T))));
node.*.type = tag;
return @ptrCast(@alignCast(node));
}
This is not that convenient as one needs to ensure that T
and tag
are compatible. Using code generation it would be nice if we could use makeNode
like so:
const node = makeNode(pg.FdwRoutines);
The makeNode
functions would use a comptime table with type/tag information like:
const typeTagTable = .{
// tuple of: type -> tag
{ pg.Query, pg.T_Query },
...
};
inline fn findTypeInfo(comptime T: type) type {
comptime for (typeTagTable) |e| {
if (e[0] == T)
return e;
}
@compileError("....");
}
inline fn findTypeTag(comptime T: type) pg.NodeTag {
return findTypeInfo(T)[1];
}
pub fn makeNode(comptime T: type) *T {
const node: *pg.Node = @ptrCast(@alignCast(pg.palloc0fast(@sizeof(T))));
node.*.type = findTypeTag(T);
return @ptrCast(@alignCast(node));
}
The typeTagTable
must be generated based on the the C code. Ideally from the header files.
At least the IsA
macro needs a Zig helper as well.
The readme is getting quite big and it is becoming difficult for users to find the right information or we didn't yet document things properly. To improve the DX and especially getting started experience I would like to split the readme out into a proper documentation.
TODO: sections/structure?
Hi,
Could we please get some more advanced examples?
E.g. something like https://github.com/supabase/pg_replicate but with Zig instead of Rust ?
Thank you.
For some projects the accompanying SQL file can become quite large. I'd love to have support to split the SQL file for development purposes and combine them at build time.
Due to SQL files being statements being executed one after another users must be able to correctly configure the order of those files.
We show how to build the extension using the Postgres from the development environment, but the next step is installing the resulting extension, or building it for the target production DB.
Thanks for great work and a nice demo on YouTube.
In the demo, you showed how passing an empty string when expecting one char, resulted in an error that crashed PostgreSQL, albeit with a nice user-friendly error message from Zig, which is of course much better than the segfault C would cause.
However, crashing a production PostgreSQL database just because there is a bug in a Zig extension database function, might not always be desirable.
If possible, it would be nice with a setting that allows logging the error and just returning NULL instead of crashing.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.