PostgreSQL schema and data comparing tool
michaelsogos / pg-diff Goto Github PK
View Code? Open in Web Editor NEWPostgreSQL schema and data comparing tool
License: MIT License
PostgreSQL schema and data comparing tool
License: MIT License
PostgreSQL schema and data comparing tool
error: column p.prokind does not exist
Sorry, don't have much here, and a little debugging shows that the issue is actually in the pg-diff-api node module but it's through this so I figured I'd make the issue here. I ended up fixing it by removing the AND p.prokind = 'f'
Help!!
$ pg-diff -c development initial-migration
_ _ __ __ _ _
_ __ __ _ __| | (_) / _| / _| ___ | | (_)
| '_ \ / _` | _____ / _` | | | | |_ | |_ _____ / __| | | | |
| |_) | | (_| | |_____| | (_| | | | | _| | _| |_____| | (__ | | | |
| .__/ \__, | \__,_| |_| |_| |_| \___| |_| |_|
|_| |___/
Author: [object Object]
Version: 1.2.8
PostgreSQL: 9.6+
License: MIT
Description: PostgreSQL schema and data comparing tool
CONFIGURED OPTIONS
Script Author: @G
Output Directory: .../ops/migrations/sqlscripts
Schema Namespaces: public
Idempotent Script: ENABLED
Data Compare: ENABLED
Connected to PostgreSQL 11.4 (Debian 11.4-1.pgdg90+1) on [192.168.99.100:5432/dev] ✓
Connected to PostgreSQL 11.4 (Debian 11.4-1.pgdg90+1) on [192.168.99.100:6543/pro] ✓
Collect SOURCE database objects
[|||||||||||||||||||] 100% - Database objects collected!
Collect TARGET database objects
[|||||||||||||||||||] 100% - Database objects collected!
Compare SOURCE with TARGET database objects
[|||||||||||||||||||] 100% - Database objects compared!
Collect SOURCE tables records
[|||||||||||||||||||] 100% - Table records collected!
Collect TARGET tables records
[|||||||||||||||||||] 100% - Table records collected!
Compare SOURCE with TARGET database table records
[-------------------] 0% - Comparing table "public"."entity" records
TypeError: value.replace is not a function
TypeError: value.replace is not a function
at Object.__generateSqlFormattedValue (.../ops/migrations/node_modules/pg-diff-cli/src/sqlScriptGenerator.js:353:34)
at Object.generateInsertTableRecordScript (.../ops/migrations/node_modules/pg-diff-cli/src/sqlScriptGenerator.js:316:35)
at sourceTableRecords.records.rows.forEach (.../ops/migrations/node_modules/pg-diff-cli/src/compareRecords.js:100:45)
at Array.forEach (<anonymous>)
at Object.__compareTableRecords (.../ops/migrations/node_modules/pg-diff-cli/src/compareRecords.js:69:41)
at Object.compareTablesRecords (.../ops/migrations/node_modules/pg-diff-cli/src/compareRecords.js:44:22)
at __runComparison (.../ops/migrations/node_modules/pg-diff-cli/main.js:306:49)
at process._tickCallback (internal/process/next_tick.js:68:7)
In the schema compare, if idempotent = true, when pg-diff creates the query to add a constraint, it creates this command:
ALTER TABLE IF EXISTS "public". "Account_tax_regulation" ADD CONSTRAINT "uk_4j7hbk7rp1tmvj37ae7gdfl7" UNIQUE (name);
however, this only verifies that the table exists and not that the constraint exists. so when the script is executed, if the constraint already exists, it fails.
to be able to check existence, there is no "ADD COSTRAINT IF EXISTS" in postgress, therefore it is necessary to resort to the search query.
in order to do this, I will give you an example:
DO $$ begin
IF NOT EXISTS (SELECT constraint_name
FROM information_schema.constraint_column_usage
WHERE table_schema = 'public'
AND table_name = 'account_tax_regulation'
AND constraint_name = 'uk_4j7hbk7rp1tmvj37ae7gdfl7') THEN
ALTER TABLE IF EXISTS "public". "Account_tax_regulation" ADD CONSTRAINT
"uk_4j7hbk7rp1tmvj37ae7gdfl7" UNIQUE (name);
END IF;
END $$;
the commands DO $$
and END $$;
are necessary in order to run IF STATEMENT in a script
IMPORTANT NOTE:
in case you have to look for a foreign key you have to look for it in the external table as a contraint not in the one in which the foreign Key is missing
eg:
DO $$ begin
IF NOT EXISTS (SELECT constraint_name
FROM information_schema.constraint_column_usage
WHERE table_schema = 'public'
AND table_name = 'base_partner'
AND constraint_name = 'fk_qlqlr0w9e99cy3fu0y9339rnq') THEN
ALTER TABLE IF EXISTS "public". "Sale_sale_order" ADD CONSTRAINT
"fk_qlqlr0w9e99cy3fu0y9339rnq" FOREIGN KEY (client_partner)
REFERENCES base_partner (id);
END IF;
END $$;
base_partner in this case is the external reference table of the foreign key and the SELECT must be made on that table
best regards
Mattia
Is there a way to generate a separate scripts for the functions and tables.
And also how can we ignore onwers while generating the script.
Hello Michaelsogos.
I started using pg-diff, and it works like a charm.
Could you just tell me if it's possible to revert to a certain version on a source system.
Target system aside for now, let's consider the following case.
I am developing my data model, adding some functions and saving it to migration table - V1. Then I add some other db stuff and save it again - V2. Next I start working on some new feature and create V3. What if I realize that this feature is not needed or I just didn't implement it correctly, is there some convenient way to revert back to V2 or V1?
To summarize, I would like to track my development progress on the source system and revert to a certain version if needed. For example: Compare V2 and V3 and rollback all changes made during V2 >> V3 migration
Best regards,
Ivan
Hello,
The comments in our database are mission critical. Tables, schema, columns, constraints, and views all have comments that I need to compare and create a diff for. Can this functionality be added to pg-diff?
The goal is to add 'COMMENT ON' lines in the generated patch file that will add, set to null, and/or update comments on targetClient so that it matches up with the comments on sourceClient.
Thank you for your time
If I have fields with break lines inside, every time I create the migration file, pg-diff includes these fields as if there were differences.
MBE
Replication Steps:
"schemaCompare": {
"namespaces": ["public"],
"idempotentScript": true,
"dropMissingTable": false,
"dropMissingView": false,
"dropMissingFunction": false
},
Expected: Fails with an error saying roles is required (or a null check making them not required)
Actual: TypeError: Cannot read property 'length' of undefined
hi, Do you plan to implement this?
I would prefer that the tool simply compares all schema's it finds in the databases. Right now, if I create a new schema and forget to add it in my migration config the migration script will be wrong.
Bit of background here... I'm using Sequelize to create the models and in there I have one column which is declared as a string array.
@column(DataType.ARRAY({type: DataType.STRING}))
attributes: string[]
This generates a table where the column is defined as...
attributes character varying(255)[]
When I store data in that column, it takes the format...
'{ATTR_1, ATTR_2}'
Inside the __generateSqlFormattedValue function, it tries to handle arrays by doing a value.join as if the data was genuinely an array as opposed to a comma separated value enclosed in curly braces as above. And thus, join is not a function leading to...
TypeError: value.join is not a function
at Object.__generateSqlFormattedValue (C:\Users\rossc\AppData\Local\Yarn\Data\global\node_modules\pg-diff-api\src\sqlScriptGenerator.js:631:35) at Object.generateDeleteTableRecordScript (C:\Users\rossc\AppData\Local\Yarn\Data\global\node_modules\pg-diff-api\src\sqlScriptGenerator.js:593:54)
at C:\Users\rossc\AppData\Local\Yarn\Data\global\node_modules\pg-diff-api\src\api\CompareApi.js:1361:30
Not sure whether somehow my data is stored incorrectly by Postgres or whether its a bug in the code.
Would appreciate your thoughts.
hello, great tool!
With version 2.4.1 and PostgreSQL: 9.6+ (15.3) if you set migrationOptions.historyTableSchema to a schema different from 'public':
Running this:
pg-diff -c generate migration > /dev/null
results in no output. Even though there is no file there.
$ pg-diff -c generate migration
_ _ __ __ _ _
_ __ __ _ __| | (_) / _| / _| ___ | | (_)
| '_ \ / _` | _____ / _` | | | | |_ | |_ _____ / __| | | | |
| |_) | | (_| | |_____| | (_| | | | | _| | _| |_____| | (__ | | | |
| .__/ \__, | \__,_| |_| |_| |_| \___| |_| |_|
|_| |___/
Author: Michael Sogos <[email protected]> (https://github.com/michaelsogos)
Version: 2.1.0
PostgreSQL: 9.6+
License: MIT
Description: PostgreSQL schema and data comparing tool
Error: Cannot find module '/project-dir/pg-diff-config.json'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
at Function.Module._load (internal/modules/cjs/loader.js:562:25)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Function.LoadConfig (/usr/local/lib/node_modules/pg-diff-cli/src/ConfigHandler.js:11:20)
at Run (/usr/local/lib/node_modules/pg-diff-cli/main.js:68:32)
at Object.<anonymous> (/usr/local/lib/node_modules/pg-diff-cli/main.js:15:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
Please create the configuration file "pg-diff-config.json" in the same folder where you run pg-diff!
Hi !
I'm comparing two schema, where the newer schema has new columns with data.
The data compare feature correctly creates the INSERT statement (populating the new columns with the values), but does not include the new columns in UPDATE statements.
Example :
-- SCHEMA A
CREATE TABLE public.test (
id serial NOT NULL,
"name" varchar NULL,
CONSTRAINT test_pk PRIMARY KEY (id)
);
INSERT INTO test VALUES (1, 'john');
-- SCHEMA B
CREATE TABLE public.test (
id serial NOT NULL,
"name" varchar NULL,
"surname" varchar NULL,
CONSTRAINT test_pk PRIMARY KEY (id)
);
INSERT INTO test VALUES (1, 'johnny', 'wayne');
INSERT INTO test VALUES (2, 'alfred', 'hitchcock');
Using config
{
"test": {
"targetClient": {
...
"database": "testa",
...
},
"sourceClient": {
...
"database": "testb",
...
},
"compareOptions": {
...
"dataCompare": {
"enable": true,
"tables": [
{"tableSchema":"public", "tableName":"test", "tableKeyFields":["id"]}
]
}
}
}
}
I get the script
-- ...
ALTER TABLE IF EXISTS "public"."test" ADD COLUMN IF NOT EXISTS "surname" varchar NULL ;
-- ...
INSERT INTO "public"."test" ("id", "name", "surname") VALUES (2, 'alfred', 'hitchcock');
UPDATE "public"."test" SET "name" = 'johnny' WHERE "id" = 1;
-- ...
See that "john" is correctly updated to "johnny", but the "wayne" is not populated at all, while "hitchcock" is correctly populated for the newly inserted value.
aws redshift uses a flavour of postgres and a couple of funcoitns aren't exactly the same.
for instance:
show server_version;
actually is SELECT version();
because of this error the comparison fails instantly as I'm guessing that's pretty early in the process.
please would you consider allowing us to specify redshift and alter some sys queries?
` Author: [object Object]
Version: 2.2.0
PostgreSQL: 9.6+
License: MIT
Description: PostgreSQL schema and data comparing tool
CONFIGURED OPTIONS
Script Author: your-name-or-nickname-or-anything-else
Output Directory: /home/zeubs/Downloads/pg-diff/sqlscripts
Schema Namespaces: will be retrieve dynamically from database
Data Compare: DISABLED
[|------------------] 10% - Connecting to source database ...
error: must be superuser to examine "server_version"
at Parser.parseErrorMessage (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.stream.on (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:224:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
`
Does pg-diff allow for filtering for a list of tables in a schema to perform schema compare for? Does it support filtering for a list of schemas i.e. schema1.employee?
Does pg-diff enable snapshotting a schema to a file and using this file as in input to schema compare?
I think triggers are ignored for the source/target comparison. I tested with two databases where the source has a few new tables with triggers on it, but there are no CREATE TRIGGER ...
statements in the resulting script.
Hi there,
Firstly, thank you so much for your amazing tool.
Do you think pg-diff might one day support the use of this extension: https://github.com/sraoss/pg_ivm?
Thanks :)
Lydia
Hi !
In my scripts, the following statements are generated :
COMMENT ON INDEX sidx_landvaluezone_geom IS NULL;
They fail to apply with error ERROR: relation "sidx_landvaluezone_geom" does not exist
.
If I manually specify the schema
COMMENT ON INDEX topo.sidx_landvaluezone_geom IS NULL;
it works as expected.
Hi @michaelsogos,
thank you for this awesome tool! Sadly, I'm experiencing a TypeError with data-compare as soon as a table contains a filled integer[] field. If data-compare is disabled or the field contains null, everything works fine. I'm looking forward for any help. Thank you!
This is how it looks like:
$ pg-diff -c development initial-script
_ _ __ __ _ _
_ __ __ _ __| | (_) / _| / _| ___ | | (_)
| '_ \ / _` | _____ / _` | | | | |_ | |_ _____ / __| | | | |
| |_) | | (_| | |_____| | (_| | | | | _| | _| |_____| | (__ | | | |
| .__/ \__, | \__,_| |_| |_| |_| \___| |_| |_|
|_| |___/
Author: [object Object]
Version: 2.0.4
PostgreSQL: 9.6+
License: MIT
Description: PostgreSQL schema and data comparing tool
CONFIGURED OPTIONS
Script Author: pg-diff-cli
Output Directory: /data/pg-diff/sqlscripts
Schema Namespaces: public
Data Compare: ENABLED
[||||||||||||||-----] 75% - SEQUENCE objects have been compared
TypeError: value.replace is not a function
at Object.__generateSqlFormattedValue (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/sqlScriptGenerator.js:598:22)
at Object.generateInsertTableRecordScript (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/sqlScriptGenerator.js:545:26)
at /usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:1180:10
at Array.forEach (<anonymous>)
at Function.compareTableRecords (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:1140:37)
at Function.compareTablesRecords (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:1018:31)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async Function.compare (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:63:9)
at async PgDiff.compare (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/index.js:36:10)
at async Run (/usr/local/lib/node_modules/pg-diff-cli/main.js:79:26)
Steps to reproduce:
pg-diff -c development initial-script
pg-diff-config.json:
...
"dataCompare": {
"enable": true,
"tables": [
{
"tableName": "diff_test",
"tableSchema": "public",
"tableKeyFields": ["id"]
}
]
}
...
diff_test.sql
create table public.diff_test (
id serial not null
constraint diff_test_pk
primary key,
nums integer[]
);
INSERT INTO public.diff_test (id, nums) VALUES (1, '{1,2}');
From the documentation, it's not quite clear if this would provide the functionality I want. Could you add a "Hello World" type of example output? It could be based on just one or maybe two tables with only a few rows. Then show the output from running pg-diff
on this small example.
In my development database i dropped some tables that also exist on the "production" database. Then when running "pg-diff -c development patch_name" it gave empty file. I guess it doesn't detect deleted tables.
Is there a possibility to have this kind of feature?
Version: 1.2.8
PG Version: 11.2
Plugins: PostGIS is installed
When comparing two databases and PostGIS is installed, this seems to blow up with the following error:
error: "st_extent" is an aggregate function
at Connection.parseE (/Users/ashish/.nvm/versions/node/v10.15.1/lib/node_modules/pg-diff-cli/node_modules/pg/lib/connection.js:602:11)
at Connection.parseMessage (/Users/ashish/.nvm/versions/node/v10.15.1/lib/node_modules/pg-diff-cli/node_modules/pg/lib/connection.js:399:19)
at Socket.<anonymous> (/Users/ashish/.nvm/versions/node/v10.15.1/lib/node_modules/pg-diff-cli/node_modules/pg/lib/connection.js:121:22)
at Socket.emit (events.js:189:13)
at addChunk (_stream_readable.js:284:12)
at readableAddChunk (_stream_readable.js:265:11)
at Socket.Readable.push (_stream_readable.js:220:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
Could you ignore the extensions? for example the PGCRYPTO
Dear @michaelsogos with PostgresSQL 12.1 the ad.adsrc is deprecated.
I suggest you to use pg_get_expr(ad.adbin,ad.adrelid) instead of it.
Have a nice day.
pg-diff-cli: 1.2.3
In my development database i had created a table with some columns set as nullable, added data and everything worked as it should.
Later i decided to change some of the fields which had a default value of NULL to their appropriate data type, but as soon as i run "pg-diff -c development patch_name" it gave empty patch file. No changes were detected.
For fields that were not NULL from the same table but got altered, the "pg-diff -c development patch_name" worked properly.
Data compare option in pg-diff-config.json is enabled and set properly.
Replication steps:
Expected: Successful
Actual: Failure
Looks like you should be checking for null, not a falsy value.
WRONG PLACE -- sorry
I've followed the steps in https://michaelsogos.github.io/pg-diff/ but get the following error:
$ pg-diff -h
Error: : No such file or directory
$ pg-diff -c development initial-script
Error: : No such file or directory
Executed from pg-diff-master directory
I've installed node.js successfully. Node version v6.17.1. I don't believe this is an issue with Node.js because I can run run node -v, npm -v successfully and write a message back to the terminal window successfully.
node is set in PATH - /usr/bin/node
How do I resolve this issue?
Some feature requests:
This bug is happening when there is a sequence orphan .
VERSION
pg-diff-cli: 1.2.3
Repro steps
Created new sequence using postgres:
CREATE SEQUENCE voucher_code_seq START 1
Run:
pg-diff -c development sequence_name_script
pg-diff -m development
Observed behavior
Throws error when executing migration for the next line after the statement for creating sequence
"error syntax near alter"
PostgreSQL Version: 12.2 (I upgraded both sides to make sure that the issue wasn't mismatched versions)
Example:
The diff gives me this function:
CREATE OR REPLACE FUNCTION public.get_browser_from_user_agent(user_agent text)
RETURNS text
LANGUAGE sql
PARALLEL SAFE
AS $function$
SELECT CASE
WHEN user_agent LIKE '%Firefox/%' THEN 'Firefox'
WHEN user_agent LIKE '%Chrome/%' OR user_agent LIKE '%CriOS%' THEN 'Chrome'
WHEN user_agent LIKE '%MSIE %' THEN 'IE'
WHEN user_agent LIKE '%MSIE+%' THEN 'IE'
WHEN user_agent LIKE '%Trident%' THEN 'IE'
WHEN user_agent LIKE '%iPhone%' THEN 'iPhone Safari'
WHEN user_agent LIKE '%iPad%' THEN 'iPad Safari'
WHEN user_agent LIKE '%Opera%' THEN 'Opera'
WHEN user_agent LIKE '%BlackBerry%' AND user_agent LIKE '%Version/%' THEN 'BlackBerry WebKit'
WHEN user_agent LIKE '%BlackBerry%' THEN 'BlackBerry'
WHEN user_agent LIKE '%Android%' THEN 'Android'
WHEN user_agent LIKE '%Safari%' THEN 'Safari'
WHEN user_agent LIKE '%bot%' THEN 'Bot'
WHEN user_agent LIKE '%http://%' THEN 'Bot'
WHEN user_agent LIKE '%www.%' THEN 'Bot'
WHEN user_agent LIKE '%Wget%' THEN 'Bot'
WHEN user_agent LIKE '%curl%' THEN 'Bot'
WHEN user_agent LIKE '%urllib%' THEN 'Bot'
ELSE 'Unknown'
END
$function$
;
ALTER FUNCTION "public"."get_browser_from_user_agent"(text) OWNER TO ----------;
After I run it, it still shows up in the diff exactly as if it was never ran.
Hi!
I think the generated types are ignored for source and target comparison. I tested with two databases where there are several compound types in the source code, but they didn't make it into the resulting script. CREATE TYPE ...
This issue happens on macOS after yarn add
when running yarn pg-diff -c development migration
:
env: node\r: No such file or directory
error Command failed with exit code 127.
It can be fixed when I go to node_modules/.bin/pg-diff and I manually change the line ending (I use vscode but there are scripts for that) from CRLF to LF.
It is mandatory to do this, otherwise I cannot use your program.
SELECT MD5(ROW(${config.keyFields.join(',')})::text) AS "rowHash", * FROM ${config.schema||'public'}.${table}
Is it possible to compare the DB1.schemaA with DB2.schemaB?
In other words if two schemas on two different databases have different names but are supposed to contains the same tables, etc. is it possible to perform the comparison? How to configure the mapping in this scenario?
After I finished creating my own pg-diff-config.json file, I called
'pg-diff -c development initial-script'
which began connecting to databases, however, at 25% I get this error:
error: column a.attidentity does not exist
attidentity seems to be a postgres column, not any of the columns that are in the tables in my two databases. Any ideas how to fix the error? I've already double checked to make sure that the username I'm inputting has admin rights to access the database.
Hi!
Is it possible to disable lines with comments of functions, indexes, constraints, etc.
Is it possible to specify this in the config file?
I have a database with an aggregate function, can you either support Aggregate functions or remove them from being analyzed in the diff by using AND proisagg is false
in CatalogApi.js getFunctions
Getting this error when running "Execute Shell" command with pg-diff:
TypeError: process.stdout.clearLine is not a function
at EventEmitter. (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/main.js:74:21)
at EventEmitter.emit (events.js:200:13)
at Function.compare (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:19:16)
at PgDiff.compare (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/index.js:36:27)
at Run (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/main.js:78:39)
at Object. (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/main.js:14:1)
at Module._compile (internal/modules/cjs/loader.js:776:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:643:32)
at Function.Module._load (internal/modules/cjs/loader.js:556:12)
For example: If I have a quote ' in a field, in the migration file there isn't the escape character ('').
MBE
On the face of it, this seems similar to #24. The last status output was "AGGREGATE objects have been compared. at 75%
At this point though, as it's not telling me the table so I've not yet understood which property or function it has an issue with (there's quite a few referencing nodeid).
#24 says this is fixed in 2.0.2, I'm using...
Version: 2.3.1
PostgreSQL: 9.6+
Full error message is...
error: column "nodeid" does not exist
at Parser.parseErrorMessage (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\parser.js:287:98)
at Parser.handlePacket (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\parser.js:126:29)
at Parser.parse (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\parser.js:39:38)
at Socket. (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\index.js:11:42)
at Socket.emit (events.js:315:20)
at addChunk (_stream_readable.js:295:12)
at readableAddChunk (_stream_readable.js:271:9)
at Socket.Readable.push (_stream_readable.js:212:10)
at TCP.onStreamRead (internal/stream_base_commons.js:186:23)
column "nodeid" does not exist
Anyone know what to look for or how to find out which table its referring to?
Thank you.
The regrole type appears to have been introduced in PG 9.5+
In retrieveSchema.js this query return `SELECT nspname, nspowner::regrole::name as owner FROM pg_namespace WHERE nspname IN ('${schemas.join("','")}')
fails on PG <= 9.4
I note there is a TODO to not use regrole casting. Maybe this would fix the issue?
It would be nice to have an option to remove IF EXISTS
(and similar) statements from the generated migrations scripts. These statements are dangerous in the context of schema migrations, as they can hide serious issues.
For instance, ALTER TABLE IF EXISTS ...
would silently fail if the table does not exist, and one could think the migration did correctly apply, while it failed because of a major issue (a missing table !!). Much better to have the script fail with a clear error message saying the table is missing.
Thanks for the otherwise awesome tool !
So I'm having a problem with this. While migrating data from Azure postgres to AWS RDS, I've discovered both services now require SSL enabled. I dont seem to have a way to tell the script to do that and it returns complaining about the lack of SSL in our connection.
I've worked around it by temporarily disabling SSL on both sides (Easy on Azure, and in typical fashion a little trickier in AWS [create a custom parameter group, set rds.ssl_enable [or whatever its called again] to 0 , then apply it, then reboot]) but its a pretty shady way of doing things. Thankfully I've been able to re-enable afterwards to correct the temporary lowering of security, but its the kind of move that'd give most security consultants an aneurism.
Any chance of letting us set those settings in the config? I figure this'll be a fairly common problem.
[||||||-------------] 33% - Collecting PRIVILEGES for table "public"."schedulegeneratorschedulegeneratorscollection_heal[||||||-------------] 33% - Collecting PRIVILEGES for table "public"."agroinsuranceaccident"
error: there are multiple default operator classes for data type text
error: there are multiple default operator classes for data type text
at Connection.parseE (C:\Users\bkvariani\AppData\Roaming\npm\node_modules\pg-diff-cli\node_modules\pg\lib\connection.js:614:13)
at Connection.parseMessage (C:\Users\bkvariani\AppData\Roaming\npm\node_modules\pg-diff-cli\node_modules\pg\lib\connection.js:413:19)
at Socket. (C:\Users\bkvariani\AppData\Roaming\npm\node_modules\pg-diff-cli\node_modules\pg\lib\connection.js:129:22)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:224:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
It would be really great if there was a command line option to generate a dummy config file. Or better still take 2 connection strings and populate the config file too
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.