Giter VIP home page Giter VIP logo

pgcompacttable's Introduction

pgcompacttable is a tool for reducing size of bloated tables and indexes without heavy locks.

It is designed to reorganize data in tables and rebuild indexes in order to revert back disk space without database performance impact.

Setup

pgcompacttable is written in Perl and requires Perl DBI library, obviously with PostgreSQL support module. Dependencies can be easy installed:

  • on Debian-based Linux OS with apt-get install libdbi-perl libdbd-pg-perl
  • RedHat/Centos with yum install perl-Time-HiRes perl-DBI perl-DBD-Pg

In target database contrib module pgstattuple should be installed via create extension if not exists pgstattuple;

Run

pgcompacttable can be run from any OS user (and even on another host, see --host option; although it is recommended to run it on same host with target database), but PostgreSQL Superuser access is required. Preferred way is to run as PostgreSQL cluster owner, usually postgres. In this case pgcompacttable can perform ionice -c 3 for PostgreSQL backend pid to lower IO priority.

pgcompacttable --man Prints list of available options and usage information.

pgcompacttable --all --verbose Compacts all bloated tables in all databases in the cluster, including indexes. Prints additional progress information.

pgcompacttable --dbname billing --exclude-schema pgq Compacts all bloated tables in the billing database and their bloated indexes excepts ones that are in the pgq schema.

pgcompacttable --dbname billing -t operations -f Force compact table operations in database billing. Notice, tables and indexes with bloat less than 20% (hardcoded MINIMAL_COMPACT_PERCENT constant) are considered normal and not processed until option --force is taken.

Compatibility

pgcompacttable currently supports PostgreSQL starting from 9.2. Rebuilds partial indexes, functional, unique, used for foreign keys and indexes already placed on another tablespace are supported.

What about pg_repack?

Unlike another popular tool pg_repack this tool has some advantages:

  • does not requires lots of free space
    • tables are processed in-place
    • indexes are rebuild one by one, from smallest to largest therefore maximum space required is the size of the largest index
  • tables are processed with adaptive delays to prevent heavy IO and replication lag spikes (see --delay-ratio option)

pgcompacttable's People

Contributors

alexius2 avatar aorashi avatar maximboguk avatar melkij avatar nikolays avatar oleg-st avatar rvoronin avatar webervin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pgcompacttable's Issues

Latest changes broke several databases on different servers

I used pgcompacttable on Windows with PSQL versions 9.4 and 11 for a long time. Everything went smooth with version from 2020-09-01
Yesterday I tried updated version (2021-09-10) and got few broken databases on different servers.
ERROR: could not open relation with oid
I have log from only one server. At first glance it doesn't have anything interesting, no errors. On a second server log had few "SQL errors" which never occurred before.

Please revert latest changes.
Unfortunately I'm unable to help with further testing and most likely would just use previous version.
Thanks.

Configurable `MAX_PAGES_PER_ROUND`

I changed MAX_PAGES_PER_ROUND in the code to 50, to speed things up, and didn't notice any downsides.
Is there any reason to hardcode it to 5 in the first place and not make is configurable through, let's say command line parameter?

RDS support

Do you know if it is feasible in any way to use pgcompacttable in AWS RDS-managed PostgreSQL?

--exclude-table not working

./pgcompacttable -h localhost -U postgres -d .... --exclude-table products -v -s

.....
[Thu Oct  1 13:19:02 2020] (....:public.products) Start handling table public.products
[Thu Oct  1 13:19:40 2020] (....:public.products) Vacuum initial: 4229952 pages left, duration 37.336 seconds.
[Thu Oct  1 13:23:38 2020] (....:public.products) Bloat statistics with pgstattuple: duration 237.898 seconds.
[Thu Oct  1 13:23:38 2020] (....:public.products) Statistics: 4229952 pages (5263584 pages including toasts and indexes), it is expected that ~87.200% (3688486 pages) can be compacted with the estimated space saving being 28.141GB.
[Thu Oct  1 13:23:38 2020] (....:public.products) Update by column: free_delivery.
......

creating invalid index every run.

If table contains duplicate data with an existing constraint (invalid index), then pgcompacttable will create a new invalid index every time it is run.

                 Table "public.t2"
 Column |  Type   | Collation | Nullable | Default
--------+---------+-----------+----------+---------
 id     | integer |           | not null |
 num    | integer |           | not null |
Indexes:
    "t2_con" PRIMARY KEY, btree (id, num)

select * from t2;
 id | num
----+-----
  1 |   2
  1 |   3
(2 rows)

then disabling index, insert some duplicate data and reenable index

select * from t2;
 id | num
----+-----
  1 |   2
  1 |   3
  1 |   2
(3 rows)

insert into t2 values ( '1', '2' );
ERROR:  duplicate key value violates unique constraint "t2_con"
DETAIL:  Key (id, num)=(1, 2) already exists.

reindex table t2;
ERROR:  could not create unique index "t2_con"
DETAIL:  Key (id, num)=(1, 2) is duplicated.

then start pgcompacttable and got error in log, but pgcompacttable didnt fail or exited, it continued working:

[Fri Feb 10 10:41:20 2023] (bloat:public.t2) SQL Error: ERROR:  could not create unique index "t2_con_ccnew"

next run:

[Fri Feb 10 10:41:23 2023] (bloat:public.t2) SQL Error: ERROR:  could not create unique index "t2_con_ccnew1"

and so on...

final view of table after 4 runs:

 Table "public.t2"
 Column |  Type   | Collation | Nullable | Default
--------+---------+-----------+----------+---------
 id     | integer |           | not null |
 num    | integer |           | not null |
Indexes:
    "t2_con" PRIMARY KEY, btree (id, num)
    "t2_con_ccnew" UNIQUE, btree (id, num) INVALID
    "t2_con_ccnew1" UNIQUE, btree (id, num) INVALID
    "t2_con_ccnew2" UNIQUE, btree (id, num) INVALID
    "t2_con_ccnew3" UNIQUE, btree (id, num) INVALID

So, why pgcompacttable does not remove invalid indexes after itself?

Is there a way to use another drive for processing tables/indexes?

For example:
I have one drive with database (full at 99%) and another drive (almost free).

When I run pgcompacttable, I get messages like:

Skipping processing: 10.38% space to compact from 20% minimum required.

Is there a way to use another (free) drive for processing tables/indexes?

Compression table does not occur

Hi,

I run compression procedure is started with the command:

[root@hostname]# perl /tmp/pgcompacttable-master/bin/pgcompacttable -U postgres -d database -t table
Thu Oct 29 10:37:55 2015 Connecting to database
Thu Oct 29 10:37:55 2015 Postgress backend pid: 25779
Thu Oct 29 10:37:56 2015 Handling tables. Attempt 1
Thu Oct 29 14:14:51 2015 Statistics: 48059264 pages (65754320 pages including toasts and indexes) , approximately 19.240% (9249084 pages) can be compacted reducing the size by 70.565GB.
Thu Oct 29 14:21:22 2015 Reindex queries: public.Таблицаtable_pkey, initial size 2882369 pages (21.991GB), will be reduced by 51% (11.328GB)
Thu Oct 29 14:21:22 2015 CREATE UNIQUE INDEX CONCURRENTLY pgcompact_index_25771 ON table USING btree (id); --database
Thu Oct 29 14:21:22 2015 BEGIN; SET LOCAL statement_timeout TO 1000;
ALTER TABLE "public"."table" DROP CONSTRAINT "Таблицаtable_pkey";
ALTER TABLE "public"."table" ADD CONSTRAINT "Таблицаtable_pkey" PRIMARY KEY USING INDEX pgcompact_index_25771;
END;; --database
Thu Oct 29 14:24:08 2015 Processing complete.
Thu Oct 29 14:24:08 2015 Processing results: size reduced by 256.000KB (-130.172MB including toasts and indexes) in total.
Thu Oct 29 14:24:08 2015 Disconnecting from database
[Thu Oct 29 14:24:08 2015] Processing complete: 1 retries to process has been done
[Thu Oct 29 14:24:08 2015] Processing results: size reduced by 256.000KB (-130.172MB including toasts and indexes) in total, 256.000KB (-130.172MB) database.

Why not start the procedures for data compression and not run for re-indexing procedure ?

Thank you.

Question: How many extra space do I need when I run this tool?

I want to run this tool against a table on production database. I would like to know how many extra space I need when I run it.

I saw in the read me section, the space I need is the size of the largest index. I want to confirm couple things:

  • If I skip reindex, does that mean I don't need any extra space?
  • If I only compact table and skip reindex, will it still free my disk space from the bloated table?
  • The tool first compact table, then reindex. So given this scenario: my table has a waste of 100GB, and my largest index is 20GB. Does that mean even if I dont have 20GB free space in the beginning, ideally after table is compacted and before reindex occurs, I should have 100GB extra disk space, and I can safely let the tool run reindex?

Thanks in advance!

Error "relation "public.pg_toast_5987783_index" does not exist" when running the utility

Hi!
I successfully run pgcompacttable on different tables separately:

postgres@database:/home/dimon/pgcompacttable/bin$ ./pgcompacttable -d moyklass -t user_subscriptions
[Mon May 16 23:13:52 2022] (moyklass) Connecting to database
[Mon May 16 23:13:52 2022] (moyklass) Postgres backend pid: 100889
[Mon May 16 23:13:52 2022] (moyklass) Handling tables. Attempt 1
[Mon May 16 23:13:58 2022] (moyklass:public.user_subscriptions) Statistics: 88846 pages (114069 pages including toasts and indexes), it is expected that ~13.670% (12149 pages) can be compacted with the estimated space saving being 94.919MB.
[Mon May 16 23:14:02 2022] (moyklass:public.user_subscriptions) Reindex: public.user_subscriptions_subscription_id_idx, initial size 4099 pages(32.023MB), has been reduced by 33% (10.633MB), duration 3 seconds.
[Mon May 16 23:14:02 2022] (moyklass:public.user_subscriptions) Processing results: 88846 pages left (112708 pages including toasts and indexes), size reduced by 0.000B (10.633MB including toasts and indexes) in total.
[Mon May 16 23:14:02 2022] (moyklass) Processing complete.
[Mon May 16 23:14:02 2022] (moyklass) Processing results: size reduced by 0.000B (10.633MB including toasts and indexes) in total.
[Mon May 16 23:14:02 2022] (moyklass) Disconnecting from database
[Mon May 16 23:14:02 2022] Processing complete: 1 retries to process has been done
[Mon May 16 23:14:02 2022] Processing results: size reduced by 0.000B (10.633MB including toasts and indexes) in total.

After that I run the utility on full database:

./pgcompacttable -d moyklass

Some tables where optimized successfully, but after some time an error "relation "public.pg_toast_5987783_index" does not exist" started appearing for all tables. Relation pg_toast_5987783_index is the same for all tables.
This is the first table the error occurs (table name is classes, maybe it is important):

[Mon May 16 23:34:59 2022] (moyklass:public.classes) Statistics: 36086 pages (38337 pages including toasts and indexes), it is expected that ~21.820% (7873 pages) can be compacted with the estimated space saving being 61.513MB.
[Mon May 16 23:35:27 2022] (moyklass:public.classes) SQL Error: ERROR:  relation "public.pg_toast_5987783_index" does not exist
[Mon May 16 23:35:27 2022] (moyklass:public.classes) Table handling interrupt.
[Mon May 16 23:35:27 2022] (moyklass:public.classes) Processing results: 36086 pages (38437 pages including toasts and indexes), size has been reduced by 0.000B (-816.000KB including toasts and indexes) in total. This attempt has been initially expected to compact ~21% more space (7871 pages, 61.498MB)

Now this error appears in any utility run, ex:

postgres@database:/home/dimon/pgcompacttable/bin$ ./pgcompacttable -d moyklass -t user_subscriptions
[Mon May 16 23:51:57 2022] (moyklass) Connecting to database
[Mon May 16 23:51:57 2022] (moyklass) Postgres backend pid: 13168
[Mon May 16 23:51:57 2022] (moyklass) Handling tables. Attempt 1
[Mon May 16 23:51:58 2022] (moyklass:public.user_subscriptions) Statistics: 88846 pages (112709 pages including toasts and indexes), it is expected that ~13.670% (12147 pages) can be compacted with the estimated space saving being 94.902MB.
[Mon May 16 23:51:58 2022] (moyklass:public.user_subscriptions) SQL Error: ERROR:  relation "public.pg_toast_5987783_index" does not exist
[Mon May 16 23:51:58 2022] (moyklass:public.user_subscriptions) Table handling interrupt.
[Mon May 16 23:51:58 2022] (moyklass:public.user_subscriptions) Processing results: 88846 pages left (112709 pages including toasts and indexes), size reduced by 0.000B (0.000B including toasts and indexes) in total.
[Mon May 16 23:51:58 2022] (moyklass) Processing complete.
[Mon May 16 23:51:58 2022] (moyklass) Processing results: size reduced by 0.000B (0.000B including toasts and indexes) in total.
[Mon May 16 23:51:58 2022] (moyklass) Disconnecting from database
[Mon May 16 23:51:58 2022] Processing complete: 1 retries to process has been done
[Mon May 16 23:51:58 2022] Processing results: size reduced by 0.000B (0.000B including toasts and indexes) in total.

The error occurs in this query:

   SELECT
    indexname, tablespace, indexdef,
    regexp_replace(indexdef, E'.* USING (\\w+) .*', E'\\1') AS indmethod,
    conname,
    CASE
        WHEN contype = 'p' THEN 'PRIMARY KEY'
        WHEN contype = 'u' THEN 'UNIQUE'
        ELSE NULL END AS contypedef,
    (
        SELECT
            bool_and(
                deptype IN ('n', 'a', 'i') AND
                NOT (refobjid = indexoid AND deptype = 'n') AND
                NOT (
                    objid = indexoid AND deptype = 'i'
                ))
        FROM pg_catalog.pg_depend
        LEFT JOIN pg_catalog.pg_constraint ON
            pg_catalog.pg_constraint.oid = refobjid
        WHERE
            (objid = indexoid AND classid = pgclassid) OR
            (refobjid = indexoid AND refclassid = pgclassid)
    )::integer AS replace_index_possible,
    (
        SELECT string_to_array(indkey::text, ' ')::int2[] operator(pg_catalog.@>) array[0::int2]
        FROM pg_catalog.pg_index
        WHERE indexrelid = indexoid
    )::integer as is_functional,
    condeferrable as is_deferrable,
    condeferred as is_deferred,
    (contype = 'x') as is_exclude_constraint,
    pg_catalog.pg_relation_size(indexoid) as idxsize
FROM (
    SELECT
        indexname, COALESCE(tablespace, (SELECT spcname AS tablespace FROM pg_catalog.pg_tablespace WHERE oid = (SELECT dattablespace
            FROM pg_catalog.pg_database
            WHERE 
                datname = current_database() AND
                spcname != current_setting('default_tablespace')))) AS tablespace, indexdef,
        (
            quote_ident(schemaname) || '.' ||
            quote_ident(indexname))::regclass AS indexoid,
        'pg_catalog.pg_class'::regclass AS pgclassid
    FROM pg_catalog.pg_indexes
    WHERE
        schemaname = 'public' AND
        tablename = 'user_subscriptions'
) AS sq
LEFT JOIN pg_catalog.pg_constraint ON
    conindid = indexoid AND contype IN ('p', 'u', 'x')
ORDER BY idxsize

It is interesting that if I change LEFT JOIN to JOIN, query executes without any error. Also if I remove condition schemaname = 'public', query executes without error too. Also if I change contype IN ('p', 'u', 'x') to contype IN ('p', 'u'), query works too!

I don't understand why this error happens, I didn't find any relation or index with the name pg_toast_5987783_index. But in fact that this has been started during pgcompacttable run and probably the utility made something wrong in database.

I've restored the dump made after this error happened on other server, and this query runs without errors on restored database. I dont know how to fix this in production database.

Try to install pgstattuple

Hi there,

First of all, thanks for this tool. pg_repack was causing a real bad time on some DBs and this works like charm.

I'll have a small suggestion. Can't the binary try to run create extension pgstattuple; on the DB it's running (preferably via a parameter)? It can still fail with same message if it's not available, but that would make it a pinch easier to use.

Thanks again & have a nice day 👋

Progress more than 100% ??

Very interesting experience, why can it be?

[Fri Aug 18 12:00:23 2017] (somedb:public.sometable) Vacuum initial: 2394828 pages left, duration 201.816 seconds.
[Fri Aug 18 12:03:12 2017] (somedb:public.sometable) Bloat statistics with pgstattuple: duration 169.006 seconds.
[Fri Aug 18 12:03:12 2017] (somedb:public.sometable) Statistics: 2394828 pages (3944168 pages including toasts and indexes) , approximately 48.560% (1162809 pages) can be compacted reducing the size by 8.872GB.
[Fri Aug 18 12:03:12 2017] (somedb:public.sometable) Update by column: ns.
[Fri Aug 18 12:03:12 2017] (somedb:public.sometable) Set pages/round: 5.
[Fri Aug 18 12:03:12 2017] (somedb:public.sometable) Set pages/vacuum: 47897.
[Fri Aug 18 12:04:12 2017] (somedb:public.sometable) Progress: 1%, 13485 pages completed.
[Fri Aug 18 12:05:13 2017] (somedb:public.sometable) Progress: 2%, 26365 pages completed.
[Fri Aug 18 12:06:13 2017] (somedb:public.sometable) Progress: 3%, 39265 pages completed.
[Fri Aug 18 12:07:13 2017] (somedb:public.sometable) Progress: 4%, 51825 pages completed.
[Fri Aug 18 12:08:13 2017] (somedb:public.sometable) Progress: 5%, 64265 pages completed.
...
[Fri Aug 18 13:24:22 2017] (somedb:public.sometable) Progress: 93%, 1084660 pages completed.
[Fri Aug 18 13:25:22 2017] (somedb:public.sometable) Progress: 96%, 1120520 pages completed.
[Fri Aug 18 13:26:22 2017] (somedb:public.sometable) Progress: 99%, 1154950 pages completed.
[Fri Aug 18 13:27:22 2017] (somedb:public.sometable) Progress: 102%, 1188925 pages completed.
[Fri Aug 18 13:28:22 2017] (somedb:public.sometable) Progress: 105%, 1223730 pages completed.

Feature request: Skip handling tables

Please add option to skip handling tables. Because it much less effective than handling indexes.
It takes too much time and it fails almost always.

option "--routine-vacuum"

Hi.
Can you explain more detailed option:
--routine-vacuum
Turn on the routine vacuum. By default all the vacuums are off.

I did not find any ALTER about disabling VACUUM on tables in code
Thanks

SQL Error: ОШИБКА: нет прав для изменения параметра "session_replication_role"

Hello, Maxim!
There is problem (in subject) that I can't understand.
Also I can't find default mechanism of authentication without password (under 'postgres' user)

Here is listing:

root@someserver:~# perl pgcompacttable.pl -U someuser -W somepass -d somedb -t history -v
[Fri Aug 18 09:39:02 2017] (somedb) Connecting to database
[Fri Aug 18 09:39:02 2017] (somedb) Postgress backend pid: 8163
Wide character in print at pgcompacttable.pl line 187.
[Fri Aug 18 09:39:02 2017] (somedb) SQL Error: ОШИБКА: нет прав для изменения параметра "session_replication_role"
[Fri Aug 18 09:39:02 2017] (somedb) Database handling interrupt.
[Fri Aug 18 09:39:02 2017] (somedb) Disconnecting from database
[Fri Aug 18 09:39:02 2017] Processing incomplete: 1 databases left.

Best regards,
Vladimir

Exclude db doesn't work ?

pgcompacttable -a -D test -U postgres

Unknown option: D
Usage:
pgcompacttable [OPTION...]

General options:
    [-?mV] [(-q | -v LEVEL)]

Connection options:
    [-h HOST] [-p PORT] [-U USER] [-W PASSWD] [-P PATH]

Targeting options:
    (-a | -d DBNAME...) [-n SCHEMA...] [-t TABLE...] [-D DBNAME...] [-N
    SCHEMA...] [-T TABLE...]

pgcompacttable -a --exclude-dbname test -U postgres
Unknown option: exclude-dbname
Usage:
pgcompacttable [OPTION...]

General options:
    [-?mV] [(-q | -v LEVEL)]

Connection options:
    [-h HOST] [-p PORT] [-U USER] [-W PASSWD] [-P PATH]

Targeting options:
    (-a | -d DBNAME...) [-n SCHEMA...] [-t TABLE...] [-D DBNAME...] [-N
    SCHEMA...] [-T TABLE...]

Use of uninitialized value in subtraction

I get a lot of errors at startup
Use of uninitialized value in subtraction (-) at ./pgcompacttable line 1764.

./pgcompacttable --all --verbose -d database
Centos 7
Postgres 12.3

SQL Error: ERROR: operator is not unique: smallint[] @> smallint[]

Hello, I'm getting this issue while trying to compact my table:

[Tue Feb 27 11:17:51 2018] (mydb:public.call_project_cases) SQL Error: ERROR: operator is not unique: smallint[] @> smallint[]
LINE 26: ...SELECT string_to_array(indkey::text, ' ')::int2[] @> array[0...
^
HINT: Could not choose a best candidate operator. You might need to add explicit type casts.
[Tue Feb 27 11:17:51 2018] (mydb:public.call_project_cases) Table handling interrupt.

Is there anything I can do to get rid of it?

postgresql-9.6 9.6.7-1.pgdg16.04+1 amd64
Ubuntu 16.04.3 LTS

recreating constraints missing deferrable

We have noticed that the script is not respecting all settings of original constraints.

In our case we use DEFERRABLE with a constraint after using the script deferrable was gone.

ALTER TABLE ONLY lot
-ADD CONSTRAINT ukey_lot UNIQUE (street_id, city_id, zip_id, housenumber, block, staircase) DEFERRABLE;
+ADD CONSTRAINT ukey_lot UNIQUE (street_id, city_id, zip_id, housenumber, block, staircase);

In our case we actually did not need deferrable but the script should respect all original settings

Huge deadlocks count on the last phase operations ALTER INDEX RENAME TO

Hello, Maxim!

Thank you for very useful tool. We have deadlock errors in postgresql logs related to last phase of pgcompacttable - "REINDEX".
For example:

2020-10-31 21:05:37 MSK [21303] 172.20.2.144 PostgreSQL JDBC Driver queue2@queue2 40P01 UPDATE ERROR: deadlock detected
2020-10-31 21:05:37 MSK [21303] 172.20.2.144 PostgreSQL JDBC Driver queue2@queue2 40P01 UPDATE DETAIL: Process 21303 waits for RowExclusiveLock on relation 38747785 of database 16619; blocked by process 20639.
Process 20639 waits for AccessExclusiveLock on relation 39109132 of database 16619; blocked by process 21303.
Process 21303: UPDATE public.queue_message2 as x SET sended_at = now(), state='SENDED'
WHERE id = (
SELECT id
FROM public.queue_message2
WHERE queue_id = $1
AND state = 'QUEUED'
AND router_tag = $2
AND expired_at > now()
ORDER BY queue_id, state, router_tag,
priority DESC, queued_at ASC FOR UPDATE SKIP LOCKED LIMIT 1)
RETURNING x.*
Process 20639:
ALTER INDEX "public".pgcompact_index_20528 RENAME TO "pgcompact_index_14090";

Can there be a positive effect from using "Options controlling the behaviour" like --reindex-*** or any else?
May be --print-reindex-queries will help in such situation to perform manual reindex later?

Problem arising only on huge DML-loaded database shardes.

huge index need more space

Would create a temporal index into a temporal tablespace, on a separated filesystem. When finished, remove temporal tablespace and recovery space.

Question about --reindex-* parameters

Hello,

Can you please clarify the following:

--reindex-replace
    Avoid using REINDEX INDEX CONCURRENTLY even when it is available. By
    default this native PostgreSQL feature is preferred.

--reindex-retry-count
    Attempts count to concurrently safe replace bloated index to new.
    Default 100

--reindex-retry-pause
    Pause between reindex attempts in seconds. Default is 1 second

--reindex-lock-timeout
    Statement timeout for reindex ALTER TABLE queries. Default is 1000
    (ms)

In my understanding if parameter --reindex-replace passed to your tool then only in this case there is a sense of defining the rest of --reindex-* options. Because in this case pgcompacttable will use not use native Postgres feature for index rebuild REINDEX INDEX CONCURRENTLY , but will perform steps like below:

  1. Create new index concurrently;
  2. Try to replace old index with new one (this is the place where --reindex-* can help to control this)
  3. Drop old index.

If --reindex-replace is not passed to pgcompacttable then native Postgres feature for index rebuild REINDEX INDEX CONCURRENTLY will be used and setting additinal --reindex-* doesn't make any sense.

Am I right?

Thank you.

Question : Can I stop it ?

Hello,

I want to run this tool on a very big table (~600G to retreive). We test it on a preproduction DB when activity occurs on the table and the process took 52h and generated an overcost of 25% of DB load-average.

I'm quite confident to run it in production, but I have a last question : can I stop safely the process if something goes wrong on production (too high latency for users, multiplication of locks, anything else...) ? If the process is killed, can we loose some datas ?

I will (obviously) backup my DB before running this tool, but I prefer to have all informations before going to war :).

Regards,

set session_replication_role to replica; issue

since we use Postgresql as a service on azure, there is not permission to execute 'set session_replication_role to replica;';

after we disabled "set session_replication_role to replica;" in pgcompacttable the tool started running, however we dont know what is the impact to run tool with disabled "set session_replication_role to replica;", can you please advice on that?

thanks upfront

Question: why do reindex?

A full vacuum does a rewrite of all the table and indexes, so why have REINDEX options?

btw, I love your verbose output!

Question about --routine-vacuum parameter

Hello,

I'm still testing your tool to beat some bloat in my database on test lab. There is a critical table in my db and autovacuum is running every ~ 20-30 min. If autovacuum is not running for a long time => queries (customers) to that table experiencing a problems.

From pgcompacttable man:

-R
--routine-vacuum
    Turn on the routine vacuum. By default all the vacuums are off.

Based on above statement I conclude that during default pgcompacttable run autovacuum will not be able to run for that table.

If I will pass additional parameter -R this allow autovacuum to be run during pgcompacttable work, right? I can't check it on my test lab, that why I want to get some confirmation that I understood it correctly.

Am I right? If yes - then how you actually "blocking" autovacuum run during pgcompacttable run? I thought you are temporary setting autovacuum_enabled = off, but looks like it's not. Can you share this trick?

Thank you.

`session_replication_role` lost on reconnect

Hi,

Tried to compact some big table; after a few days the process was interrupted with:

[Thu Mar 21 12:03:52 2024] (my-db:my.table) Progress: 66%,  22145360 pages completed.
[Thu Mar 21 12:03:52 2024] (my-db:my.table) Connecting to database
[Thu Mar 21 12:03:52 2024] (my-db:my.table) SQL Error: ERROR:  The session_replication_role must be set to replica.

Should the role be set within db_connect?

This attempt has been initially expected to compact ~24% more space

Why is my table not getting compacted? :)

Time and again I get the following results:
Processing results: 30430 pages (66700 pages including toasts and indexes), size has been reduced by 0.000B (0.000B including toasts and indexes) in total. This attempt has been initially expected to compact ~24% more space (7444 pages, 58.161MB)

Can not process the big table.

Command: ./pgcompacttable -h localhost -U postgres -d .... -v -s --reindex-lock-timeout 2000 -t products

How can I fix it?
Sorry about the big log.

Mon Nov 16 01:20:01 MSK 2020
[Mon Nov 16 01:20:01 2020] (ww_de_furniture) Connecting to database
[Mon Nov 16 01:20:01 2020] (ww_de_furniture) Postgres backend pid: 11539
[Mon Nov 16 01:20:01 2020] (ww_de_furniture) It is recommended to set ionice -c 3 for pgcompacttable: ionice -c 3 -p 11539
[Mon Nov 16 01:20:01 2020] (ww_de_furniture) Handling tables. Attempt 1
[Mon Nov 16 01:20:01 2020] (ww_de_furniture:public.products) Start handling table public.products
[Mon Nov 16 01:20:01 2020] (ww_de_furniture:public.products) Skipping processing: another instance is working with table public.products
[Mon Nov 16 01:20:01 2020] (ww_de_furniture:public.products) Finish handling table public.products
[Mon Nov 16 01:20:01 2020] (ww_de_furniture) Processing complete.
[Mon Nov 16 01:20:01 2020] (ww_de_furniture) Processing results: size reduced by 0.000B (0.000B including toasts and indexes) in total.
[Mon Nov 16 01:20:01 2020] (ww_de_furniture) Disconnecting from database
[Mon Nov 16 01:20:01 2020] Processing complete: 1 retries to process has been done
[Mon Nov 16 01:20:01 2020] Processing results: size reduced by 0.000B (0.000B including toasts and indexes) in total.
[Mon Nov 16 01:20:32 2020] (ww_de_furniture:public.products) Progress: 66%,  893550 pages completed.
[Mon Nov 16 01:21:33 2020] (ww_de_furniture:public.products) Progress: 66%,  894140 pages completed.
[Mon Nov 16 01:22:33 2020] (ww_de_furniture:public.products) Progress: 66%,  894670 pages completed.
[Mon Nov 16 01:23:33 2020] (ww_de_furniture:public.products) Progress: 66%,  895460 pages completed.
[Mon Nov 16 01:24:33 2020] (ww_de_furniture:public.products) Progress: 66%,  896190 pages completed.
[Mon Nov 16 01:25:34 2020] (ww_de_furniture:public.products) Progress: 66%,  896960 pages completed.
[Mon Nov 16 01:26:34 2020] (ww_de_furniture:public.products) Progress: 66%,  897640 pages completed.
[Mon Nov 16 01:27:34 2020] (ww_de_furniture:public.products) Progress: 66%,  898320 pages completed.
[Mon Nov 16 01:28:34 2020] (ww_de_furniture:public.products) Progress: 66%,  899060 pages completed.
[Mon Nov 16 01:29:35 2020] (ww_de_furniture:public.products) Progress: 66%,  899870 pages completed.
[Mon Nov 16 01:30:35 2020] (ww_de_furniture:public.products) Progress: 66%,  900585 pages completed.
[Mon Nov 16 01:31:35 2020] (ww_de_furniture:public.products) Progress: 66%,  901315 pages completed.
[Mon Nov 16 01:32:36 2020] (ww_de_furniture:public.products) Progress: 66%,  901930 pages completed.
[Mon Nov 16 01:33:38 2020] (ww_de_furniture:public.products) Progress: 66%,  902170 pages completed.
[Mon Nov 16 01:34:38 2020] (ww_de_furniture:public.products) Progress: 66%,  902635 pages completed.
[Mon Nov 16 01:35:38 2020] (ww_de_furniture:public.products) Progress: 67%,  903335 pages completed.
[Mon Nov 16 01:36:38 2020] (ww_de_furniture:public.products) Progress: 67%,  904105 pages completed.
[Mon Nov 16 01:37:39 2020] (ww_de_furniture:public.products) Progress: 67%,  904770 pages completed.
[Mon Nov 16 01:38:39 2020] (ww_de_furniture:public.products) Progress: 67%,  905635 pages completed.
[Mon Nov 16 01:40:07 2020] (ww_de_furniture:public.products) Progress: 67%,  906530 pages completed.
[Mon Nov 16 01:41:08 2020] (ww_de_furniture:public.products) Progress: 67%,  907545 pages completed.
[Mon Nov 16 01:42:08 2020] (ww_de_furniture:public.products) Progress: 67%,  908575 pages completed.
[Mon Nov 16 01:43:08 2020] (ww_de_furniture:public.products) Progress: 67%,  909530 pages completed.
[Mon Nov 16 01:44:08 2020] (ww_de_furniture:public.products) Progress: 67%,  910510 pages completed.
[Mon Nov 16 01:45:08 2020] (ww_de_furniture:public.products) Progress: 67%,  911320 pages completed.
[Mon Nov 16 01:46:08 2020] (ww_de_furniture:public.products) Progress: 67%,  912365 pages completed.
[Mon Nov 16 01:47:10 2020] (ww_de_furniture:public.products) Progress: 67%,  913125 pages completed.
[Mon Nov 16 01:48:13 2020] (ww_de_furniture:public.products) Progress: 67%,  913820 pages completed.
[Mon Nov 16 01:49:14 2020] (ww_de_furniture:public.products) Progress: 67%,  914300 pages completed.
[Mon Nov 16 01:50:14 2020] (ww_de_furniture:public.products) Progress: 67%,  914740 pages completed.
[Mon Nov 16 01:51:14 2020] (ww_de_furniture:public.products) Progress: 67%,  915510 pages completed.
[Mon Nov 16 01:52:14 2020] (ww_de_furniture:public.products) Progress: 67%,  916335 pages completed.
[Mon Nov 16 01:53:15 2020] (ww_de_furniture:public.products) Progress: 68%,  917045 pages completed.
[Mon Nov 16 01:54:15 2020] (ww_de_furniture:public.products) Progress: 68%,  917875 pages completed.
[Mon Nov 16 01:55:15 2020] (ww_de_furniture:public.products) Progress: 68%,  918795 pages completed.
[Mon Nov 16 01:56:15 2020] (ww_de_furniture:public.products) Progress: 68%,  919820 pages completed.
[Mon Nov 16 01:57:15 2020] (ww_de_furniture:public.products) Progress: 68%,  920865 pages completed.
[Mon Nov 16 01:58:16 2020] (ww_de_furniture:public.products) Progress: 68%,  921590 pages completed.
[Mon Nov 16 01:59:16 2020] (ww_de_furniture:public.products) Progress: 68%,  922210 pages completed.
[Mon Nov 16 02:00:16 2020] (ww_de_furniture:public.products) Progress: 68%,  922890 pages completed.
[Mon Nov 16 02:01:35 2020] (ww_de_furniture:public.products) Progress: 68%,  922990 pages completed.
[Mon Nov 16 02:02:35 2020] (ww_de_furniture:public.products) Progress: 68%,  923795 pages completed.
[Mon Nov 16 02:03:36 2020] (ww_de_furniture:public.products) Progress: 68%,  924515 pages completed.
[Mon Nov 16 02:04:39 2020] (ww_de_furniture:public.products) Progress: 68%,  924905 pages completed.
[Mon Nov 16 02:05:39 2020] (ww_de_furniture:public.products) Progress: 68%,  925345 pages completed.
[Mon Nov 16 02:06:39 2020] (ww_de_furniture:public.products) Progress: 68%,  925965 pages completed.
[Mon Nov 16 02:07:39 2020] (ww_de_furniture:public.products) Progress: 68%,  926700 pages completed.
[Mon Nov 16 02:08:40 2020] (ww_de_furniture:public.products) Progress: 68%,  927560 pages completed.
[Mon Nov 16 02:09:41 2020] (ww_de_furniture:public.products) Progress: 68%,  928320 pages completed.
[Mon Nov 16 02:10:41 2020] (ww_de_furniture:public.products) Progress: 68%,  929115 pages completed.
[Mon Nov 16 02:11:41 2020] (ww_de_furniture:public.products) Progress: 68%,  930045 pages completed.
[Mon Nov 16 02:12:48 2020] (ww_de_furniture:public.products) Progress: 69%,  930950 pages completed.
[Mon Nov 16 02:13:59 2020] (ww_de_furniture:public.products) Progress: 69%,  931730 pages completed.
[Mon Nov 16 02:14:59 2020] (ww_de_furniture:public.products) Progress: 69%,  932635 pages completed.
[Mon Nov 16 02:15:59 2020] (ww_de_furniture:public.products) Progress: 69%,  933595 pages completed.
[Mon Nov 16 02:17:00 2020] (ww_de_furniture:public.products) Progress: 69%,  934200 pages completed.
[Mon Nov 16 02:18:00 2020] (ww_de_furniture:public.products) Progress: 69%,  934875 pages completed.
[Mon Nov 16 02:19:00 2020] (ww_de_furniture:public.products) Progress: 69%,  935715 pages completed.
[Mon Nov 16 02:21:12 2020] (ww_de_furniture:public.products) Progress: 69%,  936110 pages completed.
[Mon Nov 16 02:22:12 2020] (ww_de_furniture:public.products) Progress: 69%,  936610 pages completed.
[Mon Nov 16 02:23:12 2020] (ww_de_furniture:public.products) Progress: 69%,  937730 pages completed.
[Mon Nov 16 02:24:12 2020] (ww_de_furniture:public.products) Progress: 69%,  939715 pages completed.
[Mon Nov 16 02:25:12 2020] (ww_de_furniture:public.products) Progress: 69%,  940980 pages completed.
[Mon Nov 16 02:26:12 2020] (ww_de_furniture:public.products) Progress: 69%,  942480 pages completed.
[Mon Nov 16 02:27:13 2020] (ww_de_furniture:public.products) Progress: 69%,  943340 pages completed.
[Mon Nov 16 02:28:13 2020] (ww_de_furniture:public.products) Progress: 70%,  944330 pages completed.
[Mon Nov 16 02:29:13 2020] (ww_de_furniture:public.products) Progress: 70%,  945290 pages completed.
[Mon Nov 16 02:30:14 2020] (ww_de_furniture:public.products) Progress: 70%,  946355 pages completed.
[Mon Nov 16 02:31:14 2020] (ww_de_furniture:public.products) Progress: 70%,  947390 pages completed.
[Mon Nov 16 02:32:14 2020] (ww_de_furniture:public.products) Progress: 70%,  948195 pages completed.
[Mon Nov 16 02:33:14 2020] (ww_de_furniture:public.products) Progress: 70%,  948955 pages completed.
[Mon Nov 16 02:34:15 2020] (ww_de_furniture:public.products) Progress: 70%,  949920 pages completed.
[Mon Nov 16 02:35:15 2020] (ww_de_furniture:public.products) Progress: 70%,  950635 pages completed.
[Mon Nov 16 02:36:15 2020] (ww_de_furniture:public.products) Progress: 70%,  951120 pages completed.
[Mon Nov 16 02:37:15 2020] (ww_de_furniture:public.products) Progress: 70%,  951550 pages completed.
[Mon Nov 16 02:38:15 2020] (ww_de_furniture:public.products) Progress: 70%,  952525 pages completed.
[Mon Nov 16 02:39:15 2020] (ww_de_furniture:public.products) Progress: 70%,  953485 pages completed.
[Mon Nov 16 02:40:16 2020] (ww_de_furniture:public.products) Progress: 70%,  954605 pages completed.
[Mon Nov 16 02:41:16 2020] (ww_de_furniture:public.products) Progress: 70%,  955450 pages completed.
[Mon Nov 16 02:42:16 2020] (ww_de_furniture:public.products) Progress: 70%,  956470 pages completed.
[Mon Nov 16 02:43:16 2020] (ww_de_furniture:public.products) Progress: 71%,  957630 pages completed.
[Mon Nov 16 02:44:17 2020] (ww_de_furniture:public.products) Progress: 71%,  958415 pages completed.
[Mon Nov 16 02:45:17 2020] (ww_de_furniture:public.products) Progress: 71%,  959710 pages completed.
[Mon Nov 16 02:46:17 2020] (ww_de_furniture:public.products) Progress: 71%,  961165 pages completed.
[Mon Nov 16 02:47:17 2020] (ww_de_furniture:public.products) Progress: 71%,  962100 pages completed.
[Mon Nov 16 02:48:17 2020] (ww_de_furniture:public.products) Progress: 71%,  963435 pages completed.
[Mon Nov 16 02:49:17 2020] (ww_de_furniture:public.products) Progress: 71%,  965165 pages completed.
[Mon Nov 16 02:50:17 2020] (ww_de_furniture:public.products) Progress: 71%,  966485 pages completed.
[Mon Nov 16 02:51:17 2020] (ww_de_furniture:public.products) Progress: 71%,  967435 pages completed.
[Mon Nov 16 02:52:17 2020] (ww_de_furniture:public.products) Progress: 71%,  968405 pages completed.
[Mon Nov 16 02:53:18 2020] (ww_de_furniture:public.products) Progress: 71%,  969200 pages completed.
[Mon Nov 16 02:54:19 2020] (ww_de_furniture:public.products) Progress: 71%,  969605 pages completed.
[Mon Nov 16 02:55:19 2020] (ww_de_furniture:public.products) Progress: 71%,  970120 pages completed.
[Mon Nov 16 02:56:20 2020] (ww_de_furniture:public.products) Progress: 72%,  970925 pages completed.
[Mon Nov 16 02:57:59 2020] (ww_de_furniture:public.products) Progress: 72%,  971585 pages completed.
[Mon Nov 16 02:58:59 2020] (ww_de_furniture:public.products) Progress: 72%,  972100 pages completed.
[Mon Nov 16 02:59:59 2020] (ww_de_furniture:public.products) Progress: 72%,  972860 pages completed.
[Mon Nov 16 03:00:59 2020] (ww_de_furniture:public.products) Progress: 72%,  973250 pages completed.
[Mon Nov 16 03:02:00 2020] (ww_de_furniture:public.products) Progress: 72%,  973650 pages completed.
[Mon Nov 16 03:03:00 2020] (ww_de_furniture:public.products) Progress: 72%,  974035 pages completed.
[Mon Nov 16 03:04:01 2020] (ww_de_furniture:public.products) Progress: 72%,  974790 pages completed.
[Mon Nov 16 03:05:01 2020] (ww_de_furniture:public.products) Progress: 72%,  975590 pages completed.
[Mon Nov 16 03:06:01 2020] (ww_de_furniture:public.products) Progress: 72%,  976385 pages completed.
[Mon Nov 16 03:07:01 2020] (ww_de_furniture:public.products) Progress: 72%,  977125 pages completed.
[Mon Nov 16 03:08:01 2020] (ww_de_furniture:public.products) Progress: 72%,  977735 pages completed.
[Mon Nov 16 03:09:01 2020] (ww_de_furniture:public.products) Progress: 72%,  978425 pages completed.
[Mon Nov 16 03:10:02 2020] (ww_de_furniture:public.products) Progress: 72%,  979180 pages completed.
[Mon Nov 16 03:11:25 2020] (ww_de_furniture:public.products) Progress: 72%,  979605 pages completed.
[Mon Nov 16 03:12:25 2020] (ww_de_furniture:public.products) Progress: 72%,  980050 pages completed.
[Mon Nov 16 03:13:26 2020] (ww_de_furniture:public.products) Progress: 72%,  980485 pages completed.
[Mon Nov 16 03:14:26 2020] (ww_de_furniture:public.products) Progress: 72%,  981315 pages completed.
[Mon Nov 16 03:15:26 2020] (ww_de_furniture:public.products) Progress: 72%,  982125 pages completed.
[Mon Nov 16 03:16:27 2020] (ww_de_furniture:public.products) Progress: 72%,  982710 pages completed.
[Mon Nov 16 03:17:27 2020] (ww_de_furniture:public.products) Progress: 72%,  983225 pages completed.
[Mon Nov 16 03:18:27 2020] (ww_de_furniture:public.products) Progress: 72%,  983750 pages completed.
[Mon Nov 16 03:19:27 2020] (ww_de_furniture:public.products) Progress: 73%,  984190 pages completed.
[Mon Nov 16 03:20:28 2020] (ww_de_furniture:public.products) Progress: 73%,  984650 pages completed.
[Mon Nov 16 03:21:28 2020] (ww_de_furniture:public.products) Progress: 73%,  985160 pages completed.
[Mon Nov 16 03:22:28 2020] (ww_de_furniture:public.products) Progress: 73%,  985480 pages completed.
[Mon Nov 16 03:23:29 2020] (ww_de_furniture:public.products) Progress: 73%,  986020 pages completed.
[Mon Nov 16 03:24:29 2020] (ww_de_furniture:public.products) Progress: 73%,  986620 pages completed.
[Mon Nov 16 03:25:30 2020] (ww_de_furniture:public.products) Progress: 73%,  987295 pages completed.
[Mon Nov 16 03:26:30 2020] (ww_de_furniture:public.products) Progress: 73%,  987790 pages completed.
[Mon Nov 16 03:27:30 2020] (ww_de_furniture:public.products) Progress: 73%,  988295 pages completed.
[Mon Nov 16 03:28:31 2020] (ww_de_furniture:public.products) Progress: 73%,  988720 pages completed.
[Mon Nov 16 03:29:31 2020] (ww_de_furniture:public.products) Progress: 73%,  989345 pages completed.
[Mon Nov 16 03:30:31 2020] (ww_de_furniture:public.products) Progress: 73%,  990000 pages completed.
[Mon Nov 16 03:31:31 2020] (ww_de_furniture:public.products) Progress: 73%,  990580 pages completed.
[Mon Nov 16 03:32:31 2020] (ww_de_furniture:public.products) Progress: 73%,  991430 pages completed.
[Mon Nov 16 03:33:31 2020] (ww_de_furniture:public.products) Progress: 73%,  991985 pages completed.
[Mon Nov 16 03:34:32 2020] (ww_de_furniture:public.products) Progress: 73%,  992580 pages completed.
[Mon Nov 16 03:35:32 2020] (ww_de_furniture:public.products) Progress: 73%,  992975 pages completed.
[Mon Nov 16 03:36:32 2020] (ww_de_furniture:public.products) Progress: 73%,  993360 pages completed.
[Mon Nov 16 03:37:33 2020] (ww_de_furniture:public.products) Progress: 73%,  994190 pages completed.
[Mon Nov 16 03:38:33 2020] (ww_de_furniture:public.products) Progress: 73%,  994890 pages completed.
[Mon Nov 16 03:40:13 2020] (ww_de_furniture:public.products) Progress: 73%,  995330 pages completed.
[Mon Nov 16 03:41:13 2020] (ww_de_furniture:public.products) Progress: 73%,  996015 pages completed.
[Mon Nov 16 03:42:14 2020] (ww_de_furniture:public.products) Progress: 73%,  996905 pages completed.
[Mon Nov 16 03:43:14 2020] (ww_de_furniture:public.products) Progress: 74%,  997660 pages completed.
[Mon Nov 16 03:44:14 2020] (ww_de_furniture:public.products) Progress: 74%,  998350 pages completed.
[Mon Nov 16 03:45:15 2020] (ww_de_furniture:public.products) Progress: 74%,  999005 pages completed.
[Mon Nov 16 03:46:15 2020] (ww_de_furniture:public.products) Progress: 74%,  999595 pages completed.
[Mon Nov 16 03:47:15 2020] (ww_de_furniture:public.products) Progress: 74%,  1000310 pages completed.
[Mon Nov 16 03:48:15 2020] (ww_de_furniture:public.products) Progress: 74%,  1001000 pages completed.
[Mon Nov 16 03:49:16 2020] (ww_de_furniture:public.products) Progress: 74%,  1001570 pages completed.
[Mon Nov 16 03:50:16 2020] (ww_de_furniture:public.products) Progress: 74%,  1002420 pages completed.
[Mon Nov 16 03:51:16 2020] (ww_de_furniture:public.products) Progress: 74%,  1003295 pages completed.
[Mon Nov 16 03:52:16 2020] (ww_de_furniture:public.products) Progress: 74%,  1004190 pages completed.
[Mon Nov 16 03:53:26 2020] (ww_de_furniture:public.products) Progress: 74%,  1005030 pages completed.
[Mon Nov 16 03:54:26 2020] (ww_de_furniture:public.products) Progress: 74%,  1005495 pages completed.
[Mon Nov 16 03:55:26 2020] (ww_de_furniture:public.products) Progress: 74%,  1005840 pages completed.
[Mon Nov 16 03:56:27 2020] (ww_de_furniture:public.products) Progress: 74%,  1006730 pages completed.
[Mon Nov 16 03:57:27 2020] (ww_de_furniture:public.products) Progress: 74%,  1007440 pages completed.
[Mon Nov 16 03:58:27 2020] (ww_de_furniture:public.products) Progress: 74%,  1008100 pages completed.
[Mon Nov 16 03:59:27 2020] (ww_de_furniture:public.products) Progress: 74%,  1008750 pages completed.
[Mon Nov 16 04:00:27 2020] (ww_de_furniture:public.products) Progress: 74%,  1009280 pages completed.
[Mon Nov 16 04:01:28 2020] (ww_de_furniture:public.products) Progress: 74%,  1009890 pages completed.
[Mon Nov 16 04:02:28 2020] (ww_de_furniture:public.products) Progress: 74%,  1010565 pages completed.
[Mon Nov 16 04:03:28 2020] (ww_de_furniture:public.products) Progress: 75%,  1011310 pages completed.
[Mon Nov 16 04:04:28 2020] (ww_de_furniture:public.products) Progress: 75%,  1012030 pages completed.
[Mon Nov 16 04:05:29 2020] (ww_de_furniture:public.products) Progress: 75%,  1012815 pages completed.
[Mon Nov 16 04:06:29 2020] (ww_de_furniture:public.products) Progress: 75%,  1013540 pages completed.
[Mon Nov 16 04:07:29 2020] (ww_de_furniture:public.products) Progress: 75%,  1014150 pages completed.
[Mon Nov 16 04:08:30 2020] (ww_de_furniture:public.products) Progress: 75%,  1015225 pages completed.
[Mon Nov 16 04:09:30 2020] (ww_de_furniture:public.products) Progress: 75%,  1015695 pages completed.
[Mon Nov 16 04:10:30 2020] (ww_de_furniture:public.products) Progress: 75%,  1016380 pages completed.
[Mon Nov 16 04:11:30 2020] (ww_de_furniture:public.products) Progress: 75%,  1018105 pages completed.
[Mon Nov 16 04:12:31 2020] (ww_de_furniture:public.products) Progress: 75%,  1018990 pages completed.
[Mon Nov 16 04:13:31 2020] (ww_de_furniture:public.products) Progress: 75%,  1019720 pages completed.
[Mon Nov 16 04:14:33 2020] (ww_de_furniture:public.products) Progress: 75%,  1020480 pages completed.
[Mon Nov 16 04:15:33 2020] (ww_de_furniture:public.products) Progress: 75%,  1020985 pages completed.
[Mon Nov 16 04:16:34 2020] (ww_de_furniture:public.products) Progress: 75%,  1021470 pages completed.
[Mon Nov 16 04:17:34 2020] (ww_de_furniture:public.products) Progress: 75%,  1021875 pages completed.
[Mon Nov 16 04:18:34 2020] (ww_de_furniture:public.products) Progress: 75%,  1022495 pages completed.
[Mon Nov 16 04:19:34 2020] (ww_de_furniture:public.products) Progress: 75%,  1023355 pages completed.
[Mon Nov 16 04:20:35 2020] (ww_de_furniture:public.products) Progress: 75%,  1024170 pages completed.
[Mon Nov 16 04:21:35 2020] (ww_de_furniture:public.products) Progress: 76%,  1024910 pages completed.
[Mon Nov 16 04:22:35 2020] (ww_de_furniture:public.products) Progress: 76%,  1025595 pages completed.
[Mon Nov 16 04:23:36 2020] (ww_de_furniture:public.products) Progress: 76%,  1026300 pages completed.
[Mon Nov 16 04:24:36 2020] (ww_de_furniture:public.products) Progress: 76%,  1026890 pages completed.
[Mon Nov 16 04:25:36 2020] (ww_de_furniture:public.products) Progress: 76%,  1027535 pages completed.
[Mon Nov 16 04:26:36 2020] (ww_de_furniture:public.products) Progress: 76%,  1028170 pages completed.
[Mon Nov 16 04:27:36 2020] (ww_de_furniture:public.products) Progress: 76%,  1028885 pages completed.
[Mon Nov 16 04:28:36 2020] (ww_de_furniture:public.products) Progress: 76%,  1029520 pages completed.
[Mon Nov 16 04:29:37 2020] (ww_de_furniture:public.products) Progress: 76%,  1030220 pages completed.
[Mon Nov 16 04:30:37 2020] (ww_de_furniture:public.products) Progress: 76%,  1030745 pages completed.
[Mon Nov 16 04:31:38 2020] (ww_de_furniture:public.products) Progress: 76%,  1031395 pages completed.
[Mon Nov 16 04:32:38 2020] (ww_de_furniture:public.products) Progress: 76%,  1032225 pages completed.
[Mon Nov 16 04:33:38 2020] (ww_de_furniture:public.products) Progress: 76%,  1033275 pages completed.
[Mon Nov 16 04:34:38 2020] (ww_de_furniture:public.products) Progress: 76%,  1034125 pages completed.
[Mon Nov 16 04:35:39 2020] (ww_de_furniture:public.products) Progress: 76%,  1034835 pages completed.
[Mon Nov 16 04:36:39 2020] (ww_de_furniture:public.products) Progress: 76%,  1035525 pages completed.
[Mon Nov 16 04:37:39 2020] (ww_de_furniture:public.products) Progress: 76%,  1036230 pages completed.
[Mon Nov 16 04:38:40 2020] (ww_de_furniture:public.products) Progress: 76%,  1036760 pages completed.
[Mon Nov 16 04:39:40 2020] (ww_de_furniture:public.products) Progress: 76%,  1037175 pages completed.
[Mon Nov 16 04:40:41 2020] (ww_de_furniture:public.products) Progress: 76%,  1037845 pages completed.
[Mon Nov 16 04:41:41 2020] (ww_de_furniture:public.products) Progress: 77%,  1038415 pages completed.
[Mon Nov 16 04:42:41 2020] (ww_de_furniture:public.products) Progress: 77%,  1039255 pages completed.
[Mon Nov 16 04:43:41 2020] (ww_de_furniture:public.products) Progress: 77%,  1040145 pages completed.
[Mon Nov 16 04:44:42 2020] (ww_de_furniture:public.products) Progress: 77%,  1042555 pages completed.
[Mon Nov 16 04:45:42 2020] (ww_de_furniture:public.products) Progress: 77%,  1043325 pages completed.
[Mon Nov 16 04:46:42 2020] (ww_de_furniture:public.products) Progress: 77%,  1044110 pages completed.
[Mon Nov 16 04:47:42 2020] (ww_de_furniture:public.products) Progress: 77%,  1044930 pages completed.
[Mon Nov 16 04:48:43 2020] (ww_de_furniture:public.products) Progress: 77%,  1045880 pages completed.
[Mon Nov 16 04:49:43 2020] (ww_de_furniture:public.products) Progress: 77%,  1046655 pages completed.
[Mon Nov 16 04:50:43 2020] (ww_de_furniture:public.products) Progress: 77%,  1047455 pages completed.
[Mon Nov 16 04:51:43 2020] (ww_de_furniture:public.products) Progress: 77%,  1048255 pages completed.
[Mon Nov 16 04:53:29 2020] (ww_de_furniture:public.products) Progress: 77%,  1048670 pages completed.
[Mon Nov 16 04:54:29 2020] (ww_de_furniture:public.products) Progress: 77%,  1049515 pages completed.
[Mon Nov 16 04:55:29 2020] (ww_de_furniture:public.products) Progress: 77%,  1050450 pages completed.
[Mon Nov 16 04:56:30 2020] (ww_de_furniture:public.products) Progress: 77%,  1051340 pages completed.
[Mon Nov 16 04:57:30 2020] (ww_de_furniture:public.products) Progress: 78%,  1052200 pages completed.
[Mon Nov 16 05:11:24 2020] (ww_de_furniture:public.products) Progress: 78%,  1052975 pages completed.
[Mon Nov 16 05:12:24 2020] (ww_de_furniture:public.products) Progress: 78%,  1053940 pages completed.
[Mon Nov 16 05:13:25 2020] (ww_de_furniture:public.products) Progress: 78%,  1054410 pages completed.
[Mon Nov 16 05:14:25 2020] (ww_de_furniture:public.products) Progress: 78%,  1055165 pages completed.
[Mon Nov 16 05:15:25 2020] (ww_de_furniture:public.products) Progress: 78%,  1056030 pages completed.
[Mon Nov 16 05:16:25 2020] (ww_de_furniture:public.products) Progress: 78%,  1056795 pages completed.
[Mon Nov 16 05:17:25 2020] (ww_de_furniture:public.products) Progress: 78%,  1057720 pages completed.
[Mon Nov 16 05:18:25 2020] (ww_de_furniture:public.products) Progress: 78%,  1058570 pages completed.
[Mon Nov 16 05:20:10 2020] (ww_de_furniture:public.products) Progress: 78%,  1059110 pages completed.
[Mon Nov 16 05:21:10 2020] (ww_de_furniture:public.products) Progress: 78%,  1059525 pages completed.
[Mon Nov 16 05:22:10 2020] (ww_de_furniture:public.products) Progress: 78%,  1060110 pages completed.
[Mon Nov 16 05:23:10 2020] (ww_de_furniture:public.products) Progress: 78%,  1061100 pages completed.
[Mon Nov 16 05:24:11 2020] (ww_de_furniture:public.products) Progress: 78%,  1061890 pages completed.
[Mon Nov 16 05:25:11 2020] (ww_de_furniture:public.products) Progress: 78%,  1062610 pages completed.
[Mon Nov 16 05:26:11 2020] (ww_de_furniture:public.products) Progress: 78%,  1063535 pages completed.
[Mon Nov 16 05:27:11 2020] (ww_de_furniture:public.products) Progress: 78%,  1064215 pages completed.
[Mon Nov 16 05:28:11 2020] (ww_de_furniture:public.products) Progress: 79%,  1065275 pages completed.
[Mon Nov 16 05:29:11 2020] (ww_de_furniture:public.products) Progress: 79%,  1066455 pages completed.
[Mon Nov 16 05:31:46 2020] (ww_de_furniture:public.products) Progress: 79%,  1066855 pages completed.
[Mon Nov 16 05:32:46 2020] (ww_de_furniture:public.products) Progress: 79%,  1070495 pages completed.
[Mon Nov 16 05:33:46 2020] (ww_de_furniture:public.products) Progress: 79%,  1073375 pages completed.
[Mon Nov 16 05:34:46 2020] (ww_de_furniture:public.products) Progress: 79%,  1076235 pages completed.
[Mon Nov 16 05:35:46 2020] (ww_de_furniture:public.products) Progress: 80%,  1079685 pages completed.
[Mon Nov 16 05:36:46 2020] (ww_de_furniture:public.products) Progress: 80%,  1082840 pages completed.
[Mon Nov 16 05:37:46 2020] (ww_de_furniture:public.products) Progress: 80%,  1086020 pages completed.
[Mon Nov 16 05:38:47 2020] (ww_de_furniture:public.products) Progress: 80%,  1089310 pages completed.
[Mon Nov 16 05:39:47 2020] (ww_de_furniture:public.products) Progress: 81%,  1092945 pages completed.
[Mon Nov 16 05:40:47 2020] (ww_de_furniture:public.products) Progress: 81%,  1096195 pages completed.
[Mon Nov 16 05:41:47 2020] (ww_de_furniture:public.products) Progress: 81%,  1099070 pages completed.
[Mon Nov 16 05:42:47 2020] (ww_de_furniture:public.products) Progress: 81%,  1101280 pages completed.
[Mon Nov 16 05:43:47 2020] (ww_de_furniture:public.products) Progress: 81%,  1103685 pages completed.
[Mon Nov 16 05:44:47 2020] (ww_de_furniture:public.products) Progress: 82%,  1106795 pages completed.
[Mon Nov 16 05:45:47 2020] (ww_de_furniture:public.products) Progress: 82%,  1109565 pages completed.
[Mon Nov 16 05:46:47 2020] (ww_de_furniture:public.products) Progress: 82%,  1111980 pages completed.
[Mon Nov 16 05:47:47 2020] (ww_de_furniture:public.products) Progress: 82%,  1114475 pages completed.
[Mon Nov 16 05:48:47 2020] (ww_de_furniture:public.products) Progress: 82%,  1116775 pages completed.
[Mon Nov 16 05:49:47 2020] (ww_de_furniture:public.products) Progress: 83%,  1119175 pages completed.
[Mon Nov 16 05:50:48 2020] (ww_de_furniture:public.products) Progress: 83%,  1121730 pages completed.
[Mon Nov 16 05:51:48 2020] (ww_de_furniture:public.products) Progress: 83%,  1124055 pages completed.
[Mon Nov 16 05:52:48 2020] (ww_de_furniture:public.products) Progress: 83%,  1126330 pages completed.
[Mon Nov 16 05:53:48 2020] (ww_de_furniture:public.products) Progress: 83%,  1128905 pages completed.
[Mon Nov 16 05:54:48 2020] (ww_de_furniture:public.products) Progress: 83%,  1130735 pages completed.
[Mon Nov 16 05:55:48 2020] (ww_de_furniture:public.products) Progress: 84%,  1133760 pages completed.
[Mon Nov 16 05:56:48 2020] (ww_de_furniture:public.products) Progress: 84%,  1136815 pages completed.
[Mon Nov 16 05:57:48 2020] (ww_de_furniture:public.products) Progress: 84%,  1140130 pages completed.
[Mon Nov 16 05:58:48 2020] (ww_de_furniture:public.products) Progress: 84%,  1143640 pages completed.
[Mon Nov 16 05:59:48 2020] (ww_de_furniture:public.products) Progress: 85%,  1145860 pages completed.
[Mon Nov 16 06:00:48 2020] (ww_de_furniture:public.products) Progress: 85%,  1146935 pages completed.
[Mon Nov 16 06:01:48 2020] (ww_de_furniture:public.products) Progress: 85%,  1148060 pages completed.
[Mon Nov 16 06:02:49 2020] (ww_de_furniture:public.products) Progress: 85%,  1149680 pages completed.
[Mon Nov 16 06:03:49 2020] (ww_de_furniture:public.products) Progress: 85%,  1151640 pages completed.
[Mon Nov 16 06:04:49 2020] (ww_de_furniture:public.products) Progress: 85%,  1153380 pages completed.
[Mon Nov 16 06:05:49 2020] (ww_de_furniture:public.products) Progress: 85%,  1155845 pages completed.
[Mon Nov 16 06:06:49 2020] (ww_de_furniture:public.products) Progress: 85%,  1158310 pages completed.
[Mon Nov 16 06:07:49 2020] (ww_de_furniture:public.products) Progress: 86%,  1160305 pages completed.
[Mon Nov 16 06:08:49 2020] (ww_de_furniture:public.products) Progress: 86%,  1162430 pages completed.
[Mon Nov 16 06:09:49 2020] (ww_de_furniture:public.products) Progress: 86%,  1165850 pages completed.
[Mon Nov 16 06:10:50 2020] (ww_de_furniture:public.products) Progress: 86%,  1170235 pages completed.
[Mon Nov 16 06:11:50 2020] (ww_de_furniture:public.products) Progress: 86%,  1172390 pages completed.
[Mon Nov 16 06:12:50 2020] (ww_de_furniture:public.products) Progress: 87%,  1175660 pages completed.
[Mon Nov 16 06:13:50 2020] (ww_de_furniture:public.products) Progress: 87%,  1177985 pages completed.
[Mon Nov 16 06:14:50 2020] (ww_de_furniture:public.products) Progress: 87%,  1180845 pages completed.
[Mon Nov 16 06:15:50 2020] (ww_de_furniture:public.products) Progress: 87%,  1182655 pages completed.
[Mon Nov 16 06:16:50 2020] (ww_de_furniture:public.products) Progress: 87%,  1184710 pages completed.
[Mon Nov 16 06:26:07 2020] (ww_de_furniture:public.products) Vacuum final: cannot clean 1185436 pages, 1922769 pages left, duration 531.897 seconds.
[Mon Nov 16 06:26:45 2020] (ww_de_furniture:public.products) Analyze final: duration 37.215 second.
[Mon Nov 16 06:30:39 2020] (ww_de_furniture:public.products) Bloat statistics with pgstattuple: duration 234.622 seconds.
[Mon Nov 16 07:32:54 2020] (ww_de_furniture:public.products) Reindex: public.products__checked_broken_images_at, initial size 47 pages(376.000KB), has been reduced by 34% (128.000KB), duration 3734 seconds, attempts 0.
[Mon Nov 16 07:32:54 2020] (ww_de_furniture:public.products) Reindex queries: public.products__checked_broken_images_at, initial size 47 pages (376.000KB), will be reduced by 49% (185.368KB)
[Mon Nov 16 07:32:54 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (checked_broken_images_at) TABLESPACE pg_default WHERE ((checked_broken_images_at IS NOT NULL) AND (broken_images = true)); --ww_de_furniture
[Mon Nov 16 07:32:54 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products__checked_broken_images_at" RENAME TO "tmp_84437558";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products__checked_broken_images_at";
END;
DROP INDEX CONCURRENTLY "public"."tmp_84437558";; --ww_de_furniture
[Mon Nov 16 08:24:07 2020] (ww_de_furniture:public.products) Reindex: public.products__broken_images, initial size 58 pages(464.000KB), has been reduced by 46% (216.000KB), duration 3072 seconds, attempts 0.
[Mon Nov 16 08:24:07 2020] (ww_de_furniture:public.products) Reindex queries: public.products__broken_images, initial size 58 pages (464.000KB), will be reduced by 43% (201.892KB)
[Mon Nov 16 08:24:07 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (broken_images) TABLESPACE pg_default WHERE ((broken_images = true) OR (broken_images IS NULL)); --ww_de_furniture
[Mon Nov 16 08:24:07 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products__broken_images" RENAME TO "tmp_65522692";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products__broken_images";
END;
DROP INDEX CONCURRENTLY "public"."tmp_65522692";; --ww_de_furniture
[Mon Nov 16 09:10:00 2020] (ww_de_furniture:public.products) Reindex: public.products__need_reindex, initial size 1664 pages(13.000MB), has been reduced by 90% (11.719MB), duration 2752 seconds, attempts 0.
[Mon Nov 16 09:10:00 2020] (ww_de_furniture:public.products) Reindex queries: public.products__need_reindex, initial size 1664 pages (13.000MB), will be reduced by 46% (6.098MB)
[Mon Nov 16 09:10:00 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (need_reindex) TABLESPACE pg_default WHERE (need_reindex = true); --ww_de_furniture
[Mon Nov 16 09:10:00 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products__need_reindex" RENAME TO "tmp_405189153";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products__need_reindex";
END;
DROP INDEX CONCURRENTLY "public"."tmp_405189153";; --ww_de_furniture
[Mon Nov 16 09:10:00 2020] (ww_de_furniture:public.products) Skipping reindex: public.products__unique_short_ean is a gin index not a btree, reindexing is up to you.
[Mon Nov 16 09:10:00 2020] (ww_de_furniture:public.products) Reindex queries: public.products__unique_short_ean, initial size 9038 pages (70.609MB)
[Mon Nov 16 09:10:00 2020] (ww_de_furniture:public.products) REINDEX INDEX ('"public"."products__unique_short_ean"'); --ww_de_furniture
[Mon Nov 16 10:03:39 2020] (ww_de_furniture:public.products) Reindex: public.products__canonical_url__partner_id__is_null__unique, initial size 10190 pages(79.609MB), has been reduced by 55% (43.867MB), duration 3218 seconds, attempts 0.
[Mon Nov 16 10:03:39 2020] (ww_de_furniture:public.products) Reindex queries: public.products__canonical_url__partner_id__is_null__unique, initial size 10190 pages (79.609MB), will be reduced by 35% (28.208MB)
[Mon Nov 16 10:03:39 2020] (ww_de_furniture:public.products) CREATE UNIQUE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (canonical_url) TABLESPACE pg_default WHERE (partner_id IS NULL); --ww_de_furniture
[Mon Nov 16 10:03:39 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products__canonical_url__partner_id__is_null__unique" RENAME TO "tmp_865339717";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products__canonical_url__partner_id__is_null__unique";
END;
DROP INDEX CONCURRENTLY "public"."tmp_865339717";; --ww_de_furniture
[Mon Nov 16 10:03:39 2020] (ww_de_furniture:public.products) Skipping reindex: public.products__ean is a gin index not a btree, reindexing is up to you.
[Mon Nov 16 10:03:39 2020] (ww_de_furniture:public.products) Reindex queries: public.products__ean, initial size 19857 pages (155.133MB)
[Mon Nov 16 10:03:39 2020] (ww_de_furniture:public.products) REINDEX INDEX ('"public"."products__ean"'); --ww_de_furniture
[Mon Nov 16 10:39:45 2020] (ww_de_furniture:public.products) Reindex: public.products__mark_as_deleted_at, initial size 24915 pages(194.648MB), has been reduced by 53% (104.320MB), duration 2165 seconds, attempts 0.
[Mon Nov 16 10:39:45 2020] (ww_de_furniture:public.products) Reindex queries: public.products__mark_as_deleted_at, initial size 24915 pages (194.648MB), will be reduced by 29% (56.686MB)
[Mon Nov 16 10:39:45 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (mark_as_deleted_at) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 10:39:45 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products__mark_as_deleted_at" RENAME TO "tmp_490914356";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products__mark_as_deleted_at";
END;
DROP INDEX CONCURRENTLY "public"."tmp_490914356";; --ww_de_furniture
[Mon Nov 16 11:32:42 2020] (ww_de_furniture:public.products) Reindex: public.products__last_external_visit_to_sp_page_at, initial size 24723 pages(193.148MB), has been reduced by 53% (102.812MB), duration 3174 seconds, attempts 0.
[Mon Nov 16 11:32:42 2020] (ww_de_furniture:public.products) Reindex queries: public.products__last_external_visit_to_sp_page_at, initial size 24723 pages (193.148MB), will be reduced by 29% (56.850MB)
[Mon Nov 16 11:32:42 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (last_external_visit_to_sp_page_at) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 11:32:42 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products__last_external_visit_to_sp_page_at" RENAME TO "tmp_869721077";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products__last_external_visit_to_sp_page_at";
END;
DROP INDEX CONCURRENTLY "public"."tmp_869721077";; --ww_de_furniture
[Mon Nov 16 12:28:56 2020] (ww_de_furniture:public.products) Reindex: public.products_created_at_index, initial size 25574 pages(199.797MB), has been reduced by 54% (109.492MB), duration 3371 seconds, attempts 0.
[Mon Nov 16 12:28:56 2020] (ww_de_furniture:public.products) Reindex queries: public.products_created_at_index, initial size 25574 pages (199.797MB), will be reduced by 35% (70.817MB)
[Mon Nov 16 12:28:56 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (created_at) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 12:28:56 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products_created_at_index" RENAME TO "tmp_228049313";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products_created_at_index";
END;
DROP INDEX CONCURRENTLY "public"."tmp_228049313";; --ww_de_furniture
[Mon Nov 16 13:06:42 2020] (ww_de_furniture:public.products) Reindex: public.products_partner_id_index, initial size 28369 pages(221.633MB), has been reduced by 59% (131.156MB), duration 2264 seconds, attempts 0.
[Mon Nov 16 13:06:42 2020] (ww_de_furniture:public.products) Reindex queries: public.products_partner_id_index, initial size 28369 pages (221.633MB), will be reduced by 37% (82.472MB)
[Mon Nov 16 13:06:42 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (partner_id) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 13:06:42 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products_partner_id_index" RENAME TO "tmp_958547825";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products_partner_id_index";
END;
DROP INDEX CONCURRENTLY "public"."tmp_958547825";; --ww_de_furniture
[Mon Nov 16 14:00:29 2020] (ww_de_furniture:public.products) Reindex: public.products_category_id_index, initial size 32689 pages(255.383MB), has been reduced by 64% (164.906MB), duration 3223 seconds, attempts 0.
[Mon Nov 16 14:00:29 2020] (ww_de_furniture:public.products) Reindex queries: public.products_category_id_index, initial size 32689 pages (255.383MB), will be reduced by 45% (116.114MB)
[Mon Nov 16 14:00:29 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (category_id) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 14:00:29 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products_category_id_index" RENAME TO "tmp_419669745";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products_category_id_index";
END;
DROP INDEX CONCURRENTLY "public"."tmp_419669745";; --ww_de_furniture
[Mon Nov 16 14:00:31 2020] (ww_de_furniture:public.products) Skipping reindex: public.products_pkey, can not reindex without heavy locks because of its dependencies, reindexing is up to you.
[Mon Nov 16 14:00:31 2020] (ww_de_furniture:public.products) Reindex queries: public.products_pkey, initial size 31412 pages (245.406MB), will be reduced by 47% (117.004MB)
[Mon Nov 16 14:00:31 2020] (ww_de_furniture:public.products) CREATE UNIQUE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (id) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 14:00:31 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER TABLE "public"."products" DROP CONSTRAINT "products_pkey";
ALTER TABLE "public"."products" ADD CONSTRAINT "products_pkey" PRIMARY KEY USING INDEX pgcompact_index_3035;
END;; --ww_de_furniture
[Mon Nov 16 14:16:51 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 1
[Mon Nov 16 14:16:54 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 2
[Mon Nov 16 14:16:57 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 3
[Mon Nov 16 14:17:00 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 4
[Mon Nov 16 14:17:03 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 5
[Mon Nov 16 14:17:06 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 6
[Mon Nov 16 14:17:09 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 7
[Mon Nov 16 14:17:12 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 8
[Mon Nov 16 14:17:15 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 9
[Mon Nov 16 14:17:18 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 10
[Mon Nov 16 14:17:21 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 11
[Mon Nov 16 14:17:24 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 12
[Mon Nov 16 14:17:27 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 13
[Mon Nov 16 14:17:30 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 14
[Mon Nov 16 14:17:33 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 15
[Mon Nov 16 14:17:36 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 16
[Mon Nov 16 14:17:39 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 17
[Mon Nov 16 14:17:42 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 18
[Mon Nov 16 14:17:45 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 19
[Mon Nov 16 14:17:48 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 20
[Mon Nov 16 14:17:51 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 21
[Mon Nov 16 14:17:54 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 22
[Mon Nov 16 14:17:57 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 23
[Mon Nov 16 14:18:00 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 24
[Mon Nov 16 14:18:03 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 25
[Mon Nov 16 14:18:06 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 26
[Mon Nov 16 14:18:09 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 27
[Mon Nov 16 14:18:12 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 28
[Mon Nov 16 14:18:15 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 29
[Mon Nov 16 14:18:18 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 30
[Mon Nov 16 14:18:21 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 31
[Mon Nov 16 14:18:24 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 32
[Mon Nov 16 14:18:27 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 33
[Mon Nov 16 14:18:30 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 34
[Mon Nov 16 14:18:33 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 35
[Mon Nov 16 14:18:36 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 36
[Mon Nov 16 14:18:39 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 37
[Mon Nov 16 14:18:42 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 38
[Mon Nov 16 14:18:45 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 39
[Mon Nov 16 14:18:48 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 40
[Mon Nov 16 14:18:51 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 41
[Mon Nov 16 14:18:54 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 42
[Mon Nov 16 14:18:57 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 43
[Mon Nov 16 14:19:00 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 44
[Mon Nov 16 14:19:03 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 45
[Mon Nov 16 14:19:06 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 46
[Mon Nov 16 14:19:09 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 47
[Mon Nov 16 14:19:12 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 48
[Mon Nov 16 14:19:15 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 49
[Mon Nov 16 14:19:18 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 50
[Mon Nov 16 14:19:21 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 51
[Mon Nov 16 14:19:24 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 52
[Mon Nov 16 14:19:27 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 53
[Mon Nov 16 14:19:30 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 54
[Mon Nov 16 14:19:33 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 55
[Mon Nov 16 14:19:36 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 56
[Mon Nov 16 14:19:39 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 57
[Mon Nov 16 14:19:42 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 58
[Mon Nov 16 14:19:45 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 59
[Mon Nov 16 14:19:48 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 60
[Mon Nov 16 14:19:51 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 61
[Mon Nov 16 14:19:54 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 62
[Mon Nov 16 14:19:57 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 63
[Mon Nov 16 14:20:00 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 64
[Mon Nov 16 14:20:03 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 65
[Mon Nov 16 14:20:06 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 66
[Mon Nov 16 14:20:09 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 67
[Mon Nov 16 14:20:12 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 68
[Mon Nov 16 14:20:15 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 69
[Mon Nov 16 14:20:18 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 70
[Mon Nov 16 14:20:21 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 71
[Mon Nov 16 14:20:24 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 72
[Mon Nov 16 14:20:27 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 73
[Mon Nov 16 14:20:30 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 74
[Mon Nov 16 14:20:33 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 75
[Mon Nov 16 14:20:36 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 76
[Mon Nov 16 14:20:39 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 77
[Mon Nov 16 14:20:42 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 78
[Mon Nov 16 14:20:45 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 79
[Mon Nov 16 14:20:48 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 80
[Mon Nov 16 14:20:51 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 81
[Mon Nov 16 14:20:54 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 82
[Mon Nov 16 14:20:57 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 83
[Mon Nov 16 14:21:00 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 84
[Mon Nov 16 14:21:03 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 85
[Mon Nov 16 14:21:06 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 86
[Mon Nov 16 14:21:09 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 87
[Mon Nov 16 14:21:12 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 88
[Mon Nov 16 14:21:15 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 89
[Mon Nov 16 14:21:18 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 90
[Mon Nov 16 14:21:21 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 91
[Mon Nov 16 14:21:24 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 92
[Mon Nov 16 14:21:27 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 93
[Mon Nov 16 14:21:30 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 94
[Mon Nov 16 14:21:33 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 95
[Mon Nov 16 14:21:36 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 96
[Mon Nov 16 14:21:39 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 97
[Mon Nov 16 14:21:42 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 98
[Mon Nov 16 14:21:45 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 99
[Mon Nov 16 14:21:48 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock retry 100
[Mon Nov 16 14:21:49 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, unable lock, delete index
[Mon Nov 16 14:47:59 2020] (ww_de_furniture:public.products) Reindex: public.products_uid_partner_id_unique, lock has not been acquired, initial size 42108 pages(328.969MB)
[Mon Nov 16 14:47:59 2020] (ww_de_furniture:public.products) Reindex queries: public.products_uid_partner_id_unique, initial size 42108 pages (328.969MB), will be reduced by 29% (96.169MB)
[Mon Nov 16 14:47:59 2020] (ww_de_furniture:public.products) CREATE UNIQUE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (uid, partner_id) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 14:47:59 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER TABLE "public"."products" DROP CONSTRAINT "products_uid_partner_id_unique";
ALTER TABLE "public"."products" ADD CONSTRAINT "products_uid_partner_id_unique" UNIQUE USING INDEX pgcompact_index_3035;
END;; --ww_de_furniture
[Mon Nov 16 15:26:40 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 1
[Mon Nov 16 15:26:43 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 2
[Mon Nov 16 15:26:46 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 3
[Mon Nov 16 15:26:49 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 4
[Mon Nov 16 15:26:52 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 5
[Mon Nov 16 15:26:55 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 6
[Mon Nov 16 15:26:58 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 7
[Mon Nov 16 15:27:01 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 8
[Mon Nov 16 15:27:04 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 9
[Mon Nov 16 15:27:07 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 10
[Mon Nov 16 15:27:10 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 11
[Mon Nov 16 15:27:13 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 12
[Mon Nov 16 15:27:16 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 13
[Mon Nov 16 15:27:19 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 14
[Mon Nov 16 15:27:22 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 15
[Mon Nov 16 15:27:25 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 16
[Mon Nov 16 15:27:28 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 17
[Mon Nov 16 15:27:31 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 18
[Mon Nov 16 15:27:34 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 19
[Mon Nov 16 15:27:37 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 20
[Mon Nov 16 15:27:40 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 21
[Mon Nov 16 15:27:43 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 22
[Mon Nov 16 15:27:46 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 23
[Mon Nov 16 15:27:49 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 24
[Mon Nov 16 15:27:52 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 25
[Mon Nov 16 15:27:55 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 26
[Mon Nov 16 15:27:58 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 27
[Mon Nov 16 15:28:01 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 28
[Mon Nov 16 15:28:04 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 29
[Mon Nov 16 15:28:07 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 30
[Mon Nov 16 15:28:10 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 31
[Mon Nov 16 15:28:13 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 32
[Mon Nov 16 15:28:16 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 33
[Mon Nov 16 15:28:19 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 34
[Mon Nov 16 15:28:22 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 35
[Mon Nov 16 15:28:25 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 36
[Mon Nov 16 15:28:28 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 37
[Mon Nov 16 15:28:31 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 38
[Mon Nov 16 15:28:34 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 39
[Mon Nov 16 15:28:37 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 40
[Mon Nov 16 15:28:40 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 41
[Mon Nov 16 15:28:43 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 42
[Mon Nov 16 15:28:46 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 43
[Mon Nov 16 15:28:49 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 44
[Mon Nov 16 15:28:52 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 45
[Mon Nov 16 15:28:55 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 46
[Mon Nov 16 15:28:58 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 47
[Mon Nov 16 15:29:01 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 48
[Mon Nov 16 15:29:04 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 49
[Mon Nov 16 15:29:07 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 50
[Mon Nov 16 15:29:10 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 51
[Mon Nov 16 15:29:13 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 52
[Mon Nov 16 15:29:16 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 53
[Mon Nov 16 15:29:19 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 54
[Mon Nov 16 15:29:22 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 55
[Mon Nov 16 15:29:25 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 56
[Mon Nov 16 15:29:28 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 57
[Mon Nov 16 15:29:31 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 58
[Mon Nov 16 15:29:34 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 59
[Mon Nov 16 15:29:37 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 60
[Mon Nov 16 15:29:40 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 61
[Mon Nov 16 15:29:43 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 62
[Mon Nov 16 15:29:46 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 63
[Mon Nov 16 15:29:49 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 64
[Mon Nov 16 15:29:52 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 65
[Mon Nov 16 15:29:55 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 66
[Mon Nov 16 15:29:58 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 67
[Mon Nov 16 15:30:01 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 68
[Mon Nov 16 15:30:04 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 69
[Mon Nov 16 15:30:07 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 70
[Mon Nov 16 15:30:10 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 71
[Mon Nov 16 15:30:13 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 72
[Mon Nov 16 15:30:16 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 73
[Mon Nov 16 15:30:19 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 74
[Mon Nov 16 15:30:22 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 75
[Mon Nov 16 15:30:25 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 76
[Mon Nov 16 15:30:28 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 77
[Mon Nov 16 15:30:31 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 78
[Mon Nov 16 15:30:34 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 79
[Mon Nov 16 15:30:37 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 80
[Mon Nov 16 15:30:40 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 81
[Mon Nov 16 15:30:43 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 82
[Mon Nov 16 15:30:46 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 83
[Mon Nov 16 15:30:49 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 84
[Mon Nov 16 15:30:52 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 85
[Mon Nov 16 15:30:55 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 86
[Mon Nov 16 15:30:58 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 87
[Mon Nov 16 15:31:01 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 88
[Mon Nov 16 15:31:04 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 89
[Mon Nov 16 15:31:07 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 90
[Mon Nov 16 15:31:10 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 91
[Mon Nov 16 15:31:13 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 92
[Mon Nov 16 15:31:16 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 93
[Mon Nov 16 15:31:19 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 94
[Mon Nov 16 15:31:22 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 95
[Mon Nov 16 15:31:25 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 96
[Mon Nov 16 15:31:28 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 97
[Mon Nov 16 15:31:31 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 98
[Mon Nov 16 15:31:34 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 99
[Mon Nov 16 15:31:37 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock retry 100
[Mon Nov 16 15:31:38 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, unable lock, delete index
[Mon Nov 16 16:09:15 2020] (ww_de_furniture:public.products) Reindex: public.products_short_uid_partner_id_unique, lock has not been acquired, initial size 45124 pages(352.531MB)
[Mon Nov 16 16:09:15 2020] (ww_de_furniture:public.products) Reindex queries: public.products_short_uid_partner_id_unique, initial size 45124 pages (352.531MB), will be reduced by 32% (114.103MB)
[Mon Nov 16 16:09:15 2020] (ww_de_furniture:public.products) CREATE UNIQUE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (short_uid, partner_id) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 16:09:15 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER TABLE "public"."products" DROP CONSTRAINT "products_short_uid_partner_id_unique";
ALTER TABLE "public"."products" ADD CONSTRAINT "products_short_uid_partner_id_unique" UNIQUE USING INDEX pgcompact_index_3035;
END;; --ww_de_furniture
[Mon Nov 16 16:09:15 2020] (ww_de_furniture:public.products) Skipping reindex: public.products__images_ids is a gin index not a btree, reindexing is up to you.
[Mon Nov 16 16:09:15 2020] (ww_de_furniture:public.products) Reindex queries: public.products__images_ids, initial size 45510 pages (355.547MB)
[Mon Nov 16 16:09:15 2020] (ww_de_furniture:public.products) REINDEX INDEX ('"public"."products__images_ids"'); --ww_de_furniture
[Mon Nov 16 17:01:51 2020] (ww_de_furniture:public.products) Reindex: public.products_canonical_url_index, initial size 136635 pages(1.042GB), has been reduced by 45% (486.211MB), duration 3146 seconds, attempts 0.
[Mon Nov 16 17:01:51 2020] (ww_de_furniture:public.products) Reindex queries: public.products_canonical_url_index, initial size 136635 pages (1.042GB), will be reduced by 32% (349.771MB)
[Mon Nov 16 17:01:51 2020] (ww_de_furniture:public.products) CREATE INDEX CONCURRENTLY pgcompact_index_3035 ON public.products USING btree (canonical_url) TABLESPACE pg_default; --ww_de_furniture
[Mon Nov 16 17:01:51 2020] (ww_de_furniture:public.products) BEGIN; SET LOCAL statement_timeout TO 2000;
ALTER INDEX "public"."products_canonical_url_index" RENAME TO "tmp_306730786";
ALTER INDEX "public".pgcompact_index_3035 RENAME TO "products_canonical_url_index";
END;
DROP INDEX CONCURRENTLY "public"."tmp_306730786";; --ww_de_furniture
[Mon Nov 16 17:01:51 2020] (ww_de_furniture:public.products) Processing incomplete.
[Mon Nov 16 17:01:51 2020] (ww_de_furniture:public.products) Processing results: 1922769 pages (2510768 pages including toasts and indexes), size has been reduced by 0.000B (2.094GB including toasts and indexes) in total. This attempt has been initially expected to compact ~69% more space (1343888 pages, 10.253GB)
[Mon Nov 16 17:01:51 2020] (ww_de_furniture:public.products) Finish handling table public.products
[Mon Nov 16 17:01:51 2020] (ww_de_furniture) Processing incomplete: 1 tables left.
[Mon Nov 16 17:01:51 2020] (ww_de_furniture) Processing results: size reduced by 0.000B (2.094GB including toasts and indexes) in total.
[Mon Nov 16 17:01:51 2020] (ww_de_furniture) Disconnecting from database
[Mon Nov 16 17:01:51 2020] Processing incomplete: 1 databases left.
[Mon Nov 16 17:01:51 2020] Processing results: size reduced by 0.000B (2.094GB including toasts and indexes) in total.

Deleting comments

The developers in my team noticed that the comments that they left on the indexes were deleted.
Apparently, it is not a reindex concurrently operation that is being produced, but the creation of a new index and the removal of the old one.
I give a diff that may close this problem:

801,802c801
<     pg_catalog.pg_relation_size(indexoid) as idxsize,
<     coalesce(obj_description(cst.oid),obj_description(indexoid)) as comm
---
>     pg_catalog.pg_relation_size(indexoid) as idxsize
819c818
< LEFT JOIN pg_catalog.pg_constraint cst ON
---
> LEFT JOIN pg_catalog.pg_constraint ON
902,903c901
< DROP INDEX CONCURRENTLY " . _dbh->quote_identifier($schema_name) . "." . _dbh->quote_identifier($tmp_index_name) . ";
< COMMENT ON INDEX " . _dbh->quote_identifier($schema_name) . "." . _dbh->quote_identifier($index_data->{indexname}) . " IS " . _dbh->quote($index_data->{comm}) ;
---
> DROP INDEX CONCURRENTLY " . _dbh->quote_identifier($schema_name) . "." . _dbh->quote_identifier($tmp_index_name) . ";";

Why not use pgcompacttable?

We're considering enabling pgcompacttable in our production DB, to reclaim disk space. I know this is a strange question, but what are the downsides (if any) to using pgcompacttable?

Why wouldn't one want to enable this extension? What are some common pitfalls / cons?

I guess there must be some, or else this would be already built into postgres, enabled by default 🤓

RHEL6

Hello, everyone!
I'm trying to exececute pgcompacttable on RHEL6 and got error:
Connecting to database
install_driver(Pg) failed: Global symbol "$VERSION" requires explicit package name at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 73.
Global symbol "$VERSION" requires explicit package name at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 115.
Global symbol "$VERSION" requires explicit package name at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 119.
BEGIN not safe after errors--compilation aborted at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 171.
Compilation failed in require at (eval 6) line 3.

What am I doing wrong?

Thank you for attention!

SQL Error while compacting

Hi,

I saw an SQL error while running this on a Postgresql 13 DB. relevant part is:

[Thu Sep 30 09:38:26 2021] (dbname:public.tablename) Progress: 41%,  107925 pages completed.
[Thu Sep 30 09:39:27 2021] (dbname:public.tablename) SQL Error: ERROR:  operator is not unique: smallint[] pg_catalog.@> smallint[]
LINE 26: ...SELECT string_to_array(indkey::text, ' ')::int2[] operator(p...
                                                              ^
HINT:  Could not choose a best candidate operator. You might need to add explicit type casts.
[Thu Sep 30 09:39:27 2021] (dbname:public.tablename) Table handling interrupt.
[Thu Sep 30 09:39:27 2021] (dbname:public.tablename) Processing results: 440512 pages (849670 pages including toasts and indexes), size has been reduced by 540.609MB (491.039MB including toasts and indexes) in total. This attempt has been initially expected to compact ~43% more space (189530 pages, 1.446GB)
[Thu Sep 30 09:39:27 2021] (dbname:public.tablename) Handling tables. Attempt 3
[Thu Sep 30 09:39:38 2021] (dbname:public.tablename) Statistics: 440512 pages (849670 pages including toasts and indexes), it is expected that ~43.030% (189547 pages) can be compacted with the estimated space saving being 1.446GB.
[Thu Sep 30 09:40:38 2021] (dbname:public.tablename) Progress: 41%,  107115 pages completed.
[Thu Sep 30 09:41:38 2021] (dbname:public.tablename) SQL Error: ERROR:  operator is not unique: smallint[] pg_catalog.@> smallint[]
LINE 26: ...SELECT string_to_array(indkey::text, ' ')::int2[] operator(p...

I am not sure what is the issue, there is one unique, one non-unique index and one FK constraint which binds to another external unique value.

P.s.: Not using using the latest version (just saw that it's updated 20 days ago) so if this case is not expected on the latest, please ignore.

Error in --man

  General options:
    -?
    --help
        Display short help.

    -m
    --man
        Display full manual.

    -V
    --version
        Print version.

    -q
    --quiet
        Quiet mode. Do not display any messages exept error messages and
        total result.

    -v
    --verbose
        Verbose mode. Display all the progress messages.

    -q
    --quiet
        A verbosity level. One of "warning", "notice", "info". By default
        "notice".

--quiet is duplicated

Can't compact toast tables

We have a table with a lot of wasted space in pg_toast storage (~80 GB out of 200 GB total data) and would like to be able to repack it, however specifying pg_toast schema and the pg_toast_oid table leads to the script not finding the table, and toast does not get compacted when we try to compact the original table.

Trying to force it to compact the toast table leads to some errors like the following:

SQL Error: ERROR:  cannot change TOAST relation "pg_toast_19982"
CONTEXT:  PL/pgSQL function pgcompact_clean_pages_842(text,text,integer,integer,integer) line 50 at FOR over EXECUTE statement

Sadly, we can't do a full offline vacuum as we do not have the ability to add any local storage or free up enough disk space for storing 120 GB of data temporarily - it'd be neat if this script were capable of compacting toast. 😕

Multiple runs for big tables

For large tables (~30GB), is it possible to run pgcompacttable till say 10%, do a manual VACUUM and REINDEX, and then run pgcompacttable again which will just do the remaining 90% of the work?

Advisory lock never cleared, and connection not closed on interrupt leads to wedged locks

I'm experimenting with pgcompacttable. However, it appears trivial to wedge the table processing state machine:

durr@postgres-server ~> sudo -H -u postgres pgcompacttable --verbose --dbname webarchive -t web_pages
[Tue Oct 27 02:02:25 2020] (webarchive) Connecting to database
[Tue Oct 27 02:02:25 2020] (webarchive) Postgres backend pid: 8880
[Tue Oct 27 02:02:25 2020] (webarchive) Handling tables. Attempt 1
[Tue Oct 27 02:02:25 2020] (webarchive:public.web_pages) Start handling table public.web_pages
^C⏎                                                  
durr@postgres-server ~> sudo -H -u postgres pgcompacttable --verbose info --dbname webarchive -t web_pages
[Tue Oct 27 02:08:55 2020] (webarchive) Connecting to database
[Tue Oct 27 02:08:55 2020] (webarchive) Postgres backend pid: 23824
[Tue Oct 27 02:08:55 2020] (webarchive) Handling tables. Attempt 1
[Tue Oct 27 02:08:55 2020] (webarchive:public.web_pages) Start handling table public.web_pages
[Tue Oct 27 02:08:55 2020] (webarchive:public.web_pages) Skipping processing: another instance is working with table public.web_pages
[Tue Oct 27 02:08:55 2020] (webarchive:public.web_pages) Finish handling table public.web_pages
[Tue Oct 27 02:08:55 2020] (webarchive) Processing complete.
[Tue Oct 27 02:08:55 2020] (webarchive) Processing results: size reduced by 0.000B (0.000B including toasts and indexes) in total.
[Tue Oct 27 02:08:55 2020] (webarchive) Disconnecting from database
[Tue Oct 27 02:08:55 2020] Processing complete: 1 retries to process has been done
[Tue Oct 27 02:08:55 2020] Processing results: size reduced by 0.000B (0.000B including toasts and indexes) in total.

So basically, if you run pgcompacttable, interrupt it with Ctrl+C, and then re-run it, it leaves the advisory lock active. This basically reveals two things:

  • I don't have connection keepalives (so the database isn't noticing the connection is gone), and the lock is persisting.
  • pgcompacttable apparently does no cleanup of the locks or graceful closing of the connection on Ctrl + C.

Once I figured this out, it was straightforward enough to go and release the lock:

webarchive=# select * from pg_locks WHERE locktype = 'advisory';
 locktype | database | relation | page | tuple | virtualxid | transactionid | classid |   objid   | objsubid | virtualtransaction | pid  |     mode      | granted | fastpath
----------+----------+----------+------+-------+------------+---------------+---------+-----------+----------+--------------------+------+---------------+---------+----------
 advisory |    18623 |          |      |       |            |               |    1259 | 705758310 |        2 | 19/1370012         | 8880 | ExclusiveLock | t       | f
(1 row)

webarchive=# SELECT pg_terminate_backend(8880);
 pg_terminate_backend
----------------------
 t
(1 row)

webarchive=# select * from pg_locks WHERE locktype = 'advisory';
 locktype | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction | pid | mode | granted | fastpath
----------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-----+------+---------+----------
(0 rows)

Though doing the above meant I had to spend 20 minutes trying to figure out where the lock was, digging about in the source to figure out how it was being acquired, and reading up on what the deuce a advisory lock even was.

I'm not sure what the best solution here would be. Personally, I like to have the first Ctrl+C start a graceful shutdown, and any further Ctrl+Cs raise the exception harder, but that's a opinion thing.

In any event, this seems to be a pretty annoying footgun that should probably be documented. Or there should be a way to override/clear the advisory lock in the CLI.

Missing argument in sprintf at ./pgcompacttable line 180.

[Thu Nov 26 08:12:16 2015] (dbname:tablename) Skipping index indexname:
[Thu Nov 26 08:12:16 2015] (dbname:tablename) SQL Error: ERROR:  relation "pgcompact_index_25734" already exists
Missing argument in sprintf at ./pgcompacttable line 180.

where 25734 is pid of ./pgcompacttable,
(used current master version, 122a6d2 as: ./pgcompacttable --host/var/run/postgresql/ --dbname stagingdbname --verbose)
perl:

 /usr/bin/perl --version

This is perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-linux-gnu-thread-multi
(with 41 registered patches, see perl -V for more detail)

Copyright 1987-2013, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

on ubuntu:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.3 LTS
Release:    14.04
Codename:   trusty

trying first time against db cluster:

psql -c 'show server_version';
 server_version
----------------
 9.3.10
(1 row)

Create index take a long time !!

Hi ! Im testing pgcompacttable with this command :
perl pgcompacttable -v -f -d mydb_db -U toto -W password -t offer -h 127.0.0.1

[Mon Apr 23 11:42:27 2018] (mydb_db) Connecting to database
[Mon Apr 23 11:42:27 2018] (mydb_db) Postgres backend pid: 11553
[Mon Apr 23 11:42:27 2018] (mydb_db) Handling tables. Attempt 1
[Mon Apr 23 11:42:27 2018] (mydb_db:public.offer) Start handling table public.offer
[Mon Apr 23 11:42:46 2018] (mydb_db:public.offer) Vacuum initial: 75312 pages left, duration 18.658 seconds.
[Mon Apr 23 11:42:47 2018] (mydb_db:public.offer) Bloat statistics with pgstattuple: duration 0.843 seconds.
[Mon Apr 23 11:42:47 2018] (mydb_db:public.offer) Statistics: 75312 pages (470320 pages including toasts and indexes), it is expected that ~19.610% (14771 pages) can be compacted with the estimated space saving being 115.399MB.
[Mon Apr 23 11:42:47 2018] (mydb_db:public.offer) Processing forced.
[Mon Apr 23 11:42:47 2018] (mydb_db:public.offer) Update by column: allow_quote_requests.
[Mon Apr 23 11:42:47 2018] (mydb_dbpublic.offer) Set pages/round: 5.
[Mon Apr 23 11:42:47 2018] (mydb_db:public.offer) Set pages/vacuum: 1507.
[Mon Apr 23 11:43:49 2018] (mydb_db:public.offer) Progress: 72%, 10770 pages completed.
[Mon Apr 23 11:45:13 2018] (mydb_db:public.offer) Vacuum final: 61112 pages left, duration 35.200 seconds.
[Mon Apr 23 11:45:13 2018] (mydb_db:public.offer) Analyze final: duration 0.448 second.
[Mon Apr 23 11:45:13 2018] (mydb_db:public.offer) Bloat statistics with pgstattuple: duration 0.430 seconds.
[] waiting....

And when i check pg_stat_activity, i see that he's doing a Create index :
query | CREATE INDEX CONCURRENTLY pgcompact_index_11552 ON public.offer USING gin (additional_fields_values)

My table size : 500 Mb
So is it normal that a create index take a long time ?? ( > 3h for the moment ) ?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.