OPTIMIZE TABLE: When to use it? about clickhouse HOT 6 CLOSED

No such best practice. Each user has own preferences.
Clickhouse works perfectly without using optimize.
Clickhouse has very sophisticated merge planner which has a goal to reduce a number of merges and keep reasonable number parts the same time. This approach allows to save disk and CPU resources for queries.

One of my database has partitions with a final size 100MB, another database has partitions with a final size 15TB. It will be an insane mistake to try to optimize fully in the second case.

Check also min_age_to_force_merge_seconds. But it basically the same as optimize+crond, the same concerns.

from clickhouse.

No such best practice. Each user has own preferences. Clickhouse works perfectly without using optimize. Clickhouse has very sophisticated merge planner which has a goal to reduce a number of merges and keep reasonable number parts the same time. This approach allows to save disk and CPU resources for queries.

One of my database has partitions with a final size 100MB, another database has partitions with a final size 15TB. It will be an insane mistake to try to optimize fully in the second case.

Check also min_age_to_force_merge_seconds. But it basically the same as optimize+crond, the same concerns.

Thanks for the reply. Why you say it would be an insane mistake for the second table to be OPTIMIZEd? In what terms?

note: I’m not referring to OPTIMIZE final, only OPTIMIZE

from clickhouse.

What does OPTIMIZE do? It calls the unplanned merge.

What the sense to call it? To merge parts to max size (150GB)?
It's a waste of resources to merge 2 parts, one part with size 130GB and another with a size 10KB, if you have more parts with 150GB in the same partition, or just more parts.

from clickhouse.

What is OPTIMIZE does? It calls the unplanned merge.

What the sense to call it? To merge parts to max size (150GB)? It's a waste of resources to merge 2 parts, one part with size 130GB and another with a size 10KB, if you have more parts with 150GB in the same partition, or just more parts.

Ok so the idea is to rely on the result that the automatic merges get normally out, if we have big-size partitions?

from clickhouse.

Think about OPTIMIZE like it debug/admin command. Basically you should never use it, never need.
I guess it was introduced for Clickhouse CI (for Clickhouse development), to force merges within CI checks.

from clickhouse.

Ok, with this explanation it makes more sense. Thanks!

from clickhouse.