Check for reindexing (ReindexingRequiredError) about dipdup HOT 5 CLOSED

I think it's important to give this choice to the user. The need for automatic reindexing can vary from project to project.
Personally, I would like to enable automatic reindexing.

The easiest solution is to give a cli option like --auto-reindex.
A more flexible solution can be to have this as a configuration in dipdup.yml where people can even choose on which types of reindexing reasons (dipdup.enums.ReindexingReason) they want to auto reindex.

from dipdup.

Thanks for the feedback, everybody!

We've ended up with a separate section in config. Something like this:

spec_version: ...
advanced:
  postpone_jobs: true
  early_realtime: true
  skip_hasura: true
  reindex:
    # NOTE: Default action
    _: raise
    CONFIG_HASH_MISMATCH: auto
    SCHEMA_HASH_MISMATCH: raise
    MANUAL: ignore

Flags will be still in CLI having priority over config. It's not a final agreed solution tho.

from dipdup.

Hi and sorry for the late reply 🙏

Automatic reindexing is a complex question. There are seven reasons for reindexing (see dipdup.enums.ReindexingReason), but in all cases except unprocessed "deep" rollback you can avoid reindexing if you want to. For example: you've made a typo in config, got CONFIG_HASH_MISMATCH. Fixed config, run dipdup schema approve, moving on. Plus, some users have discovered automatic wipe after looong hours of indexing and got really frustrated.

So, to sum things up. I definitely don't want reindexing to be automatic and unavoidable. But as you say, we need an option for people who want they deployments to be "unattended". What do you think about possible solutions?

from dipdup.

Hi no problem

I don't know if it's the best way to do it but I can think of one first / easy way:
a cli command to check if reindexing would be needed e.g. dipdup run --check-reindexing which returns the reason or some default value

with that users would be able to write some small script to handle what should happen

If i'm not mistaking there are two ways to resolve the "reindexingrequired" state:

• schema approve ("ignore changes")
• schema wipe && schema export (use the changes)

from dipdup.

--auto-reindex cli option would be great

from dipdup.