Handle partial backend config about tfmigrate HOT 5 CLOSED

Hi @pyranja, Thank you for sharing your use-case. For a workaround, I came up with a dirty hack. 😉

When switching a remote backend to local, the tfmigrate generates an override file named _tfmigrate_override.tf under the hood and you can replace the terraform command to a custom wrapper script with TFMIGRATE_EXEC_PATH environment variable. This means that there is room for customizing the behavior of the terraform command depending on whether the backend is configured as remote or local.

from tfmigrate.

@minamijoyo I see, thanks.

So I could check for the presence of the override in a wrapper to decide whether backend config has to be provided or not?
I'll see if I can make that work.

What is your general feeling about accepting pass-through backend-config for the remote init?
If it's ok for you I'd try my luck on a PR some time later.

from tfmigrate.

It's ok to add support for the partial backend config. I think it should be a new migration block attribute rather than a new CLI flag, because a backend config is related to a working directory. It will cause the following problems if it's a CLI flag:

(1) The multi_state migration type uses two working directories, which have potentially different backend configs.
(2) With the migration history mode enabled, it applies multiple migration files in one-shot, which have potentially different backend configs.

from tfmigrate.

Update: The workaround with a terraform wrapper worked very well, thanks for the suggestion.

For our use case adding the backend config to the migration block would not work as is:

• we deploy the same terraform code to different environments (staging and production)
• the repo consists of many working directories, where each one defines some part of the infrastructure (e.g. a service)
• the working directories only define the state key in a backend block
• all other backend config is included from a common backend config file per environment
• we would like to apply migrations in a similar way to different environments

If the backend config reference is part of the migration, we would have to edit the migrations depending on the target environment or duplicate them for each target with a different backend config.
That's why I thought about a cli flag immediately.

Your concerns with the multi_state and history mode are very relevant too though.
I'm not sure if all requirements can be met at the same time, it would need a mix of having backend config in the migration and an option to override parts. But that feels like it's even messier than the workaround. 🤷

Thanks a lot for your explanation!

from tfmigrate.

I understood the cli flag would be useful your use-case. Not all requirements must be met at the same time. In your case the multi_state migration doesn't make sense across environments. If you apply migrations per environment, you can assume all pending migrations have the same backend config override.

There is no single solution for meeting all use-cases. It's ok to add it as the cli flag. If someone want to use it as the migration attribute, then we can support both individually.

from tfmigrate.