Update Data Uses (taxonomy) about fideslang HOT 6 CLOSED

I like the idea of an "opt-out" migration: most users are going to prefer that if we rename categories/uses/etc., we just automatically update them to the new ones. That's not even a breaking change, it's just basically a cosmetic update. This'll work for users that are fully server-side.

For users that leverage the CLI and the YAMLs, I think also supporting a push/pull script to effectively "codemod" them up to date would be perfect 👌. We could also in theory support the ability to track "deprecated" keys that are no longer supported, but we could detect them being used and throw a warning, that'd be 🤌

Let's get these new uses merged in, and then support an "auto-migration" on the server-side immediately and then carve out a path for the "code mod"? That sounds good

from fideslang.

@mfbrown which consent epic was this required for? Let's just put it in that epic.

from fideslang.

I'm going to be breaking this down into smaller issues

from fideslang.

@mfbrown For the migration script, are there any hard restrictions here?

i.e. we have multiple ways to pet this puppy:

1. Add a data migration that runs automatically like all of our other database migrations. This is very opaque and somewhat intrusive as the user can't really choose when/how this happens nor compare changes. This feels potentially dangerous to me, as people may actually want to keep using the current taxonomy. We would be intrusively changing data under their noses in a way that isn't easy to revert. I don't really like this plan even though the nice part is that it happens automatically. Good for some users, bad for others potentially.

2. write a python script that leverages existing functionality (such as pull, etc) to pull everything from the server, make the changes in the local manifest files, and then give them the option to update the server. After this we'd then need to still delete the old default taxonomy from the server? or at least set old values to no longer be considered part of the default taxonomy.

3. Maybe something else I haven't thought of

These are two very different paradigms, and we'll probably need a bit of both to get exactly what you're asking for here. Since this is a non-breaking change technically, we can push this out and update fides core to 1.4, and then follow-up with a migration strategy. The long we wait to get this into Fides core the more people will be on the old Taxonomy, so I'd rather get it out earlier so we can also give ourselves some breathing room on implementing the Migration strategy and don't feel the need to rush it

NOTE: Since this won't be the last time we make these kinds of adjustments to the default taxonomy, as part of this I'd actually like to plan out a larger strategy/method to continually go about these migrations. Hence my ask for additional time here, I think we should set ourselves up for future success by doing things right in the present 🙂

from fideslang.

cc @NevilleS for the migration discussion

from fideslang.

@NevilleS good idea

I think this time around doing the auto-migration path is perfectly fine, but given the time/resources I'd love to plan out a more standard migration path(s) for when we update the taxonomy in the future

from fideslang.