Giter VIP home page Giter VIP logo

vault-dump-kv2's Introduction

vault-dump-kv2

ci

Dump your Hashicorp Vault KV version 2 secrets engine contents to a file. Inspired by vault-backup but updated to work with KV version 2 only. Not guaranteed to be consistent.

Note: if you are looking for a more mature tool, go for https://github.com/jonasvinther/medusa! Or have a look at https://github.com/kir4h/rvault :)

Environment Variables

In addition to the environment variables like VAULT_ADDR that the hvac Python client for Hashicorp Vault implicitly uses, the following are used by this script:

  • PYTHONIOENCODING is used to ensure your keys are exported in valid encoding, make sure to use the same during import/export
  • VAULT_DUMP_MOUNTPOINT optionally passed as mount_point argument to the hvac Python client
  • VAULT_DUMP_PATH_PREFIX optionally can be used to only dump a sub path (e.g. "my/nested/path/") of the KV version 2 secrets engine

Setup

You need to have the vault CLI tool and pipenv installed.

vault login # with auth method of your choice

pipenv install # reads dependencies from Pipfile

export PYTHONIOENCODING="utf-8"
export VAULT_DUMP_MOUNTPOINT="/mysecrets/"
python vault-dump-kv2.py > mysecrets.txt

The generated script can be run with sh mysecrets.txt but beware that all keys in the target Vault will be overwritten on import!

Development

To run all checks and tests locally do:

export PYTHONIOENCODING="utf-8"
export VAULT_ADDR="http://127.0.0.1:8200"
export VAULT_DEV_ROOT_TOKEN_ID="test"

docker run -d --name vault -p 8200:8200 -e VAULT_DEV_ROOT_TOKEN_ID vault

make

Alternatively you can also open a Pull Request against this repository and let the CI run the checks and tests.

vault-dump-kv2's People

Contributors

cmur2 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

rakeshgohel01 sergiogiuffrida isabella232

vault-dump-kv2's Issues

specify sub-path

may you add the possibility to specify a sub-path to recursively dump?

trying to set it like this failed:

root@da686ccbe390:/# export VAULT_DUMP_MOUNTPOINT="/kv/prod/"

root@da686ccbe390:/# python /opt/vault-dump-kv2.py 
#
# vault-dump-kv2.py backup
# backup date: 2020-04-03 12:52:21.037428 UTC
# VAULT_DUMP_MOUNTPOINT setting: /kv/online/
# STDIN encoding: utf-8
# STDOUT encoding: utf-8
#
# WARNING: not guaranteed to be consistent!
#
Recursing into path prefix ""
Traceback (most recent call last):
  File "/opt/vault-dump-kv2.py", line 51, in <module>
    recurse_secrets('', vault_dump_mountpoint)
  File "/opt/vault-dump-kv2.py", line 30, in recurse_secrets
    keys = client.secrets.kv.v2.list_secrets(path_prefix, mount_point=mountpoint)['data']['keys']
  File "/usr/local/lib/python3.8/site-packages/hvac/api/secrets_engines/kv_v2.py", line 301, in list_secrets
    response = self._adapter.list(
  File "/usr/local/lib/python3.8/site-packages/hvac/adapters.py", line 142, in list
    return self.request('list', url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/hvac/adapters.py", line 272, in request
    utils.raise_for_error(response.status_code, text, errors=errors)
  File "/usr/local/lib/python3.8/site-packages/hvac/utils.py", line 38, in raise_for_error
    raise exceptions.InvalidPath(message, errors=errors)
hvac.exceptions.InvalidPath: {"request_id":"37c70dfa-62c7-4ad4-cba3-cff2cff678b2","lease_id":"","renewable":false,"lease_duration":0,"data":null,"wrap_info":null,"warnings":["Invalid path for a versioned K/V secrets engine. See the API docs for the appropriate API endpoints to use. If using the Vault CLI, use 'vault kv list' for this operation."],"auth":null}

backup all secrets

hi, I have many secrets in my vault, could I backup all of those secrets at one time?
I used
export VAULT_DUMP_MOUNTPOINT="/secrets/"

but it was failed:

Recursing into path prefix ""
Traceback (most recent call last):
  File "vault-dump-kv2.py", line 53, in <module>
    recurse_secrets(vault_dump_path_prefix, vault_dump_mountpoint)
  File "vault-dump-kv2.py", line 30, in recurse_secrets
    keys = client.secrets.kv.v2.list_secrets(path_prefix, mount_point=mountpoint)['data']['keys']
  File "/usr/local/lib/python3.8/dist-packages/hvac/api/secrets_engines/kv_v2.py", line 298, in list_secrets
    return self._adapter.list(
  File "/usr/local/lib/python3.8/dist-packages/hvac/adapters.py", line 146, in list
    return self.request('list', url, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/hvac/adapters.py", line 342, in request
    response = super(JSONAdapter, self).request(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/hvac/adapters.py", line 304, in request
    utils.raise_for_error(
  File "/usr/local/lib/python3.8/dist-packages/hvac/utils.py", line 43, in raise_for_error
    raise exceptions.InvalidPath(message, errors=errors, method=method, url=url)
hvac.exceptions.InvalidPath: no handler for route 'secrets/metadata/', on list http://127.0.0.1:8200/v1/secrets/metadata

Backup is failing for secrets marked as deleted but not destroyed

In Vault, secrets can be marked for deletion in it's most recent version while keeping their version history to allow undo - in presence of those this script fails. Only when destroying secrets they and their version history are purged entirely.

The script should as a first measure ignore secrets can be marked for deletion in it's most recent version as the scripts scope currently is to only dump the current state of secrets in Vault and users are expected to invoke it periodically to get a history of backups.

Even nicer would be to also export the version history of all secrets current present in Vault in a form to be able to restore that history.

Public copy of INFRA-1754

No requirements.txt

Yo!
Just wanted to let you know that in the README you mention to run pip3 install -r requirements.txt to install dependencies.
You don't have a requirements.txt :-)
I understand the only one you need is hvac, so I'll you decide if you want to create a requirements file or just amend the instructions to do a pip3 install hvac :-)
Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.