spring-projects / spring-vault Goto Github PK

It would be nice to have optional reactive support with vault for when Spring 5 goes GA.

We are running into issue with request sent in XML format instead of JSON, when com.fasterxml.jackson.dataformat.xml.XmlMapper exists in the classpath. This can be fixed by specifying the content-type to application/json, but it's very cumbersome to do this each time we want to make Vault request.

Also in certain places such as ClientCertificateAuthentication the content-type is set to null, resulting in the MappingJackson2XmlHttpMessageConverter being used and XML being sent

Hi!
Since today I'm having the following error while trying to start my application, yesterday was working OK.

Is it possible that same changes in the snapshot crashes with my code?

Thanks a lot!

Caused by: java.lang.NoSuchMethodException: org.springframework.cloud.vault.config.VaultBootstrapConfiguration$$EnhancerBySpringCGLIB$$f3816d8f.<init>()
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:80)
        ... 57 more

I'm new to Vault and evaluating it's usage. In reading through the docs here: https://www.vaultproject.io/docs/auth/app-id.html it shows that they are deprecating this backend as of 0.6.1.

They recommend the use of the AppRole Auth Backend https://www.vaultproject.io/docs/auth/approle.html.

Possibly provide a ClientAuthentication for it.

The HTTP API for Vault now (as of 0.6.2) has a "warnings" field:

{
  "request_id":"f3fdbc6c-183e-1684-56dd-77b16eaac36d",
  "lease_id":"",
  "renewable":false,
  "lease_duration":2764800,
  "data":
  {
     "password":"test2",
     "username":"test"
  },
  "wrap_info":null,
  "warnings":null,
  "auth":null
}

This isn't bound in VaultResponseSupport and while VaultResponseSupport ignores unknowns, VaultResponse overrides that. When Jackson tries to deserialize for VaultOperations.read, it fails.

org.springframework.http.converter.HttpMessageNotReadableException: Could not read JSON: Unrecognized field "warnings" (class org.springframework.vault.support.VaultResponse), not marked as ignorable (8 known properties: "lease_id", "wrap_info", "renewable", "auth", "lease_duration", "data", "request_id", "metadata"])

The current Getting Started documentation states the dependencies necessary are:

<dependencies>

  <!-- other dependency elements omitted -->

  <dependency>
    <groupId>org.springframework.vault</groupId>
    <artifactId>spring-vault-core</artifactId>
    <version>{version}</version>
  </dependency>

</dependencies>

I found that in order to run the simple application, I need the following:

<dependencies>
    <dependency>
        <groupId>org.springframework.vault</groupId>
        <artifactId>spring-vault-core</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-beans</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-web</artifactId>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
    </dependency>
    <dependency>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
        <version>1.2</version>
    </dependency>
</dependencies>

It might be nice to bring in these dependencies transitively. If not, then it would at least be nice to include the necessary jars in the docs

Apply the same code formatting rules as in Spring Cloud Vault.

• Mockito 2.1.0-RC.1 -> 2.6.2
• AssertJ 3.5.2 -> 3.6.1

Netty4ClientHttpRequestFactory does not write the Content-Length header when sending POST requests using RestTemplate. Vault responds with Status 400 in such cases.

Adopt transit key property change from cipher_mode to type.

Spring Vault usage requires either subclassing AbstractVaultConfiguration or providing beans by a custom app configuration. It would make sense to provide a EnvironmentVaultConfiguration to improve dev experience and resolve configuration from Environment.

A possible implementation could look like:

@Configuration
public class EnvironmentVaultConfiguration extends AbstractVaultConfiguration implements ApplicationContextAware {

    /**
     * Specify an endpoint for connecting to Vault.
     */
    @Override
    public VaultEndpoint vaultEndpoint() {
        String uri = getEnvironment().getProperty("vault.uri");
        if(uri != null) {
            return VaultEndpoint.from(URI.create(uri));
        }
        // ...
        throw new IllegalStateException();
    }

    /**
     * Configure a client authentication. Please consider a more secure
     * authentication method for production use.
     */
    @Override
    public ClientAuthentication clientAuthentication() {
        // ...
        String token = getEnvironment().getProperty("vault.token");
        if(token != null) {
            return new TokenAuthentication(token);
        }
        // ...
        throw new IllegalStateException();
    }
}

Thanks to @rwinch who proposed the idea.

Creating policies in Vault is cumbersome due to the missing documentation and a rather inconvenient format. It would make sense to provide data structures that reflect Vault's policy format so administration of policies could be leveraged.

Is it intentional to use SLF4J as apposed to commons-logging (as Spring Framework does)? If it is, this doesn't seem ideal given the framework uses commons-logging. We should probably also update the dependencies for #20 to have slf4j an implementation and switch to jcl-over-slf4j

Provide a dedicated API for the transit backend (see https://www.vaultproject.io/docs/secrets/transit/index.html).

Add method to obtain a certificate and export the certificate bundle into a KeyStore.

It would be nice if the SNAPSHOT repository was documented in the reference http://docs.spring.io/spring-vault/docs/1.0.0.BUILD-SNAPSHOT/reference/html/#vault.core.getting-started

Tokens retrieved by any authentication method should be revoked when shutting down the context.

Flatten hierarchical JSON objects to property paths using dot-notation

  {
    "database": {
      "password": ...
    },
    "items": ["one", "two"],  
    "user.name": ...,
  }

should result in

database.password=...
items[0]=one
items[1]=two
user.name=...

Currently VaultPropertySource does not renew leases. Since every secret in Vault has a lease, it should renew the lease to ensure that the application continues to work even after the lease expires.

Spring Cloud Vault provides LeasingVaultPropertySource, but this means that VaultPropertySource really has no value for a long running application since the credentials will expire.

We should ensure that spring-vault is part of Spring IO Platform so that projects part of the platform can rely on it. See http://docs.spring.io/platform/docs/current/reference/htmlsingle/#maintenance-adding-dependencies

The use of AsyncTaskExecutor.execute(Runnable runnable, long delay) executes tasks immediately and not deferred. This causes an infinite loop and massive load on Vault.

The renewal should adopt the nature of its AsyncTaskExecutor: If the executor is a TaskScheduler it should schedule tasks otherwise delay task execution.

It would be nice if Registering a Vault instance using Java based metadata discussed how it is different than Instantiating VaultTemplate. At the moment, I don't feel like this transitions as well as the rest of the documentation.

See spring-cloud/spring-cloud-vault#75.

It would be nice to document how to externalize Vault's Token when using VaultPropertySource. At the moment, there is a bit of a chicken and the egg problem when using standard Spring Environment.

One solution might be to instruct users to leverage system properties for this. We might also discuss how Spring Cloud (and in the future Spring Boot) have bootstrap.properties.

http://docs.spring.io/spring-vault/docs/1.0.0.BUILD-SNAPSHOT/reference/html/#vault.core.propertysupport

Given we cannot control the vault API, how would we add additional functionality to VaultOperations and remain passive?

From the README, they both result in 404 not found pages.

[INFO] [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-source-plugin is missing.
[INFO] [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-javadoc-plugin is missing.
[INFO] [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-deploy-plugin is missing.

Allow enumerating transit keys with GET /transit/keys via VaultTransitOperations.

http://docs.spring.io/spring-vault/docs/1.0.0.BUILD-SNAPSHOT/reference/html/#dependencies

Cubbyhole authentication uses Vault primitives to provide a secured authentication workflow. Cubbyhole authentication uses tokens as primary login method. An ephemeral token is used to obtain a second, login VaultToken from Vault’s Cubbyhole secret backend. The login token is usually longer-lived and used to interact with Vault. The login token can be retrieved either from a wrapped response or from the data section.

See also: spring-cloud/spring-cloud-vault#15

See https://github.com/spring-projects/spring-framework/wiki/gradle-build-and-release-faq#user-content-wiki-docs_schema_dist_publication

String.format(…) misses s in its %s definition.

Currently @VaultPropertySource requires a vault path to function and registers all the keys associated to that path in the environment. This is very convenient, but a relative path loses some of the meaning of the variable name. For example, consider a setup which is @VaultPropertySource("mysql/creds/readonly"). Consuming the database username and password is now tied to the variables username and password. This isn't very meaningful in the context of the application, but it was in the context of the vault path.

It would be nice if the @VaultPropertySource supported a prefix. Something like: @VaultPropertySource(value="mysql/creds/readonly", propertyNamePrefix = "database."). Now the database username and password would be associated to the variables database.username and database.password.

I'm trying to get a consul token from vault, so that spring cloud consul can get additional config from consul.

When I use this config:

# bootstrap.yml

## Config for vault
spring.cloud.vault:
  host: vault.host
  port: 443
  scheme: https # must be https for production
  config:
    lifecycle:
      enabled: true
    order: -10
  authentication: APPROLE # Same thing happens when using token
  app-role:
    role-id: ******
    secret-id: *******
  consul:
    enabled: true
    role: application
  fail-fast: false

## Consul config
spring.cloud.consul:
  enabled: true
  host: 127.0.0.1 
  port: 8500
  config:
    enabled: true
    format: FILES
    failFast: true
    profile-separator: '-'
    default-context: application

Here's what I see on app startup:

2016-12-01 11:01:19.701 DEBUG 58221 --- [           main] o.s.c.e.PropertySourcesPropertyResolver  : Could not find key 'spring.cloud.consul.token' in any property source
2016-12-01 11:01:19.701 DEBUG 58221 --- [           main] o.s.c.e.PropertySourcesPropertyResolver  : Could not find key 'CONSUL_TOKEN' in any property source
2016-12-01 11:01:21.383 DEBUG 58221 --- [           main] org.apache.http.wire                     : http-outgoing-0 << "{"request_id":"3d26b618-1636-fa7b-2a1b-6039d3e4383f","lease_id":"consul/creds/application/570da8f4-7b23-6cf6-5956-8bcec0f9b735","renewable":true,"lease_duration":2592000,"data":{"token":"xxxxxxxxxx"},"wrap_info":null,"warnings":null,"auth":null}[\n]"
2016-12-01 11:01:21.854 DEBUG 58221 --- [           main] org.apache.http.wire                     :  >> "GET /v1/kv/config/application.properties?token= HTTP/1.1[\r][\n]"
2016-12-01 11:01:21.878 DEBUG 58221 --- [           main] org.apache.http.wire                     :  >> "GET /v1/kv/config/application.yaml?token= HTTP/1.1[\r][\n]"
2016-12-01 11:01:21.905 DEBUG 58221 --- [           main] org.apache.http.wire                     :  >> "GET /v1/kv/config/application.yml?token= HTTP/1.1[\r][\n]"
2016-12-01 11:01:21.920 DEBUG 58221 --- [           main] o.s.c.e.PropertySourcesPropertyResolver  : Found key 'spring.cloud.consul.token' in [bootstrapProperties] with type [String]

Note that while a vault token is obtained, Spring Cloud Consul does not seem to be picking it up.

Here's the dependencies in play (With Spring boot 1.4.2.RELEASE)

[INFO] +- org.springframework.cloud:spring-cloud-consul-config:jar:1.1.2.RELEASE:compile
[INFO] +- org.springframework.cloud:spring-cloud-starter-consul-discovery:jar:1.1.2.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-starter-consul:jar:1.1.2.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-commons:jar:1.1.6.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-context:jar:1.1.6.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-consul-core:jar:1.1.2.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-consul-discovery:jar:1.1.2.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-netflix-core:jar:1.2.3.RELEASE:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-starter-ribbon:jar:1.2.3.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-starter:jar:1.1.6.RELEASE:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-starter-archaius:jar:1.2.3.RELEASE:compile
[INFO] +- org.springframework.cloud:spring-cloud-vault-starter-config:jar:1.0.0.M1:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-vault-config:jar:1.0.0.M1:compile
[INFO] +- org.springframework.cloud:spring-cloud-vault-config-consul:jar:1.0.0.M1:compile# cloud dependencies

Looking at the /env endpoint, the property is set, but apparently not in time, for consul to start.

Currently the APIs of VaultTemplate and VaultClient look a lot like RestTemplate. If users want to do the REST operations, they already have RestTemplate to do this. It would be nice if the VaultClient APIs reflected the intent rather than the underlying REST semantics.

Add support to retrieve data from Vault paths and use it as PropertySource inside a Spring Environment.