Put task to an S3 bucket fails even though the actual upload succeeded about s3-resource HOT 9 OPEN

One of regexp or versioned_file must be specified for the resource's get and check to work. This is mentioned here but unfortunately because one of them has to be present, they're both marked "optional".

Should probably find a better way of formatting that, and/or add validations to the resource itself so it tells you what went wrong.

from s3-resource.

I got the same error trying to store files with this in Minio. It doesn't support versioned_file so I'm stuck with regexp (and there seems to be no notion of folder in Minio either).
Whatever I put into regex seems to make no difference (e.g. a group test-(.*).tgz or even version named group test-(?version.*).tgz).

How is this supposed to be used?

from s3-resource.

Hi there!

We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.

The current status is as follows:

• #140235895 Put task to an S3 bucket fails even though the actual upload succeeded
• #112642505 S3 resource should validate that either regexp or versioned_file is specified

This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.

from s3-resource.

Thanks. Yeah, we ended up using versioned_file option, and it's now working nicely.

from s3-resource.

+1 for validation.

from s3-resource.

This seems to still be a problem when using regexp in the get as per example configuration here:
https://github.com/concourse/s3-resource#example-configuration

Errors with:
error running command: InvalidParameter: 1 validation error(s) found.

• minimum field size of 1, HeadObjectInput.Key.

While using in the resource get:
regexp: directory_on_s3/release-(.*).tgz

It works with versioned_file - but seems you cant have * in that field

from s3-resource.

Have to correct myself: Minio has folders but I still wasn't able to get it working.

from s3-resource.

Hello
Is there any update on how to solve this ?

I am using this template

terraform {
  backend "s3" {}
}

data "terraform_remote_state" "state" {
  backend = "s3"
  config {
    access_key = "${var.access_key}"
    secret_key = "$(var.secret_key)"
    bucket     = "${var.tf_state_bucket}"
    dynamodb_table = "${var.tf_state_table}"
    region     = "${var.region}"
    key        = "${var.pathtokey}"
  }
}

The terraform init :

terraform init -backend-config "bucket=$tf_state_bucket" -backend-config "key=$pathtokey" -backend-config "region=${var.region}" -backend-config "dynamodb_table=$tf_state_table" -backend-config "access_key=XXXXXXXXXXX"  -backend-config "secret_key=XXXXXXXXXX" -plugin-dir=.

I get this error every time:

- minimum field size of 1, GetObjectInput.Key.
2017/10/19 14:49:19 [DEBUG] [aws-sdk-go] DEBUG: Validate Request s3/GetObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, GetObjectInput.Key.
2017/10/19 14:49:19 [DEBUG] [aws-sdk-go] DEBUG: Build Request s3/GetObject failed, not retrying, error InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, GetObjectInput.Key.
2017/10/19 14:49:19 [DEBUG] plugin: waiting for all plugin processes to complete...

My S3 bucket has versioning enabled and the file in the bucket is "terraform.tfstate". How should I be using the regexp or versioned_file? I tried to use them but did not work out well.

Thank you

from s3-resource.

I want to get all the files from s3 bucket/folder, how to pass the regex to read all the files

from s3-resource.