Feature suggestion: initial resource version & content about s3-resource HOT 11 CLOSED

Hi there!

We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.

The current status is as follows:

• #110557866 Feature suggestion: initial resource version & content

This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.

from s3-resource.

Any feedback on this? We're happy to schedule the work, but need to know if the design seems correct and is suitable for merging upstream.

from s3-resource.

Yep, sorry for the delay.

This sounds good! We'd like to merge something like this. @vito mentioned that in the long term this may point to a missing CloudFormation/Terraform resource but in the short term this is a good solution. 👍

from s3-resource.

@bleach We've been feeling the need for this too - we'd been thinking of an "allow empty" flag for the S3 resource so it still fires with a version for a file that doesn't exist, rather than default content. We'd then create that content in a task... Use case is generating credentials.

Lemme know if you've got anywhere with yours!

from s3-resource.

@dcarley && @bleach, the interface is sound, but would you ever need one without the other? An initial version would have some content and some content would still require a version. An initial version makes more sense to me.

from s3-resource.

We've been feeling the need for "fire on non-existent file" more and more - typically when trying to create idempotent pipelines, where things like credentials and SSL certs are generated.

I'm thinking two modes of operation:

1. When no files are matched, emit one static, never-changing version
2. When no files are matched, emit a new version number

The former would be useful for when you only want this to happen once, and the latter for whenever a file is missing (in case its deletion is part of the object's lifecycle).

Does this sound like utter madness? Are we going about this the wrong way?

from s3-resource.

@DanielJonesEB, why do you need the file from s3 before it has been generated?

from s3-resource.

@jtarchie If the file does not exist, no version is emitted, and no job fires as a result. So we want a job that can create things if they don't exist, and no-op if they do. The only way of doing this at present is with tasks looking to the outside state of the world, which isn't so functional.

from s3-resource.

A +1 for the use case of @DanielJonesEB; we're having a very similar problem. S3 is acting as a cache for a build process; we want a job that checks the input file to see if it's up to date, and replaces it if not. As a job will never fire on a missing S3 resource, we're having to manually seed the bucket with a dummy file.

from s3-resource.

We opened #99 as a proposed solution.

from s3-resource.

Merged #99!

from s3-resource.