adding aws_session_token to config about target-s3 HOT 9 CLOSED

@crowemi more than happy to raise a PR can you grant me permission to create a new branch?
I run the following locally

➜  target-s3 git:(main) git checkout -b add-aws-session-token
Switched to a new branch 'add-aws-session-token'
➜  target-s3 git:(add-aws-session-token) git push origin add-aws-session-token 
ERROR: Permission to crowemi/target-s3.git denied to Vzzarr.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Let me know if that is ok with you

from target-s3.

@crowemi my bad, this is actually my first time I fork and PR and didn't know that was possible.
But there you go: #12
Let me know if anything else is needed

from target-s3.

@Vzzarr -- I'd prefer if you made the changes and submitted a PR. I think the issue may be that you need to create a branch on your end, as main is protected 🤷‍♂️

from target-s3.

@Vzzarr, thank you for your patience 🙃 - I think this is supposed to be done from your fork: create a new branch off your fork then create a PR from that branch back into the main repo. If that doesn't work I'll set you up as collaborator. I just don't want to (and shouldn't) have to do that for everyone that wants to contribute. Again, I may be missing a setting on the repo.

from target-s3.

@crowemi One more thing I wanted to raise, I configured this target via Meltano and I had to create the following:

    config:
      prefix: $S3_KEY
      format:
        format_type: json
      cloud_provider:
        cloud_provider_type: aws
        aws:
          aws_access_key_id: $AWS_ACCESS_KEY_ID
          aws_secret_access_key: $AWS_SECRET_ACCESS_KEY
          aws_region: $AWS_DEFAULT_REGION
          aws_session_token: $AWS_SESSION_TOKEN
          aws_bucket: $S3_BUCKET

But using the meltano config target-s3 set aws_access_key $AWS_ACCESS_KEY_ID as per https://hub.meltano.com/loaders/target-s3/ it didn't add params like cloud_provider, cloud_provider_type or aws.
I had to manually add them after many tests and reading through the code base.
But maybe this can be addressed in a different PR, just wanted to raise my experience to your attention.

from target-s3.

@Vzzarr I think you need to use dot notation for this, but I'm not 💯 (cloud_provider.aws.aws_access_key_id)

meltano config target-s3 set cloud_provider.aws.aws_access_key_id $AWS_ACCESS_KEY_ID -- give this a try, and if it doesn't work let open a new issue and I'll investigate.

from target-s3.

Complete #12

from target-s3.

I run and got the following:

➜  meltano config target-s3 set cloud_provider.aws.aws_access_key_id $AWS_ACCESS_KEY_ID
2023-04-21T14:11:53.324541Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
/usr/local/lib/python3.10/site-packages/meltano/cli/interactive/config.py:390: RuntimeWarning: Unknown setting ''
  value, metadata = settings.set_with_metadata(
Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

not enough values to unpack (expected at least 1, got 0)

Also I'm kinda new to Meltano so I might miss something but was this documentation written by you?
https://hub.meltano.com/loaders/target-s3/

from target-s3.

I'm not sure where that documentation comes from. I typically set the config directly in the yaml file. You could use the meltano config <plugin> list command to see what the CLI outputs for that config node, docs.meltano.com

from target-s3.