pivotal-cf / email-resource Goto Github PK

It looks like with today's release, there seem to be some header issues.

Originally I used the headers filed as described in the README.md

MIME-version: 1.0
Content-Type: text/html; charset="UTF-8"

However, I'm now running into errors looking like this:

v1.0.15-d5ab14181de9a8f66a5bed57e60da6adc7a212b8 - Error during execute - Error closing:: 554 Transaction failed: Duplicate header 'Content-Type'.

And after removing content-type:

v1.0.15-d5ab14181de9a8f66a5bed57e60da6adc7a212b8 - Error during execute - Error closing:: 554 Transaction failed: Duplicate header 'MIME-Version'.

Once I remove both of them, it's able to send email and is setting these headers for the values:

Content-Type: multipart/mixed; boundary="0d545cae4a990f55ba95086fc85790d23a2deb3fd787767402526f53e99d"; charset=UTF-8
Mime-Version: 1.0

It seems like it's related to this commit:
8bdfdd1

The header processing looks fine (in out.go), but then I think something goes wrong in the sender.go additions.

Maybe it's getting added a second time here?
https://github.com/pivotal-cf/email-resource/blob/master/out/sender.go#L69-L70

Hi,
I have a script named generate_report.js that generates cucumber_report.html. I need the cucumber_report.html file to be sent via e-mail. When I provide it in attachments field, I'm unable to see that file in e-mail. But inside my pipeline, I could print and see the contents of cucumber_report.html.

Please provide a solution for this

Hi,

I could not find a manifest file on vagrant box. Could you please let us know the path of the manifest file to update the concourse manifest to add email as a resource-type?

Thanks.

Hi, we've been using pcfseceng/email-resource on our team for a couple years and it's been great. For each pipeline, we have been emailing one or more developers. Around mid-August, we started receiving emails from sendgrid support on all pipelines which have multiple recipients in the "to_text" field.

The emails from sendgrid have the subject line "Invalid SMTPAPI Header"
and body:

A message was received from this address by our systems that had errors in the SMTPAPI header, and cannot be processed due to the following:

• 2 duplicate to header detected, rfc5322 states that there should be a max of 1

If you require assistance in working with the SendGrid API, please contact us at [email protected]

Here is an excerpt from an affected pipeline with some simplifications/generalizations

resource_types:
  - name: email
    type: docker-image
    source:
      repository: pcfseceng/email-resource
...

  - name: send-an-email
    type: email
    source:
      smtp:
        host: ((smtp-host))
        port: ((smtp-port))
        username: apikey
        password: ((sendgrid-api-key))
      from: "[email protected]"
...
    on_success:
      in_parallel:
        - put: send-an-email
          params:
            subject_text: "our subject"
            body_text: "body text"
            to_text: [ "[email protected]", "[email protected]" ]

Concourse indicates that it is indeed using the image with the "latest" tag
fb79dc8e3d7d03379d1592faa16ea46e6c7454e63688c94e28962262b45a9205
which matches the sha here: https://hub.docker.com/layers/pcfseceng/email-resource/latest/images/sha256-fb79dc8e3d7d03379d1592faa16ea46e6c7454e63688c94e28962262b45a9205?context=explore

The emails from sendgrid support have an attachment showing the email received from pcfseceng/email-resource, which seems to indicate the multiple "To" headers.

Received: by filterdrecv-7bf5c69d5-kwxgp with SMTP id filterdrecv-7bf5c69d5-kwxgp-1-614A67CC-43
2021-09-21 23:16:28.567144502 +0000 UTC m=+1731390.697352871
Received: from localhost (unknown [35.193.20.185])
by geopod-ismtpd-4-4 (SG) with ESMTP id N7aNpXgfRc2yJ4OPN8HUcA
Tue, 21 Sep 2021 23:16:28.308 +0000 (UTC)
From: [email protected]
Date: Tue, 21 Sep 2021 23:16:27 +0000
Subject: our subject
To: [email protected]
To: [email protected]
Content-Type: multipart/mixed;
boundary="0b6b34c35065cb31646bbd4d74b6d9d6ab998c6efbeb8816e65f37b9dc27"; charset=UTF-8
MIME-Version: 1.0

--0b6b34c35065cb31646bbd4d74b6d9d6ab998c6efbeb8816e65f37b9dc27
Content-Type: multipart/alternative;
boundary="4beee005b04ff286d9a68e1efc9f08c75136ec7268221d0265c05de9ab4c"

--4beee005b04ff286d9a68e1efc9f08c75136ec7268221d0265c05de9ab4c
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

body text

--4beee005b04ff286d9a68e1efc9f08c75136ec7268221d0265c05de9ab4c--

--0b6b34c35065cb31646bbd4d74b6d9d6ab998c6efbeb8816e65f37b9dc27--

We followed up with SendGrid and they replied with the following:

Hello

Thank you for contacting Twilio SendGrid, this is Sachin and I'll be glad to help you today.

As we continue to modernize and monitor our email delivery infrastructure to comply with industry standards, we are changing a few behaviours on the API. We had notificed about this change in June/July via email sent to the email address we have on file. The changes are as follows:

1. The word “enabled” (instead of “enable”) will no longer be a recognized parameter for the Mail Send API. The “enable” parameter is required, and if the parameter is omitted, settings will not be applied. Please use “enable” to ensure the desired behaviour.

2. Only one of each type of header is permitted. We will no longer support duplicate headers for the following:

· from

· sender

· reply-to

· to

· cc

· bcc

· x-smtpapi

3. We will not apply any personalization block substitutions to your custom argument fields in the V3 endpoint.

What do you need to do?

To avoid disruption to your mail send service, please be sure to take the following actions before August 10, 2021:

1. Check your code that uses the mail send endpoint and ensure that the “enable” parameter is included under the filter when applicable.

2. Check your mail send header to ensure that you are using just one X-SMTPAPI header and address header of the same kind. Remove multiple headers of the same kind, so you have only 1 header of the same kind. For example, if you are currently using multiple “from” headers, you should modify your code so that you have a single “from” header.

3. Check your code to ensure that you are not applying any personalization block substitution to your custom argument fields.

We'd greatly appreciate any guidance or help to resolve this.
Thanks,
Jim

Hi!

This resource is very useful in my projects, good work! 👏👏

However, one thing is quite uncomfortable: the lack of to_text field for the out parameters (that is an option to supply a string instead of file).

I mean, most of time I only need to mail constant / hardcoded recipients. In such case, I need to create a whole task at the beginning of the job just to prepare a file with comma-separated values (emails).

Also note that I can't define recipients under resource section, since in different jobs I want to mail different users.

I would be very glad if you added this feature. I really need that.

Cheers! ✋

Looking for a way to use login auth for smtp. Some servers require "login auth" rather than PlainAuth or CRAMMD5Auth.

I get the following when I try to send email using AWS SES

dial tcp: lookup on :53: read udp 10.254.0.66:50036->:53: i/o timeout

I have concourse running as docker. I am able to send email from concourse server and concourse worker as well but when triggered through job failure I am getting the above error.\

Any clue on this?

Hi,

I'd like to integrate this email-resource on a development landscape. I do have access to an SMTP server there, but without user/password for authentication. I don't think I'd need username/password in a dedicated development landscape.

Unfortunately with this plugin I have to provide username/password, and it does not works for my case. I'd be happy to see that feature. Or is that something that won't be supported as part of this resource?

Cheers,
Fabian

Concourse Version: 7.3.2

PUT email task fails with this error.

Here is my pipeline.

We use smtp-relay.gmail.com as a SMTP and until recently all works great.
But now email task ends with: v1.0.10-17dce345143614aa96d9a57c3f6ab8c5d6f3e664 - EOF

Well obviously it doesn't say much.. :(

I wanted to figure out the reason so compiled the out and debugged it with gdb and by hand created json file with the correct parameters.

And I figured that the problem is with this:

if err = c.Hello("localhost"); err != nil {
	return "", err
}

This ends up with EOL error.

When I tried:

telnet smtp-relay.gmail.com 587

helo localhost
421 4.7.0 Try again later, closing connection. (EHLO) h4-v6sm68320wrq.12 - gsmtp
Connection closed by foreign host.

So it indeed fails for localhost but not for our domain.

In the smtp relay documentation (https://support.google.com/a/answer/2956491?hl=en) is this:

We recommend that you configure your mail server to present a unique identifier (such as your domain name or the name of your mail server) in the HELO or EHLO command in the SMTP relay connections your server makes to Google. Avoid using generic names such as "localhost" or "smtp-relay.gmail.com," which can occasionally result in issues with DoS limits.

So I guess the better approach would be to use domain part from the sender address instead of localhost

Right now, you can only specify an array with the To parameter, or a file. You could make a variable, and use it within that array, however you can then only specify one URL from my testing.

What would be useful, is for each item in the array, try and split it by comma delineation (much like having it be in a file), or support a string rather than an array, and do comma delineation there (I'm not sure if this is a limitation on email-resource, or concourse).

I.e:

  name: send-mail
  type: email
  source:
    from: ((email-notify-address))
    smtp:
      host: smtp.domain.com
      password: ((email-notify-password))
      port: "25"
      skip_ssl_validation: true
      username: ((email-notify-username))
   to: "[email protected],[email protected]"
# parameterized would be:
#    to: [ ((email-list-variable)) ]

This way, [email protected],[email protected] as a string can be hoisted out into concourse param files.

Or:

  name: send-mail
  type: email
  source:
    from: ((email-notify-address))
    smtp:
      host: smtp.domain.com
      password: ((email-notify-password))
      port: "25"
      skip_ssl_validation: true
      username: ((email-notify-username))
   to: [ "[email protected],[email protected]", "[email protected],[email protected]" ]
# parameterized would be:
#    to:
#      - ((email-list-developers))
#      - ((email-list-po))

Where we can specify two different lists in the params files (e.g. email-high-priority, email-high-priority, or maybe email-fail-list, email-success-list), etc.

Hi,
our pipeline is using email-resource and we are observing some breaking behavior since the latest release.

When e-mail body is empty, our job now unexpectedly fails (with the error "Message not sent because the message body is empty and send_empty_body parameter was set to false. Github readme: https://github.com/pivotal-cf/email-resource").

Seems this error output used to simply be emitted, but now bubbles up and triggers the fail in this change:
393b32a#diff-095c0632d71d85e648abdfd1442cb59fR143

However, we don't want to set "send_empty_body" to true, as receiving empty e-mails would not only be useless but annoying; the behavior we have been relying on is that no e-mail is sent, and the job should not fail.

I have a concourse with a great many build jobs on it.
Many of my build names have spaces in them.
I use the email-resource to send emails on failure.

I would like to be able to link to the build in question in the case of failure.
My email step looks like this (based on the example in the project README):

    send-failure-email: &send-failure-email
      put: email
      params:
        subject_text: "MyProject ${BUILD_JOB_NAME}: build ${BUILD_ID} failed!"
        body_text: |
          ${BUILD_NAME} failed!
          Build ID: ${BUILD_ID}
          Build Name: ${BUILD_NAME}
          Build Job Name: ${BUILD_JOB_NAME}
          Build Pipeline Name: ${BUILD_PIPELINE_NAME}
          ATC External URL: ${ATC_EXTERNAL_URL}
          Direct link: ${ATC_EXTERNAL_URL}/${BUILD_PIPELINE_NAME}/jobs/${BUILD_JOB_NAME}/builds/${BUILD_NAME}

This produces broken links when the build name has a space in it.

How can fix this?

I can't see any way to URL-escape the "BUILD_JOB_NAME" when building the email, because the code which inserts the value from the Concourse metadata does a simple find-and-replace and has no scripting ability:

I can't see any way to URL-escape the build name in a previous step in the build, because as far as I can tell, there is no mechanism in Concourse to pass an environment variable into the email resource, is there?

Perhaps there is no way to fix this for a user, so this is a feature request for Concourse to add a new build "meta data" param which is the URL of the job, correctly escaped?

I'd like to send an email with a file attached from concourse pipeline and I faced below error.
Error during execute - unmarshalling input: json: cannot unmarshal string into Go struct field Params.params.attachment_globs of type []string

I tried below pipeline.

    on_failure:
      do:
        - put: send-an-email
          params:
            subject_text: "Build finished: ${BUILD_PIPELINE_NAME}/${BUILD_JOB_NAME}/${BUILD_NAME}"
            body_text: "Build failed: ${ATC_EXTERNAL_URL}/teams/main/pipelines/${BUILD_PIPELINE_NAME}/jobs/${BUILD_JOB_NAME}/builds/${BUILD_NAME}"
            attachment_globs: repository/close_file_attach.txt

Please help me out with this.

I was trying to setup email notifications using this resource and following the steps from readme but I am getting the following error:

v1.0.13-b83fdf4d8c967bce40e77019a19a0c451a59e3c6 - EOF

I also tried using demo-pipeline.yml from the examples and still got the same error. I cannot really debug this as I'm unable to intercept into this resource. Concourse version I am running is v3.14.1. Any help is appreciated.

Hi Team,

I am trying to use “email-resource” over ARM64v8 but it seems it does not have an arm64 supported tag in docker hub. I have successfully built binaries using the following steps:

1. go build -o bin/check check/cmd/*.go
2. go build -o bin/in in/cmd/*.go
3. go build -o bin/out out/cmd/*.go

I am successfully able to create an arm64 image using the available Dockerfile .

ARM64 Image: https://hub.docker.com/repository/docker/odidev/email-resource_arm64

I have tested email-resource successfully on arm64 platform using command “go test $(glide nv)”.

Do you have any plans for releasing an ARM64 image?

Currently, we can build this image directly on arm64 without any code changes. It will be helpful if the arm64 image is released.

@calebwashburn Thank you very much for reviving this repository. Good luck to you and the awesome Concourse project!

REASON550 5.7.1 [167.89.13.30 11] Our system has detected that this message is not RFC 5322 compliant: 'From' header is missing. To reduce the amount of spam sent to Gmail, this message has been blocked. Please visit https://support.google.com/mail/?p=RfcMessageNonCompliant and review RFC 5322 specifications for more information. REDACTED - gsmtp SMTP-IDPROCESSED STRINGSeptember 14, 2016 - 02:13:36 REDACTED

We noticed that sending an email message using a traditional SMTP server causes the server to block the requests as the body doesn't contain the default To and From headers needed to be RFC 5322 compliant.

From the godocs for smtp library
https://golang.org/pkg/net/smtp/#SendMail
The msg parameter should be an RFC 822-style email with headers first, a blank line, and then the message body.

v1.0.14-53e61dda6f626d66267b4890c0653cc87f728881 - EOF

ranmanda-Mac:gated-pipelines ranmanda$ fly -t tutorial trigger-job -w -j shipit/send-email
started shipit/send-email #4

initializing
running sh -exc # ensure you esape the ${BUILD_ID} variable with leading
echo -e "Email resource demo on $(date): build ${BUILD_ID}" > email-out/${OUTPUT_SUBJECT_FILE_NAME}
echo -e "Cheers!\n\n
Build ID: ${BUILD_ID} \n
Build Name: ${BUILD_NAME} \n
Build Job Name: ${BUILD_JOB_NAME} \n
Build Pipeline Name: ${BUILD_PIPELINE_NAME} \n
ATC External URL: ${ATC_EXTERNAL_URL}" > email-out/${OUTPUT_BODY_FILE_NAME}

• date
• echo -e Email resource demo on Thu Feb 28 22:08:33 UTC 2019: build ${BUILD_ID}
• echo -e Cheers!\n\n Build ID: ${BUILD_ID} \n Build Name: ${BUILD_NAME} \n Build Job Name: ${BUILD_JOB_NAME} \n Build Pipeline Name: ${BUILD_PIPELINE_NAME} \n ATC External URL: ${ATC_EXTERNAL_URL}
  v1.0.14-53e61dda6f626d66267b4890c0653cc87f728881 - EOF
  failed

Hi,

When trying to debug the put step for sending an email, I found that the debug parameter should be provided as a string; other boolean parameters like send_empty_body work as expected.

E.g. when I configure:

  subject: email-text/email-subject
  body: email-text/email-body
  send_empty_body: false
  debug: true

I get the following error when the pipeline runs:

v1.0.19-1f211315c12fd17cddd92c03843301640aca9b11 - Error during execute - unmarshalling input: json: cannot unmarshal bool into Go struct field Params.params.debug of type string

I think it is caused by line 39 in out/out.go
debug := strings.EqualFold("true", params.Debug)

When I change my pipeline to the following, it does not give errors:

  subject: email-text/email-subject
  body: email-text/email-body
  send_empty_body: false
  debug: "true"

Current documentation says: "You can use the values below in any of the source files or text properties to access the corresponding metadata made available by concourse".

It would be nice if the concourse metadata (such as $BUILD_JOB_NAME) was also made available in the "body" property, to be used like this, for example:

    on_success:
      try:
        put: email
        params:
          subject_text: "${BUILD_JOB_NAME} (${BUILD_NAME})"
          body: uploads/${BUILD_JOB_NAME}.log

Currently, no variable expansion takes place within the body property.

For me it is an essential use case for email resource to send an email in case of a build failure. Could you provide an example of how to do so in the examples/ directory?

It look like Christian Joergensen has remove all his code from Bitbucket.org as I am receiving a Repository not found.

How can I use this resource to add attachments generated in a task to an email?

We have switched our Concourse testing instance over to containerd runtime for the workers.
The only thing that broke has been the email notifications.

During the check step, the following error is reported through Concourse check UI:

run check step: run check step: find or create container: starting task: new task: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "/tmp/gdn-init": stat /tmp/gdn-init: no such file or directory: unknown

This behaviour could be spotted throughout all pipelines consistently. We switched over to another email resource for the meantime, but would rather like to switch back as this resource feels better maintained.