The rabbit-amazon-forwarder from airhelp

Getting Failed to declare a queue error

When i execute docker-compose up i am getting the following errors

rabbitamazonforwarder_1 | {"level":"error","msg":"Failed to declare a queue:scheduled_posts: Exception (406) Reason: "PRECONDITION_FAILED - inequivalent arg 'x-dead-letter-exchange' for queue 'scheduled_posts' in vhost 'tweebr_dev': received the value 'scheduled_posts-dead-letter' of type 'longstr' but current is none"","time":"2019-10-30T07:22:49Z"}

rabbitamazonforwarder_1 | {"error":"Exception (504) Reason: "channel/connection is not open"","level":"error","msg":"Could not close channel","time":"2019-10-30T07:22:49Z"}

Typo

Hello. Firstly, great work!

I'm here just to tell you about a nitpick in your repository's description. 😄

[DepShield] (CVSS 7.5) Vulnerability due to usage of golang.org/x:net:0.0.0-20180906233101-161cd47e91fd

Vulnerabilities

DepShield reports that this application's usage of golang.org/x:net:0.0.0-20180906233101-161cd47e91fd results in the following vulnerability(s):

(CVSS 7.5) [CVE-2018-17847] Improper Input Validation
(CVSS 7.5) [CVE-2018-17142] Improper Input Validation
(CVSS 7.5) [CVE-2018-17846] Resource Management Errors
(CVSS 7.5) [CVE-2018-17075] Improper Input Validation
(CVSS 7.5) [CVE-2018-17848] Data Handling
(CVSS 7.5) [CVE-2018-17143] Improper Input Validation

Occurrences

golang.org/x:net:0.0.0-20180906233101-161cd47e91fd is a transitive dependency introduced by the following direct dependency(s):

• github.com/onsi:ginkgo:1.16.5
        └─ github.com/onsi:gomega:1.10.1
              └─ github.com/onsi:ginkgo:1.12.1
                    └─ github.com/onsi:gomega:1.7.1
                          └─ golang.org/x:net:0.0.0-20180906233101-161cd47e91fd

• github.com/onsi:gomega:1.19.0
        └─ github.com/onsi/ginkgo:v2:2.1.3
              └─ github.com/onsi:gomega:1.17.0
                    └─ github.com/onsi:ginkgo:1.16.4
                          └─ github.com/onsi:gomega:1.10.1
                                └─ github.com/onsi:ginkgo:1.12.1
                                      └─ github.com/onsi:gomega:1.7.1
                                            └─ golang.org/x:net:0.0.0-20180906233101-161cd47e91fd

_{This is an automated GitHub Issue created by Sonatype DepShield. Details on managing GitHub Apps, including DepShield, are available for personal and organization accounts. Please submit questions or feedback about DepShield to the Sonatype DepShield Community.}

[DepShield] (CVSS 7.5) Vulnerability due to usage of golang.org/x:net:0.0.0-20200520004742-59133d7f0dd7

Vulnerabilities

DepShield reports that this application's usage of golang.org/x:net:0.0.0-20200520004742-59133d7f0dd7 results in the following vulnerability(s):

(CVSS 7.5) [CVE-2018-17847] Improper Input Validation
(CVSS 7.5) [CVE-2018-17142] Improper Input Validation
(CVSS 7.5) [CVE-2018-17846] Resource Management Errors
(CVSS 7.5) [CVE-2018-17075] Improper Input Validation
(CVSS 7.5) [CVE-2018-17848] Data Handling
(CVSS 7.5) [CVE-2018-17143] Improper Input Validation

Occurrences

golang.org/x:net:0.0.0-20200520004742-59133d7f0dd7 is a transitive dependency introduced by the following direct dependency(s):

• github.com/onsi:ginkgo:1.16.5
└─ github.com/onsi:gomega:1.10.1
└─ golang.org/x:net:0.0.0-20200520004742-59133d7f0dd7

• github.com/onsi:gomega:1.19.0
        └─ github.com/onsi/ginkgo:v2:2.1.3
              └─ github.com/onsi:gomega:1.17.0
                    └─ github.com/onsi:ginkgo:1.16.4
                          └─ github.com/onsi:gomega:1.10.1
                                └─ golang.org/x:net:0.0.0-20200520004742-59133d7f0dd7

_{This is an automated GitHub Issue created by Sonatype DepShield. Details on managing GitHub Apps, including DepShield, are available for personal and organization accounts. Please submit questions or feedback about DepShield to the Sonatype DepShield Community.}

[DepShield] (CVSS 7.5) Vulnerability due to usage of golang.org/x:net:0.0.0-20190404232315-eb5bcb51f2a3

Vulnerabilities

DepShield reports that this application's usage of golang.org/x:net:0.0.0-20190404232315-eb5bcb51f2a3 results in the following vulnerability(s):

(CVSS 7.5) [CVE-2018-17847] Improper Input Validation
(CVSS 7.5) [CVE-2018-17142] Improper Input Validation
(CVSS 7.5) [CVE-2018-17846] Resource Management Errors
(CVSS 7.5) [CVE-2018-17075] Improper Input Validation
(CVSS 7.5) [CVE-2018-17848] Data Handling
(CVSS 7.5) [CVE-2018-17143] Improper Input Validation

Occurrences

golang.org/x:net:0.0.0-20190404232315-eb5bcb51f2a3 is a transitive dependency introduced by the following direct dependency(s):

• github.com/onsi:ginkgo:1.16.5
        └─ golang.org/x:tools:0.0.0-20201224043029-2b0845dc783e
              └─ golang.org/x:mod:0.3.0
                    └─ golang.org/x:crypto:0.0.0-20191011191535-87dc89f01550
                          └─ golang.org/x:net:0.0.0-20190404232315-eb5bcb51f2a3
              └─ golang.org/x:net:0.0.0-20201021035429-f5854403a974
                    └─ golang.org/x:crypto:0.0.0-20200622213623-75b288015ac9
                          └─ golang.org/x:net:0.0.0-20190404232315-eb5bcb51f2a3

• golang.org/x:crypto:0.0.0-20200622213623-75b288015ac9
└─ golang.org/x:net:0.0.0-20190404232315-eb5bcb51f2a3

_{This is an automated GitHub Issue created by Sonatype DepShield. Details on managing GitHub Apps, including DepShield, are available for personal and organization accounts. Please submit questions or feedback about DepShield to the Sonatype DepShield Community.}

[DepShield] (CVSS 7.5) Vulnerability due to usage of golang.org/x:net:0.0.0-20190620200207-3b0461eec859

Vulnerabilities

DepShield reports that this application's usage of golang.org/x:net:0.0.0-20190620200207-3b0461eec859 results in the following vulnerability(s):

(CVSS 7.5) [CVE-2018-17847] Improper Input Validation
(CVSS 7.5) [CVE-2018-17142] Improper Input Validation
(CVSS 7.5) [CVE-2018-17846] Resource Management Errors
(CVSS 7.5) [CVE-2018-17075] Improper Input Validation
(CVSS 7.5) [CVE-2018-17848] Data Handling
(CVSS 7.5) [CVE-2018-17143] Improper Input Validation

Occurrences

golang.org/x:net:0.0.0-20190620200207-3b0461eec859 is a transitive dependency introduced by the following direct dependency(s):

• github.com/onsi:ginkgo:1.16.5
        └─ golang.org/x:tools:0.0.0-20201224043029-2b0845dc783e
              └─ golang.org/x:mod:0.3.0
                    └─ golang.org/x:tools:0.0.0-20191119224855-298f0cb1881e
                          └─ golang.org/x:net:0.0.0-20190620200207-3b0461eec859

_{This is an automated GitHub Issue created by Sonatype DepShield. Details on managing GitHub Apps, including DepShield, are available for personal and organization accounts. Please submit questions or feedback about DepShield to the Sonatype DepShield Community.}

INFO: What is the rationale behind setupExchangesAndQueues

setupExchangesAndQueues makes a lot of assumptions about the setup of the queues and naming conventions.

For instance that a queue has a a dead-letter exchange or queue and that it is exactly named -dead-letter.

	// regular queue
	if _, err = ch.QueueDeclare(c.QueueName, true, false, false, false,
		amqp.Table{
			"x-dead-letter-exchange": deadLetterExchangeName,
		}); err != nil {
		return failOnError(err, "Failed to declare a queue:"+c.QueueName)
	}

Assuming an existing queue with a dead letter exchange other than -dead-letter (for instance mine is already called -dlx)...the above will fail.

I might be missing something, but couldn't this rather be done via config?

zb.

[LAMBDA] Ack

What is the rationale for the acknowledge of a message?

My use scenario considers intermitent connectivity :

Process puts a message to a fanout queue with certain routing key
When there is connectivity the messages should be sent to lambda (only ACK when delivered)
Lambda function executes some logic based on the content body.

Request for example for multiple destinations

Is it possible to route multiple queues to multiple destinations with a single instance of Rabbit-Amazon-Forwarder?

If so, how do we go about configuring this in the mapping file?

Taking a guess, I started off with trying a single queue to multiple destinations by wrapping each destination in an array as follows:

[
  {
    "source" : {
      "type" : "RabbitMQ",
      "name" : "my-service-name",
      "connection" : "amqp://username:password@hostname:5672/",
      "topic" : "",
      "queue" : "",
      "routingKeys" : ["my-key"]
    },
    "destination" : [
      {
        "type" : "Lambda",
        "name" : "one-dest",
        "target" : "one-dest"
      },
      {
        "type" : "Lambda",
        "name" : "two-dest",
        "target" : "two-dest"
      }
    ]
  }
]

However, I got an error out saying that Go was unable to parse the array into a single field.

Thanks in advance!

Code does not work with Transient Exchange

The code in consumer.go has exchange state hard-coded as durable. This prevents the code from connecting to transient exchanges.

Lambda InvocationType

Hi,
Is it possible to add another (optional) parameter in the configuration, which is for Lambda InvocationType?
It is about this fragment of the code (/lambda/forwarder.go):

// Push pushes message to forwarding infrastructure
func (f Forwarder) Push(message string) error {

	if message == "" {
		return errors.New(forwarder.EmptyMessageError)
	}
	params := &lambda.InvokeInput{
		FunctionName: aws.String(f.function),
		Payload:      []byte(message),
		InvocationType: aws.String("Event"),
	}
	resp, err := f.lambdaClient.Invoke(params)

Of course, instead of aws.String("Event") there should be an option from the file /config/mapping.json

I think it would be a useful option

Regards

Add Helm chart

Requirements:

Helm chart for installation
description in README and samples

TLS Support

Heya,

I was looking at this tool earlier today. Have you considered adding support for TLS and client authentication on the rabbit side of things?

Multiple routing keys

Hey guys! It would be really handy if we could specify multiple routing keys 👍

[DepShield] (CVSS 5.9) Vulnerability due to usage of golang.org/x:crypto:0.0.0-20190308221718-c2843e01d9a2

Vulnerabilities

DepShield reports that this application's usage of golang.org/x:crypto:0.0.0-20190308221718-c2843e01d9a2 results in the following vulnerability(s):

(CVSS 5.9) [CVE-2019-11840] Use of Insufficiently Random Values

Occurrences

golang.org/x:crypto:0.0.0-20190308221718-c2843e01d9a2 is a transitive dependency introduced by the following direct dependency(s):

• golang.org/x:crypto:0.0.0-20200622213623-75b288015ac9
└─ golang.org/x:net:0.0.0-20190404232315-eb5bcb51f2a3
└─ golang.org/x:crypto:0.0.0-20190308221718-c2843e01d9a2

_{This is an automated GitHub Issue created by Sonatype DepShield. Details on managing GitHub Apps, including DepShield, are available for personal and organization accounts. Please submit questions or feedback about DepShield to the Sonatype DepShield Community.}

Is there possibility to setup forwarder to work with localstack?

HI. I want use your forwarder to work locally with a localstack. Is it possible?

My docker-compose env looks like this.

# This won't be used by CI/CD. This is just for local development

version: '3.8'

services:
  localstack:
    image: localstack/localstack-pro
    container_name: localstack
    networks:
      - internal-net
   ...

  rabbitmq-forwarder:
    image: airhelp/rabbit-amazon-forwarder:latest
    networks:
      - internal-net
    environment:
      AWS_REGION: eu-central-1
      MAPPING_FILE: /config/mapping.json
      AWS_ACCESS_KEY_ID: <MY_DUMMY_KEY_ID>
      AWS_SECRET_ACCESS_KEY: <MY_DUMMY_ACCESS_KEY>
      AWS_ENDPOINT_URL: http://localstack:4566
    depends_on:
      - rabbitmq
      - localstack
    volumes:
      - "./rabbitmq-forwarder/mapping.json:/config/mapping.json"

But when I send an event I've got the following error

{
    "error": "UnrecognizedClientException: The security token included in the request is invalid.\n\tstatus code: 403, request id: c9d1f1c4-7034-459e-9654-38b7d1639fed",
    "forwarderName": "test-lambda",
    "level": "error",
    "msg": "Could not forward message","time":"2023-12-07T07:47:20Z"
}

Thanks for all. Any help will be appreciated
Best regards.
Draqun

airhelp / rabbit-amazon-forwarder Goto Github PK

rabbit-amazon-forwarder's Introduction

RabbitMQ -> Amazon forwarder

Architecture

Configuration

Mapping file

Environment variables

Using TLS with rabbit

Amazon configuration

Build docker image

Run

Test

Release

Supervisor

rabbit-amazon-forwarder's People

Contributors

Stargazers

Watchers

Forkers

rabbit-amazon-forwarder's Issues

Recommend Projects

Recommend Topics

Recommend Org