Giter VIP home page Giter VIP logo

notify-service's Introduction

Flake8 Status Helm Chart Lint and Test License: MIT

notify-service

Notify service with multiple supported target

This is a backend service build with FastAPI.

Requirement

  • Python 3.9
  • Pipenv
  • Podman, buildah on Linux host
  • Helm 3
  • Redis

Check Pipfile for python packages.

Deploy with Helm

IMAGE ALT TEXT HERE

Update Helm chart value

Make a copy of the chart/values.yaml, e.g. dev.value.yaml and update.

Update api key:

apiKeyValue: The API key to authenticate and access the apis, you could generate one with cmd `$openssl rand -hex 32`

SMTP related config:

smtp.endpoint.enableTls: default to True
smtp.endpoint.port: int, required
smtp.endpoint.host: str, required
smtp.endpoint.user: str, optional
smtp.endpoint.password: str, optional

Email related config:

fromName: str, service name, e.g. Notify Service
fromEmail: str, the email from

Chat Webhook URL:

chatWebhook.enabled: boolen, enable or disable chat webhook
chatWebhook.gchat: str, Google Chat room webhook
chatWebhook.slack: str, Slack room webhook

ActiveMQ message bus:

activeMQ.enabled: boolen, enable or disable ActiveMQ message bus config
activeMQ.cert_mount_path: str, specify where the certs will be mounted in the container
activeMQ.ca_certs: multiline str, the CA server certificate
activeMQ.client_cert: multiline str, the message bus client certificate
activeMQ.client_key: multiline str, the message bus client key
activeMQ.msg_bus_host_1: str, the message bus host name
activeMQ.msg_bus_host_1: int, the message bus host port
activeMQ.msg_bus_host_2: str, optional, the second message bus host name
activeMQ.msg_bus_host_2: int, optional, the second message bus host port

IRC config:

irc.enabled: boolen, enable or disable IRC config
irc.server: str, IRC host name
irc.port: int, the IRC host port
irc.ssl: boolen, enable or disable ssl, make sure set the right port if is enabled
irc.nickname: str, IRC username
irc.password: str, set if password exist
irc.default_target: str, the default channel name start with '#' or username

Jira config:

jira.enabled: boolen, enable or disable Jira config
jira.issue_type_list: list of str, include all the available Jira issue types
jira.url: str, the Jira instance url
jira.token: str, the user's personal Jira token

Bugzilla config:

bugzilla.enabled: boolen, enable or disable Bugzilla config
bugzilla.url: str, the Bugzilla instance url
bugzilla.api_key: str, the user's personal Bugzilla API key

Persistent volume config:

persistence.enabled: boolen, true or false
persistence.subPath: str, sub path
persistence.existingClaim, str, using existing pvc
persistence.accessMode: str, ReadWriteOnce, ReadWriteMany or ReadOnlyOnce
persistence.size: str, volume size, e.g. 1G
persistence.storageClass: str, storage class name
persistence.VolumeName: str, volume name

If deploy to Openshift (default) update OpenShift route url:

openshift.enabled: boolen, true or false
openshift.hosts: str, the full app host name

or else update the ingress part.

Run helm install

Make sure you have login your cluster.

Install latest from helm repo

Add helm repo with:

$ helm repo add notify https://RedHatQE.github.io/notify-service/

Check the latest release on the project with:

$ helm search repo notify-service

Then install with:

$ helm install ns notify-service -f dev.value.yaml

Install from current repo:

$ helm install ns chart/ -f dev.value.yaml

Access the service and API docs

The API docs are automatically generated on Swagger UI and Redoc UI.

Swagger UI

After deployment is done access the app Swagger UI:

http://${ openshift.hosts }:8080/docs

You could try the apis on Swagger UI and check parameters and request body samples in description.

Redoc

After the deployment is done, Redoc UI could be accessed at:

http://${ openshift.hosts }:8080/redoc

The Redoc UI is more developer friendly with detail on descriptins, parameters, schemas, payload, code samples, etc.

Request body template and samples

For each API, both Swagger UI and Redoc UI provide request body schema, while Redoc UI with more details and provide drop list with different supported schemas and details.

Check sample dir under doc for some request body with the matching template names under app/templates/build or app/templates/src.

Templates

The repo have provided few templates under app/templates/, which includes templates for email, Google Chat, Slack, and Jira.

All templates are Jinja templates. Check following for how each target templates are generated, remember to add new request body sample under sample dir for each templates.

Email MJML template

The email templates are generated with MJML, you could create a mjml template online with try-it-live.

After edit done, you could save the mjml file with suffix .mjml under app/templates/src, and SAVE THE HTML file with suffix '.html' under app/templates/build, the HTML file will be directly used as email template. Then you could raise PR for adding new templates, make sure your template is unique.

Note: Use Jinja semantic for templating.

Google Chat template

Google chat message support simple text and cards:

  • Simple text contains plain text content with limited text formatting.
  • Cards define the format, content, and behavior of cards to be displayed in the target space.

Follow the docs and create your own google chat templates and save the template file with suffix '.jinja' under app/templates/build. Then you could raise PR for adding new templates, make sure your template is unique.

Note: Use Jinja semantic for templating.

Slack template

Slack message use mrkdwn formatting syntax and support layouts, check:

Follow the docs and create your own slack templates and save the template file with suffix '.jinja' under app/templates/build. Then you could raise PR for adding new templates, make sure your template is unique.

Note: Use Jinja semantic for templating.

Jira template

Jira issues and comments use wikimarkup formatting syntax check:

Follow the docs and create your own Jira templates and save the template file with suffix '.jinja' under app/templates/build. Then you could raise PR for adding new templates, make sure your template is unique.

Note: Use Jinja semantic for templating. Advanced Fields (Creating a new issue): In order to specify Components, Labels, Affect Versions, and Fix Versions when you create a new issue on Jira, please check under app/templates/sample the file jira_new_issue_extra_fields.jinja. Adding the fields to any Jira sample will allow you to specify Components, Labels, Affect Versions, and Fix Versions under your new issue.

Bugzilla template

Bugzilla comments templates (in the current Bugzilla version) only support basic text and hyper-links:

Follow the docs and create your own Bugzilla templates and save the template file with suffix '.jinja' under app/templates/build. Then you could raise PR for adding new templates, make sure your template is unique.

Note: Use Jinja semantic for templating.

Upload templates

No need to raise PRs for each template, if the template is not for common use for all users, user could choose upload templates to a running instance use the Update Tempalte API. Check on the Swagger UI or Redoc UI with PUT method under template APIs on a running instance.

The templates are saved on a volume mounted to dir specified as templateMountDir in helm chart, make sure persistence.enabled is true. Then the templates will be saved in a persistent volumn on your OCP cluster.

TODO: Supprt S3 storage

Remote url templates

User could also store templates on a github repo or other places which could be accessed by url, then could specify the template url in email, gchat, slack target apis.

For remote urls, cache with default 300 secs timeout will be enabled, so no fetching if request same template between 300 secs.

Note: Make sure you have matching request body for Jinja templating.

Development

For develop, build, test and debug, please check Development Doc for more info.

Helm Chart Release

For update and release the Helm chart of the repo to github webpage, please check Helm Chart Release Doc for more info.

Container Images

The latest build images are published on: https://quay.io/repository/waynesun09/notify-service

Contributing

You can contribute by:

  • Raising any issues you find using notify-service
  • Fixing issues by opening Pull Requests
  • Submitting a patch or opening a PR
  • Improving documentation
  • Talking about notify-service

All bugs, tasks or enhancements are tracked as GitHub issues.

CI

The Github Action will run flake8 against .py files, pytest and helm chart test.

notify-service's People

Contributors

dependabot[bot] avatar idob123 avatar idoburstein avatar waynesun09 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

notify-service's Issues

Helm upgrade hang with volumn could not be attached

If ReadWriteMany volume not supported, helm upgrade will hang with ReadWriteOnce volumn can't be attached to new pod.

Default helm update strategy with spec:

type: RollingUpdate

Recommend use ReadWriteMany volume if available, else user might need manually delete old replicaset to delete old pods and preserve existing templates data saved in the volume.

Need to update teflo_scenario_email template

The current teflo_Scenario_email template is missing some information
Following should be modified

https://github.com/waynesun09/notify-service/blob/main/app/templates/build/teflo_scenario_email.html#L221 > failed tasks is not printed instead passed tasks is preinted again

https://github.com/waynesun09/notify-service/blob/main/app/templates/build/teflo_scenario_email.html#L202-L208 > subject is not a part of body, so it is not getting printed

the scenario name needs to be added {{ scenario.name }}

https://github.com/waynesun09/notify-service/blob/main/app/templates/build/teflo_scenario_email.html#L224-L231 the logic to derive executes does not work.
It should be similar to what is present in the gchat template

Also need to add details for scenario.reports

Thread loop when send message to message bus failed

When send message to message bus without permission, the stomp client will retry and run into endless loop as the wait=True set in connect(), the error log will flooding as:

172.20.8.1:34246 - "POST /api/v1/message_bus/?topic=%2Ftopic%2FVirtualTopic.qe.ci.brew-build.test.complete HTTP/1.1" 500
RECEIVED AN ERROR.
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Message headers:
{'content-type': 'text/plain', 'message': 'PlatformSecurityAuthorizationBrokerPlugin - User xxxxxxxxx-xxxxx (roles: xxxx-xxxx,xxxx-xxxx) is not authorized to create topic://ActiveMQ.Advisory.Connection (authorized roles: umbAllEnv,admin,admins)'}
Message body:
java.lang.SecurityException: PlatformSecurityAuthorizationBrokerPlugin - User xxxxxxxxx-xxxxx (roles: xxxx-xxxx,xxxx-xxxx) is not authorized to create topic://ActiveMQ.Advisory.Connection (authorized roles: umbAllEnv,admin,admins)
	at com.redhat.api.platform.security.amq.PlatformSecurityAuthorizationBrokerPlugin$PlatformSecurityAuthorizationBrokerFilter.verifyOperation(PlatformSecurityAuthorizationBrokerPlugin.java:281)
	at com.redhat.api.platform.security.amq.PlatformSecurityAuthorizationBrokerPlugin$PlatformSecurityAuthorizationBrokerFilter.addDestination(PlatformSecurityAuthorizationBrokerPlugin.java:329)
	at org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:173)
	at org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:173)
	at com.redhat.api.platform.umb.core.plugins.ResourceQuotaBroker.addDestination(ResourceQuotaBroker.java:80)
	at org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:178)
	at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:480)
	at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:296)
	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:888)
	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:820)
	at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:815)
	at org.apache.activemq.advisory.AdvisoryBroker.addConnection(AdvisoryBroker.java:124)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at com.redhat.api.platform.umb.core.plugins.AccessLoggingBroker.addConnection(AccessLoggingBroker.java:75)
	at org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:103)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at com.redhat.api.platform.security.amq.PlatformSecurityAuthenticationBrokerPlugin$PlatformSecurityAuthenticationBrokerFilter.addConnection(PlatformSecurityAuthenticationBrokerPlugin.java:68)
	at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)
	at com.redhat.api.platform.umb.core.plugins.ResourceQuotaBroker.addConnection(ResourceQuotaBroker.java:64)
	at org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:103)
	at org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:837)
	at org.apache.activemq.broker.jmx.ManagedTransportConnection.processAddConnection(ManagedTransportConnection.java:77)
	at org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)
	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:321)
	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:190)
	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:45)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:301)
	at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:97)
	at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:202)
	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompConnect(ProtocolConverter.java:774)
	at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:265)
	at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:85)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:166)
	at org.apache.activemq.transport.stomp.StompSslTransportFactory$1$1.doConsume(StompSslTransportFactory.java:70)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
	at java.lang.Thread.run(Thread.java:748)

    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/.venv/lib/python3.8/site-packages/stomp/transport.py", line 362, in __receiver_loop
    self.notify("receiver_loop_completed")
  File "/.venv/lib/python3.8/site-packages/stomp/transport.py", line 242, in notify
    rtn = notify_func(headers, body)
  File "/app/utils/message_bus.py", line 65, in on_receiver_loop_completed
    self._connect()
  File "/app/utils/message_bus.py", line 81, in _connect
    self.conn.connect(wait=True)
  File "/.venv/lib/python3.8/site-packages/stomp/connect.py", line 162, in connect
    Protocol11.connect(self, *args, **kwargs)
  File "/.venv/lib/python3.8/site-packages/stomp/protocol.py", line 335, in connect
    self.transport.wait_for_connection()
  File "/.venv/lib/python3.8/site-packages/stomp/transport.py", line 322, in wait_for_connection
    raise exception.ConnectFailedException()
stomp.exception.ConnectFailedException

this might related to:
jasonrbriggs/stomp.py#323

To workaround this, could update connect with wait=False, but that will disable retry.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.