Giter VIP home page Giter VIP logo

sygnal's Introduction

Introduction

Sygnal is a reference Push Gateway for Matrix.

See https://spec.matrix.org/latest/push-gateway-api/#overview for a high level overview of how notifications work in Matrix.

The Matrix Specification describes the protocol that Matrix Home Servers use to send notifications to Push Gateways such as Sygnal.

Contributing

Looking to contribute to Sygnal? See CONTRIBUTING.md

Setup

Sygnal is configured through a YAML configuration file. By default, this configuration file is assumed to be named sygnal.yaml and to be in the working directory. To change this, set the SYGNAL_CONF environment variable to the path to your configuration file. A sample configuration file is provided in this repository; see sygnal.yaml.sample.

The apps: section is where you set up different apps that are to be handled. Each app should be given its own subsection, with the key of that subsection being the app's app_id. Keys in this section take the form of the app_id, as specified when setting up a Matrix pusher (see https://spec.matrix.org/latest/client-server-api/#post_matrixclientv3pushersset).

See the sample configuration for examples.

You can find a docker image for sygnal on DockerHub.

App Types

There are two supported App Types:

apns

This sends push notifications to iOS apps via the Apple Push Notification Service (APNS).

The expected configuration depends on which kind of authentication you wish to use.

For certificate-based authentication, it expects:

  • the certfile parameter to be a path relative to sygnal's working directory of a PEM file containing the APNS certificate and unencrypted private key.

For token-based authentication, it expects:

  • the keyfile parameter to be a path relative to Sygnal's working directory of a p8 file
  • the key_id parameter
  • the team_id parameter
  • the topic parameter, which is most commonly the 'Bundle Identifier' for your iOS application

For either type, it can accept:

  • the platform parameter which determines whether the production or sandbox APNS environment is used. Valid values are 'production' or 'sandbox'. If not provided, 'production' is used.
  • the push_type parameter which determines what value for the apns-push-type header is sent to APNs. If not provided, the header is not sent.
  • the convert_device_token_to_hex parameter which determines if the token provided from the client is b64 decoded and converted to hex. Some client libraries already provide the token in hex, and this should be set to False if so.

gcm

This sends messages via Google/Firebase Cloud Messaging (GCM/FCM) and hence can be used to deliver notifications to Android apps.

The expected configuration depends on which version of the firebase api you wish to use.

For legacy API, it expects:

  • the api_key parameter to contain the Server key, which can be acquired from Firebase Console at: https://console.firebase.google.com/project/<PROJECT NAME>/settings/cloudmessaging/

For API v1, it expects:

  • the api_version parameter to contain v1
  • the project_id parameter to contain the Project ID, which can be acquired from Firebase Console at: https://console.cloud.google.com/project/<PROJECT NAME>/settings/general/
  • the service_account_file parameter to contain the path to the service account file, which can be acquired from Firebase Console at: https://console.firebase.google.com/project/<PROJECT NAME>/settings/serviceaccounts/adminsdk

Using an HTTP Proxy for outbound traffic

Sygnal will, by default, automatically detect an HTTPS_PROXY environment variable on start-up.

If one is present, it will be used for outbound traffic to APNs and GCM/FCM.

Currently only HTTP proxies with the CONNECT method are supported. (Both APNs and FCM use HTTPS traffic which is tunnelled in a CONNECT tunnel.)

If you wish, you can instead configure a HTTP CONNECT proxy in sygnal.yaml.

Pusher data configuration

The following parameters can be specified in the data dictionary which is given when configuring the pusher via POST /_matrix/client/v3/pushers/set:

  • default_payload: a dictionary which defines the basic payload to be sent to the notification service. Sygnal will merge information specific to the push event into this dictionary. If unset, the empty dictionary is used.

    This can be useful for clients to specify default push payload content. For instance, iOS clients will have freedom to use silent/mutable notifications and be able to set some default alert/sound/badge fields.

Running

With default configuration file name of sygnal.yaml:

python -m sygnal.sygnal

With custom configuration file name:

SYGNAL_CONF=/path/to/custom_sygnal.conf python -m sygnal.sygnal

Python 3.8 or higher is required.

Log Rotation

Sygnal's logging appends to files but does not use a rotating logger. The recommended configuration is therefore to use logrotate. The log file will be automatically reopened if the log file changes, for example due to logrotate.

More Documentation

More documentation for Sygnal is available in the docs directory:

sygnal's People

Contributors

abeluck avatar anoadragon453 avatar ara4n avatar benbz avatar bwindels avatar callahad avatar clokep avatar csett86 avatar dbkr avatar dependabot[bot] avatar devonh avatar dklimpel avatar erikjohnston avatar evilham avatar h-shay avatar half-shot avatar hawkowl avatar ismailgulek avatar kibablu avatar kittykat avatar krombel avatar leonhandreke avatar matmaul avatar michaelkaye avatar morozkin avatar negativemjark avatar reivilibre avatar richvdh avatar squahtx avatar tawandamoyo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sygnal's Issues

apns requests seem to stack up forever on connection failure

When our TLS certificate expired, so that all apns requests were unsuccessful, the "requests in flight" and "apns requests in flight" metrics went up seemingly without limit:

image

I would expect that we retry each request a few (well, 12) times, and then give up (see also #116), so I'd expect requests to stack up a bit, but not to this extent.

We ended up with a 5G RSZ, so I don't think this is just a problem with the metrics.

[BUG] Problem with images

In apnpushkin.py exist a series of four if-else-if blocks, but only can enter in the first two, this because the fallback in the top in the bottom image. This fallback code, set the content_display = n.content["body"] and the else if is_image and the last else is never reached.
screen shot 2017-01-06 at 11 38 14 am

So...

IMAGE_FROM_USER_IN_ROOM, MSG_FROM_USER_IN_ROOM, IMAGE_FROM_USER, MSG_FROM_USER` are never sended

Parameterize location of the config file

Being able to specify the location of the config file would be really great. Currently configuring sygnal is less than ideal as you have to worry about the working directory.

The current parsing code has a comment stating this hasn't been implemented because it is awkward when using with gunicorn.

Method 1: gunicorn passing

The "correct" way to do this with gunicorn is like so:

gunicorn -c /etc/sygnal/gunicorn_config.py sygnal:setup("path/to/config")

Then just change the function setup to take the config path as a param, and plumb it down to parse_config()

This is ugly, I agree, but it is the right way according to gunicorn.

Method 2: environment variable

Alternatively, and perhaps more elegantly, the config could be passed via environment variable. So the line at

cfg.read("sygnal.conf")

Could become:

cfg.read(os.getenv('SYGNAL_CONFIG') or "sygnal.conf")

What are the next steps?

What are the next steps after executing:

cp ./sygnal.yaml.example ./sygnal.yaml
python ./setup.py install
python -m sygnal.sygnal

Install signal how Application Services?

Docker Image is not working

Running the image from https://hub.docker.com/r/matrixdotorg/sygnal gives back these errors:

2020-02-29 16:30:51,604 [1] INFO  twisted Received SIGINT, shutting down.
Unhandled error in Deferred:
2020-02-29 16:30:51,639 [1] CRITICAL twisted Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sygnal/sygnal.py", line 301, in <module>
    sygnal.run()
  File "/usr/local/lib/python3.7/site-packages/sygnal/sygnal.py", line 186, in run
    self._make_pushkins_then_start(port, bind_addresses, pushgateway_api)
  File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 911, in ensureDeferred
    return _cancellableInlineCallbacks(coro)
  File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python3.7/site-packages/sygnal/sygnal.py", line 169, in _make_pushkins_then_start
    sys.exit(1)
builtins.SystemExit: 1

2020-02-29 16:30:51,642 [1] CRITICAL twisted 
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python3.7/site-packages/sygnal/sygnal.py", line 169, in _make_pushkins_then_start
    sys.exit(1)
SystemExit: 1

A sygnal.yaml has been provided in /sygnal.yaml with the content of sygnal.yaml.sample.

IOS apns notification not working

I can receive notifications from android through firebase also I got this error which I don't know what is it related to:

Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: Traceback (most recent call last):
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/.sygnal/lib/python3.6/site-packages/h2/connection.py", line 224, in process_input
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: func, target_state = self.transitions[(self.state, input)]
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: During handling of the above exception, another exception occurred:
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: Traceback (most recent call last):
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/sygnal/sygnal/http.py", line 234, in _handle_dispatch
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: result = await pushkin.dispatch_notification(notif, d, context)
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/sygnal/sygnal/apnspushkin.py", line 227, in dispatch_notification
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: log, span, device, shaved_payload, prio
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/sygnal/sygnal/apnspushkin.py", line 158, in _dispatch_request
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: response = await self._send_notification(request)
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/sygnal/sygnal/apnspushkin.py", line 418, in send_notification
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: asyncio.ensure_future(self.apns_client.send_notification(request))
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/.sygnal/lib/python3.6/site-packages/twisted/internet/defer.py", line 824, in adapt
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: extracted = result.result()
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/.sygnal/lib/python3.6/site-packages/aioapns/client.py", line 49, in send_notification
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: response = await self.pool.send_notification(request)
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/.sygnal/lib/python3.6/site-packages/aioapns/connection.py", line 377, in send_notification
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: response = await connection.send_notification(request)
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/.sygnal/lib/python3.6/site-packages/aioapns/connection.py", line 176, in send_notification
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: headers=headers
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/.sygnal/lib/python3.6/site-packages/h2/connection.py", line 766, in send_headers
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: self.state_machine.process_input(ConnectionInputs.SEND_HEADERS)
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: File "/home/ubuntu/.sygnal/lib/python3.6/site-packages/h2/connection.py", line 229, in process_input
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: "Invalid input %s in state %s" % (input
, old_state)
Apr 09 22:02:44 ip-172-31-11-241 python3[29508]: h2.exceptions.ProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in state ConnectionState.CLOSED

APNS config warning on valid keys

When configuring an APNS via keyfile, sygnal warns about not understood config items:

2020-02-13 12:21:36,725 [1] WARNING sygnal.apnspushkin The following configuration fields are not understood: {'team_id', 'key_id', 'keyfile', 'topic'}

However, these are not only mentioned in the sample config, but clearly also used by by the code.

Failed to install sygnal

~/matrix/sygnal$ sudo python3.7 setup.py install
running install
Checking .pth file support in /usr/local/lib/python3.7/dist-packages/
/usr/bin/python3.7 -E -c pass
TEST PASSED: /usr/local/lib/python3.7/dist-packages/ appears to support .pth files
running bdist_egg
running egg_info
writing matrix_sygnal.egg-info/PKG-INFO
writing dependency_links to matrix_sygnal.egg-info/dependency_links.txt
writing requirements to matrix_sygnal.egg-info/requires.txt
writing top-level names to matrix_sygnal.egg-info/top_level.txt
reading manifest file 'matrix_sygnal.egg-info/SOURCES.txt'
writing manifest file 'matrix_sygnal.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying matrix_sygnal.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying matrix_sygnal.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying matrix_sygnal.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying matrix_sygnal.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying matrix_sygnal.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
Traceback (most recent call last):
  File "setup.py", line 49, in <module>
    long_description=read("README.rst"),
  File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 209, in run
    os.path.join(archive_root, 'EGG-INFO'), self.zip_safe()
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 245, in zip_safe
    return analyze_egg(self.bdist_dir, self.stubs)
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 355, in analyze_egg
    safe = scan_module(egg_dir, base, name, stubs) and safe
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 392, in scan_module
    code = marshal.load(f)
ValueError: bad marshal data (unknown type code)

VOIP: config for ios

I have following config in sygnal.yaml file for ios voip notifications

com.app.voip:
    type: apns
    certfile: certfile.pem
    platform: sandbox
    event_handlers:
      'm.call.invite': voip

but every time the rejected response and sometimes even BadDeviceToken(in logs) response

is there something I am missing?

Sygnal.apnspushkin ERROR Error sending push to token

Set up my local sygnal, uploaded pem certificate,
send POST request to http: // localhost: 5000 / _matrix / push / v1 / notify, error in logs
pushbaby.pushconnection WARNING Push to tokenfailed with status 8
2019-05-06 11: 27: 52,208 k??฿—??W?วœ?~x?V??usF???5i?y{w???us?v?ฦต???kG}, status 8
2019-05-06 11: 27: 52,215 pushbaby.pushconnection INFO Retrying 0 pushes sent after failed.
The token is correct and the certificate too, sent a request through curl and everything works, help please understand what the problem is.

Can't send APNs Push - Attached to a different loop Error

I'm getting weird connection errors but no idea what's wrong
I am using python 3.7 as stated in the readme

2019-12-20 08:41:27,323 [25792] INFO  twisted SygnalLoggedSite starting on 5000
2019-12-20 08:41:27,323 [25792] INFO  twisted Starting factory <sygnal.http.SygnalLoggedSite object at 0x7fd5b5d1c610>
2019-12-20 08:41:32,531 [25792] INFO  sygnal.apnspushkin [b14fe7c7-ae0c-43bf-8dd7-d379309a78b6] Sending as APNs-ID 23d0655b-3336-4af2-9fe9-7876e00c75fa
2019-12-20 08:41:32,532 [25792] ERROR aioapns Could not connect to server: Task <Task pending coro=<APNs.send_notification() running at /usr/local/lib/python3.7/dist-packages/aioapns/client.py:49> cb=[Deferred.fromFuture.<locals>.adapt() at /usr/lib/python3/dist-packages/twisted/internet/defer.py:819]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.7/asyncio/futures.py:348]> attached to a different loop
2019-12-20 08:41:32,532 [25792] WARNING aioapns Could not send notification 23d0655b-3336-4af2-9fe9-7876e00c75fa: ConnectionError
2019-12-20 08:41:33,534 [25792] ERROR aioapns Could not connect to server: Task <Task pending coro=<APNs.send_notification() running at /usr/local/lib/python3.7/dist-packages/aioapns/client.py:49> cb=[Deferred.fromFuture.<locals>.adapt() at /usr/lib/python3/dist-packages/twisted/internet/defer.py:819]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.7/asyncio/futures.py:348]> attached to a different loop
2019-12-20 08:41:33,534 [25792] WARNING aioapns Could not send notification 23d0655b-3336-4af2-9fe9-7876e00c75fa: ConnectionError
2019-12-20 08:41:34,535 [25792] ERROR aioapns Could not connect to server: Task <Task pending coro=<APNs.send_notification() running at /usr/local/lib/python3.7/dist-packages/aioapns/client.py:49> cb=[Deferred.fromFuture.<locals>.adapt() at /usr/lib/python3/dist-packages/twisted/internet/defer.py:819]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.7/asyncio/futures.py:348]> attached to a different loop
2019-12-20 08:41:34,536 [25792] WARNING aioapns Could not send notification 23d0655b-3336-4af2-9fe9-7876e00c75fa: ConnectionError
2019-12-20 08:41:35,537 [25792] ERROR aioapns Could not connect to server: Task <Task pending coro=<APNs.send_notification() running at /usr/local/lib/python3.7/dist-packages/aioapns/client.py:49> cb=[Deferred.fromFuture.<locals>.adapt() at /usr/lib/python3/dist-packages/twisted/internet/defer.py:819]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.7/asyncio/futures.py:348]> attached to a different loop
2019-12-20 08:41:35,537 [25792] WARNING aioapns Could not send notification 23d0655b-3336-4af2-9fe9-7876e00c75fa: ConnectionError
2019-12-20 08:41:35,537 [25792] ERROR aioapns Failed to connect after 4 attempts.
2019-12-20 08:41:35,538 [25792] WARNING sygnal.apnspushkin [b14fe7c7-ae0c-43bf-8dd7-d379309a78b6] Temporary failure, will retry in 10 seconds
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/aioapns/connection.py", line 335, in acquire
    connection = await self.create_connection()
  File "/usr/local/lib/python3.7/dist-packages/aioapns/connection.py", line 429, in create_connection
    ssl=self.ssl_context
  File "/usr/lib/python3.7/asyncio/base_events.py", line 905, in create_connection
    type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 1275, in _ensure_resolved
    proto=proto, flags=flags)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 784, in getaddrinfo
    None, getaddr_func, host, port, family, type, proto, flags)
RuntimeError: Task <Task pending coro=<APNs.send_notification() running at /usr/local/lib/python3.7/dist-packages/aioapns/client.py:49> cb=[Deferred.fromFuture.<locals>.adapt() at /usr/lib/python3/dist-packages/twisted/internet/defer.py:819]> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/lib/python3.7/asyncio/futures.py:348]> attached to a different loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/sygnal/sygnal/apnspushkin.py", line 152, in _dispatch_request
    response = await self._send_notification(request)
  File "/home/user/sygnal/sygnal/apnspushkin.py", line 412, in _send_notification
    asyncio.ensure_future(self.apns_client.send_notification(request))
  File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 821, in adapt
    extracted = result.result()
  File "/usr/local/lib/python3.7/dist-packages/aioapns/client.py", line 49, in send_notification
    response = await self.pool.send_notification(request)
  File "/usr/local/lib/python3.7/dist-packages/aioapns/connection.py", line 360, in send_notification
    connection = await self.acquire()
  File "/usr/local/lib/python3.7/dist-packages/aioapns/connection.py", line 339, in acquire
    raise ConnectionError()
aioapns.exceptions.ConnectionError

consider doing fewer retries for APNS

if we have problems with APNS, we retry each request at 10s and 20s intervals. Given that haproxy limits the number of concurrent requests to sygnal, this can easily mean that the haproxy<->signal connection pool gets saturated with failing APNS requests, so that very few GCM requests get a look-in.

We should consider doing fewer retries, or doing them faster, or marking APNS as down and failing the request more quickly.

Really, we need a mechanism to indicate to homeservers that a push failed (https://github.com/matrix-org/matrix-doc/issues/2534).

Push notifications not working..Error in sygnal gateway.

Push notifications are not working. I got the following errors in sygnal log:

2017-04-07 17:31:38,068 sygnal.apnspushkin INFO '{'aps': {'content-available': 1, 'sound': u'default', 'badge': 1, 'alert': {'loc-args': [u'@user111:my.domain.name', u'test msg 2'], 'loc-key': 'MSG_FROM_USER_WITH_CONTENT'}}, 'room_id': u'!KtyhAUyBMXMbpOVdXC:my.domain.name'}' -> /jELQ5q651Af+obOIek7CPonYcqd4W62rbh6wmEuBF0=
2017-04-07 17:31:38,068 pushbaby.pushconnection INFO Establishing new connection to ('gateway.push.apple.com', 2195)
2017-04-07 17:31:38,312 pushbaby.pushconnection WARNING Couldn't set socket timeout (only works on Linux >= 2.6.37). Unresponsive connections will take a long time to timeout and pushes during that time will be lost.
2017-04-07 17:31:38,898 pushbaby.pushconnection INFO Connection closed remotely
2017-04-07 17:31:38,899 pushbaby.pushconnection ERROR Caught exception sending push
Traceback (most recent call last):
File "build/bdist.macosx-10.12-x86_64/egg/pushbaby/pushconnection.py", line 265, in sendpush
res['ret'] = self._reallysend(payload, token, expiration, priority, identifier)
File "build/bdist.macosx-10.12-x86_64/egg/pushbaby/pushconnection.py", line 285, in _reallysend
raise ConnectionDeadException()
ConnectionDeadException
2017-04-07 17:31:38,901 pushbaby INFO Connection died: removing
2017-04-07 17:31:38,902 sygnal.apnspushkin ERROR Exception sending push

sys.exit(1) doesn't actually stop Sygnal

In _make_pushkins_then_start, we call sys.exit(1) when failing to create pushkins:

sygnal/sygnal/sygnal.py

Lines 185 to 197 in 9a1e3c8

async def _make_pushkins_then_start(self, port, bind_addresses, pushgateway_api):
for app_id, app_cfg in self.config["apps"].items():
try:
self.pushkins[app_id] = await self._make_pushkin(app_id, app_cfg)
except Exception:
logger.exception(
"Failed to load and create pushkin for kind %s", app_cfg["type"]
)
sys.exit(1)
if len(self.pushkins) == 0:
logger.error("No app IDs are configured. Edit sygnal.yaml to define some.")
sys.exit(1)

Instead of causing Sygnal to shut down, they instead get caught by Twisted, which errors but doesn't stop the reactor:

2020-05-12 12:51:20,826 [16441] CRITICAL twisted Unhandled error in Deferred:
2020-05-12 12:51:20,827 [16441] CRITICAL twisted 
Traceback (most recent call last):
  File "/home/brendan/.virtualenvs/sygnal-py38/lib/python3.8/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/home/brendan/Documents/matrix/sygnal/sygnal/sygnal.py", line 193, in _make_pushkins_then_start
    sys.exit(1)
SystemExit: 1

DeviceTokenNotForTopic

cannot send VOIP push notification

Nots: the certificate is working fine from a dummy test service.

Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: 2020-04-10 22:51:48,802 [13121] ERROR aioapns Status of notification 3af8357b-0673-497e-9546-384953672a1b is 400 (DeviceTokenNotForTopic)
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: 2020-04-10 22:51:48,803 [13121] WARNING sygnal.http [e35a264c-eaf1-4985-ab64-26dc8c439ed2] Failed to dispatch notification.
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: Traceback (most recent call last):
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: File "/home/ubuntu/sygnal/sygnal/http.py", line 234, in _handle_dispatch
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: result = await pushkin.dispatch_notification(notif, d, context)
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: File "/home/ubuntu/sygnal/sygnal/apnspushkin.py", line 227, in dispatch_notification
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: log, span, device, shaved_payload, prio
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: File "/home/ubuntu/sygnal/sygnal/apnspushkin.py", line 185, in _dispatch_request
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: f"{response.status} {response.description}"
Apr 10 22:51:48 ip-172-31-11-241 python3[13121]: sygnal.exceptions.NotificationDispatchException: 400 DeviceTokenNotForTopic

Sygnal sends all pushes with high priority level, which drains the devices resource

Sygnal sends all pushes with high priority level (apns-priority 10 for iOs, and high for android), even for low priority events.

Priority of FCM push are explained here: https://firebase.google.com/docs/cloud-messaging/concept-options#setting-the-priority-of-a-message

This make the device to wake up, and the app to wake up and perform a sync.

On big accounts, it can be more than every 30 seconds, which make the Gplay app syncs more than the F-Droid version, which syncs every 30 seconds, when the app is in the background

In Android P, the system may ignore high priority Push for applications which are abusing high priority push (see https://developer.android.com/topic/performance/power/power-details)

The Android system may consider that the application is too battery greedy and will display some warning which is a bit annoying and deceptive for the user (along with the fact that the battery of its device is drained fast) (screenshot in French):

image

So we should use high priority Push for noisy event, and for event in e2e rooms maybe, and else use normal priority Push, which let the system choose when to deliver the Push to the app, regarding the current power mode of the device (screen on, idle, idle screen off not moving, etc.).

To fix that, Sygnal should know is the push to send has to be high or low priority, which may lead to change in the API.

Related issue:

Python 3 Upgrade

Now that synapse supports python 3, what about sygnal?

Looking at the dependencies, flask supports python 3 on the currently pinned version. However gevent (now pinned at 1.0.1) will need to be at least 1.1. pushbaby also needs an update, but this shouldn't be difficult as it only depends on gevent.

The sample logging configuration doesn't work

If used as is, the sample logging configuration makes Sygnal not log anything, neither in stdout, stderr nor sygnal.log.

When tweaking it to try to make it work somehow, I also couldn't get

sygnal/sygnal/sygnal.py

Lines 190 to 192 in 9a1e3c8

logger.exception(
"Failed to load and create pushkin for kind %s", app_cfg["type"]
)

or

logger.error("No app IDs are configured. Edit sygnal.yaml to define some.")

or any log line that didn't originate from Twisted, to log, both in sygnal.log and in stderr.

Make it clear that Sygnal is Element-flavoured, and document what other apps can expect

Although much of the framework of sygnal is valid as a reference push gateway, the pushkins are specific to riot-iOS/riot-X/riot-android, making the whole riot-specific.

A few things that would help make this clear:

  • updates in the readme to explain it
  • move the project to the vector-im github org
  • move the sygnal currently on matrix.org to riot.im

The following configuration fields are not understood: {'database'}

I am running sygnal following the instructions in the README: python -m sygnal.sygnal

I am using sqlite3 exactly like in the sample configuration:

database:
  name: sqlite3
  args:
    dbfile: sygnal.db

But I get this:

The following configuration fields are not understood: {'database'}

Any idea?

Unit tests raise Exception in setUp after successfully passing

This isn't critical, as the tests appear to be passing. However, it's not a great reassurance to see errors being thrown to the console.

Traceback (most recent call last):
  File "/home/will/git/sygnal/venv/lib/python3.7/site-packages/twisted/internet/utils.py", line 217, in runWithWarningsSuppressed
    result = f(*a, **kw)
  File "/home/will/git/sygnal/tests/testutils.py", line 50, in setUp
    self.sygnal._make_pushkins_then_start(0, [], None)
  File "/home/will/git/sygnal/venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 911, in ensureDeferred
    return _cancellableInlineCallbacks(coro)
  File "/home/will/git/sygnal/venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File "/home/will/git/sygnal/venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/home/will/git/sygnal/sygnal/sygnal.py", line 169, in _make_pushkins_then_start
    sys.exit(1)
builtins.SystemExit: 1

when can we expect fix for database warning

hi,
we are planning to migrate from xmpp to matrix with mobile clients . we are running sygnal as push gateway but service running not listening on 5000 port. Any alternate push gateway for matrix synapse

'Incorrect padding' on APNS notification request

it was working on Sygnal v0.6.0 , although I send the token as a string, what is wrong with it ?

{
"notification": {
"event_id": "$hUG6SS47Lgms90h363Iwji7g6EJDMAe8SuNgtO-MnLU",
"room_id": "!DxDVMhkbmjTMTdbHsJ:friends-groups.com",
"type": "m.room.message",
"sender": "@xxxxxxxxxx:xxxxxxxxxx.com",
"sender_display_name": "Major Tom",
"room_name": "Mission Control",
"room_alias": "@xxxxxxxxxx:xxxxxxxxxx.com",
"prio": "high",
"content": {
"msgtype": "m.text",
"body": "I'm floating in a most peculiar way."
},
"counts": {
"unread": 2,
"missed_calls": 1
},
"devices": [
{
"app_id": "com.xxxxxxxxxx:xxxxxxxxxx.matrix",
"pushkey": "APA91bHPRgkF3JUikC4ENAHEeMrd41Zxv3hVZjC9KtT8OvPVGJ-hQMRKRrZuJAEcl7B338qju59zJMjw2DELjzEvxwYv7hH5Ynpc1ODQ0aT4U4OFEeco8ohsN5PjL1iC2dNtk2BAokeMCg2ZXKqpc8FXKmhX94kIxQ",
"pushkey_ts": 12345678,
"data": {
"payload": "hello"
},
"tweaks": {
"sound": "bing"
}
}
]
}
}

version: Sygnal v0.7.0

Jun 29 18:40:42 ip-172-31-11-241 python3[12784]: Traceback (most recent call last):
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:   File "/home/ubuntu/sygnal/sygnal/http.py", line 265, in _handle_dispatch
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:     result = await pushkin.dispatch_notification(notif, d, context)
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:   File "/home/ubuntu/sygnal/sygnal/apnspushkin.py", line 256, in dispatch_notification
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:     log, span, device, shaved_payload, prio
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:   File "/home/ubuntu/sygnal/sygnal/apnspushkin.py", line 174, in _dispatch_request
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:     device_token = base64.b64decode(device.pushkey).hex()
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:   File "/home/ubuntu/.sygnal/lib/python3.6/base64.py", line 87, in b64decode
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]:     return binascii.a2b_base64(s)
Jun 29 18:40:42 ip-172-31-11-241 python3[12784]: binascii.Error: Incorrect padding

Synapse and Sygnal: /_matrix/push/v1/notify 404 error

Hello. I am trying to setup synapse and sygnal on the same server. The setup seems to work well except the push notifications part. More specifically - when I send a message, synapse receives it and attempts to communicate with sygnal - but fails with the following error:

2019-02-23 12:25:49,397 - synapse.http.client - 96 - INFO - httppush.process-398- Sending request POST https://myhostname.com/_matrix/push/v1/notify
2019-02-23 12:25:49,411 - synapse.metrics - 374 - INFO - - Collecting gc 0
2019-02-23 12:25:49,415 - synapse.access.http.8008 - 233 - INFO - POST-45257- ::ffff:127.0.0.1 - 8008 - Received request: POST /_matrix/push/v1/notify
2019-02-23 12:25:49,416 - synapse.access.http.8008 - 302 - INFO - POST-45257- ::ffff:127.0.0.1 - 8008 - {None} Processed request: 0.001sec/-0.000sec (0.000sec, 0.000sec) (0.000sec/0.000sec/0) 167B 404 "POST /_matrix/push/v1/notify HTTP/1.0" "Synapse/0.34.0.1" [0 dbevts]
2019-02-23 12:25:49,418 - synapse.http.client - 111 - INFO - httppush.process-398- Received response to POST https://myhostname.com/_matrix/push/v1/notify: 404
2019-02-23 12:25:49,419 - synapse.push.httppusher - 339 - WARNING - httppush.process-398- Failed to push event

What am I doing wrong?

service running but not listening port 5000

OS VERSION : UBUNTU 16.04 LTS
PYTHON VERSION : Python 3.8.2
AWS HOSTING

After running : python3.8 setup.py install (successful)
&
python3.8 -m sygnal.sygnal :
The following configuration fields are not understood: {'database'}

ps -def | grep sygnal
root 9689 1 0 09:20 ? 00:00:00 tcpdump -i any -w sygnal.pcap
root 9977 9953 1 10:45 pts/0 00:00:00 python3.8 -m sygnal.sygnal
root 10054 10038 0 10:45 pts/2 00:00:00 grep --color=auto sygnal

netstat -pltun
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1246/sshd
tcp6 0 0 :::22 :::* LISTEN 1246/sshd
udp 0 0 0.0.0.0:68 0.0.0.0:* 985/dhclient

sygnal.log : empty

syganl.yaml
attached sygnal.yaml kindly check
sygnal.zip

Push notifications on FCM/Android fail silently

I installed Sygnal on a Raspbian Buster Lite board, added my app with the correct FCM API key in the Pushkins section of the .yaml file. I can start Sygnal without problem, listening on port 5000. I can see in the console that it correctly picks up all push notifications from the Synapse server. I also added a few log.info commands in gcmpushkin.py to check that it is connecting to Firebase. Everything seems fine, the HTTP response contains a message ID and a success flag. Still, no notifications come through to my development device (Android). I can send test notifications from the Firebase console, using the same pushkey (token) which I can see in the Sygnal console. So the device is definitely reachable. Am I missing other settings, for authorising Sygnal to send notifications to Firebase?

Failed to load and create pushkin for kind %s'

After the latest pull (commit cde647a) I get the following error when I run sygnal:

/opt/sygnal# python -m sygnal.sygnal
Error during startup:
Traceback (most recent call last):
  File "/opt/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/opt/sygnal/sygnal/sygnal.py", line 221, in start
    port, bind_addresses, pushgateway_api
  File "/opt/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py", line 911, in ensureDeferred
    return _cancellableInlineCallbacks(coro)
  File "/opt/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File "/opt/sygnal/sygnal/sygnal.py", line 221, in start
    port, bind_addresses, pushgateway_api
  File "/opt/synapse/env/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/opt/sygnal/sygnal/sygnal.py", line 194, in _make_pushkins_then_start
    "Failed to load and create pushkin for kind %s", app_cfg["type"]
builtins.RuntimeError: ('Failed to load and create pushkin for kind %s', 'apns')

Any idea?

BadDeviceToken for every push on dev and prod cert

Have no idea what is this. Somebody help?

2020-04-17 10:05:35,415 [28411] INFO sygnal.apnspushkin [1fc01ce6-285d-488c-91af-06dcacda7c66] Sending as APNs-ID bc6a234c-885d-445e-84f8-279f072d6cf6 2020-04-17 10:05:36,030 [28411] WARNING sygnal.http [1fc01ce6-285d-488c-91af-06dcacda7c66] Failed to dispatch notification. Traceback (most recent call last): File "/var/www/matrix_sygnal/sygnal/http.py", line 234, in _handle_dispatch result = await pushkin.dispatch_notification(notif, d, context) File "/var/www/matrix_sygnal/sygnal/apnspushkin.py", line 227, in dispatch_notification log, span, device, shaved_payload, prio File "/var/www/matrix_sygnal/sygnal/apnspushkin.py", line 185, in _dispatch_request f"{response.status} {response.description}" sygnal.exceptions.NotificationDispatchException: 400 BadDeviceToken

Troubleshoot running Sygnal locally

I am trying to run the Sygnal Push Gateway service locally, however I am running into an issue. After I install the required packages, and I start the service I am not seeing any output in the terminal. I am also not seeing any output being added to the sygnal.log file.

I tried visiting the Prometheus client on http://localhost:8000/, and I also tried sending a POST request to http://localhost:5000/_matrix/push/v1/notify, but I am still not seeing any logs and I am not getting a response from the service at all.

After cloning the repository, these are the command that I ran:

cp ./sygnal.yaml.sample ./sygnal.yaml
virtualenv env
source env/bin/activate
pip install .

My Python version: Python 3.7.1

Is there anything else I need to do to just get the service up and running? Or, do you have any tips for troubleshooting the issue? Any help would be much appreciated.

how to run as a service?

Hi, there:

I not good at Python, but I need run sygnal as a service, how to do ?

please give some advise!

Thanks

error in running service

python3.7 -m sygnal.sygnal
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/local/lib/python3.7/runpy.py", line 109, in _get_module_details
import(pkg_name)
File "/root/sygnal/sygnal/init.py", line 16, in
from importlib_metadata import version, PackageNotFoundError
ModuleNotFoundError: No module named 'importlib_metadata'

No sound for push notifications on iOS

The "apnspushkin" class never sends the "sound" attribute in the notification payload (s. Push notification docs). Thus, the iOS device doesn't play a sound when the push notification is received.

From my understanding the "apnspushkin" should read the tweaks information from the device object (notifications.py), but this seems to be ignored completely. Still, I don't understand how the Riot.im iOS app manages to play a sound for incoming push notifications. Maybe a customized version of Sygnal is used?

The following configuration fields are not understood: {'database'}

sygnal service running . but not able to listen port 5000 why ?

ps -def | grep sygnal
root 4115 2195 1 09:46 pts/3 00:00:00 python3.8 -m sygnal.sygnal
root 4121 2293 0 09:47 pts/4 00:00:00 grep --color=auto sygnal

netstat -pltun
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1246/sshd
tcp6 0 0 :::22 :::* LISTEN 1246/sshd
udp 0 0 0.0.0.0:68 0.0.0.0:* 985/dhclient

Testing the setup

I've got sygnal up and running with an APNS package and certfile.

Is there a simple request I can make (with curl ideally) to test that it is working?

VOIP Notifications (enhancement, not issue)

So, I installed sygnal a couple weeks back and it's working great. (had a couple issues with config, [I would suggest adding a 'platform' field to the sygnal.conf.sample file under the apns just for clarity] but nbd, it was all solved).
However, I'm trying to develop an iOS client that implements CallKit/PushKit notifications (aka VOIP Notifications) in order to improve the user experience and bring up the 'call' notifications without needing a background persistent connection.

Is anyone working towards adapting sygnal to these protocols? Can you think of any way to work around it? Should I be fine setting two pushers per iOS client (one for apns token and one for voip token) and then sending notifications through both (PushKit and regular UserNotifications)? Could I prevent double notifications by using some sort of pushrule to isolate voip/video calls?

Thank you!

MQTT Push Provider?

MQTT would be an alternative for Riot on Android/maybe iOS, Facebook is using it for their messenger instead of GCM/APNS. This way Google and Apple would get less metadata, which is one of the reasons people prefer self hosted open source solutions.

If this is not in the scope for sygnal, maybe someone could implement another push gateway that would do this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.