mautrix / whatsapp Goto Github PK

View Code? Open in Web Editor NEW

1.2K 38.0 165.0 7.19 MB

A Matrix-WhatsApp puppeting bridge

Home Page: https://maunium.net/go/mautrix-whatsapp

License: GNU Affero General Public License v3.0

Go 99.65% Dockerfile 0.11% Shell 0.24%

matrix-org matrix-appservice whatsapp whatsapp-web golang matrix bridge go

whatsapp's Introduction

mautrix-whatsapp

A Matrix-WhatsApp puppeting bridge based on whatsmeow.

Documentation

All setup and usage instructions are located on docs.mau.fi. Some quick links:

Bridge setup (or with Docker)
Basic usage: Authentication

Features & Roadmap

ROADMAP.md contains a general overview of what is supported by the bridge.

Discussion

Matrix room: #whatsapp:maunium.net

whatsapp's People

Contributors

Stargazers

Watchers

Forkers

krombel izissise bthiago jo-so devopsmike dereisele nodje matterbridge anoadragon453 renner-dev myii martijnbraam helcaraxan devilankur18 jonesmat 036006 nucleoosystem progserega rixon-gh syrusakbary freeguy1 rezafahrur appatsekhar 14mrh4x0r surakin rafaeltheraven sthales rreuvekamp n0emis pwr22 su-ex cerebralia pidongqianqian ezzalddeenali kelaresg nrj5k lieuwex whatsappauto bevgeniys legogris kubesail kangeyan16 nuka-q arpagon themanagementfirm ironashram f4814 apetresc notice4thomas 6543-forks marcool04 efraimbart forkkit widi-aryanto errorsystem69 karmanyaahm craeckie goldsteal moe8844 hanseartic gredin67 conor-f rajeshsaini2115 wlongo cs-joy isgasho jkerai1 capesepias cottic gruberdev hummans element-hq r3freshreload apmechev jaller94 maniacs-oss xaddison kryotech-ltd pictuga kidnxpped abmantis bnjbvr harharlinks bsmr technocode sahilm3 jaydesai-lintel helderfsferreira markobl josiasmc memarius bcattaneo ursinndev carlambroselli sjustein xirdigh hixio-mh jjgadget equitytrustbankxfether0xlba0ethplorer wundercomm

whatsapp's Issues

docker-android instead of Android Studio

This may be a slightly easier alternative to Android Studio: https://github.com/budtmo/docker-android

Error on go build

I've follow the instructions from: https://github.com/tulir/mautrix-whatsapp/wiki/Bridge-setup and tried with this command too: go get -u maunium.net/go/mautrix-whatsapp

And i've got this error
../golang/src/maunium.net/go/mautrix/format/markdown.go:7:2: cannot find package "github.com/russross/blackfriday/v2" in any of: /usr/lib/go-1.7/src/github.com/russross/blackfriday/v2 (from $GOROOT) /home/sven/golang/src/github.com/russross/blackfriday/v2 (from $GOPATH) commands.go:26:2: cannot find package "maunium.net/go/maulogger/v2" in any of: /usr/lib/go-1.7/src/maunium.net/go/maulogger/v2 (from $GOROOT) /home/sven/golang/src/maunium.net/go/maulogger/v2 (from $GOPATH) sven@renard:~/mautrix-whatsapp$

Confusing help message "import contacts"

Confused user is confused.

Cannot do a go get

package github.com/russross/blackfriday/v2: cannot find package "github.com/russross/blackfriday/v2" in any of:
        /usr/lib/go/src/github.com/russross/blackfriday/v2 (from $GOROOT)
        /data/gopath/src/github.com/russross/blackfriday/v2 (from $GOPATH)

Adding people to a group chat not reflected on matrix side

No new members are added to the matrix room.

Bridge unpuppeted matrix users as a bot user to Whatsapp

Group chats are syncing correctly 1:1s are not

I'm new to the bridge, but since spinning it up yesterday I've run import contacts which creates one invite per group chat (with maybe 95% success rate on naming the rooms). No 1:1s are synced.

The command import 447700900123 triggers the following logging:

[Jan 21, 2019 18:35:17] [Matrix/@whatsapp_447700900123:lant.uk/DEBUG] POST /_matrix/client/r0/register {"username":"whatsapp_447700900123","initial_device_display_name":""}
[Jan 21, 2019 18:35:17] [Matrix/@whatsapp_447700900123:lant.uk/DEBUG] PUT /_matrix/client/r0/profile/@whatsapp_447700900123:lant.uk/displayname {"displayname":"+447700900123 (WA)"}

But doesn't actually create an invite into Matrix.

No Portal creation (or reinvite) on new message on WhatsApp side

When you once either rejected the invite or left a portal room there is no way currently to reestablish a communication with a WhatsApp member

Getting binary in JSON parser

Getting some very odd stuff in the JSON parser. This is on typing login and then scanning the QR code.

[Mar  5, 2019 16:44:35] [User/@whatsapptester:localhost/DEBUG] Connecting to WhatsApp
[Mar  5, 2019 16:44:36] [User/@whatsapptester:localhost/DEBUG] WhatsApp connection successful
[Mar  5, 2019 16:44:41] [User/@whatsapptester:localhost/DEBUG] JSON message: �,fŨ�"�j�D�M++_�"״7s���Q��iA�T�/d��r��<�Q�(���|
        ��
[Mar  5, 2019 16:44:41] [User/@whatsapptester:localhost/DEBUG] JSON message: ["Props",{"wsOverrideTimeout":3,"wsOverrideAttempts":2,"wsOverride":true,"fbCrashlog":true,"bucket":"","gifSearch":"giphy","SPAM":true,"SET_BLOCK":true,"MESSAGE_INFO":true,"maxFileSize":100,"media":64,"maxSubject":25,"maxParticipants":257,"videoMaxEdge":960,"imageMaxKBytes":1024,"edit":1,"imageMaxEdge":1600,"suspiciousLinks":1,"fwdUiStartTs":1531267200,"groupsV3":1,"stickers":1,"restrictGroups":1,"announceGroups":1001,"quickForwardRestrictEnabled":0,"groupDescLength":512,"multicastLimitGlobal":5,"finalLiveLocation":1}]
[Mar  5, 2019 16:44:41] [User/@whatsapptester:localhost/DEBUG] JSON message: ["Stream","update",false,"0.3.2041"]
[Mar  5, 2019 16:44:41] [User/@whatsapptester:localhost/DEBUG] JSON message: ["Blocklist",{"id":1,"blocklist":[]}]
[Mar  5, 2019 16:44:41] [User/@whatsapptester:localhost/ERROR] WhatsApp JSON parse error: invalid character 'í' looking for beginning of value

How to deploy this bridge on a 3rd-party hosted rom?

I notice that installing the bridge needs to

Add the path to the registration file to your synapse homeserver.yaml under app_service_config_files.

Is that mean the bridge can't be use in on servers which is provided by matrix.org or whatever?
And if I was wrong , plz give me a hint to deploy this bridge in this case

DM rooms created during initial synchronisation silently discard sent messages.

I set it up and ran sync --create, which invited me to a bunch of promising looking DMs with 1-2 messages of scrollback in each one. I then tried to send messages in these rooms, but they were silently discarded by the AS. When new messages from the same users arrive, they invited me to a new DM room rather than using the existing one, and things then behaved correctly in the new room.

Log level is "error" and I got DEBUG lines

Hi,
This bridge saves me ;)

Only what I noticed if I configure:

logging:
  print_level: error

I can still see

...[User/..../DEBUG]...
...[Portal/..../DEBUG]...

lines...

Handle messages not bridged

A common pitfall with matrix bridges is that your message is sent to matrix but not actually bridged to the third party network. It's a pitty, as you need to really trust the code but have no way to ensure your message is actually bridged. For example, I developed a stupid irc bot which just reply on message just to be able to test my IRC bridge instance.

This a complex problem that cannot be totally solved at the bridge level, but I think we could help a bit the user. Obviously we can just add some retry, but I don't think it's a good solution. Imagine this:

time X: I send a message A, which is not bridged due to some error
time X+1: A user from the third party network send a message B, which is correctly bridged
time X+2: After some retries, my message A is finally bridged

Now we are in the situation where from my side I see the message A then B, but the users on the third party network see B then A.

That's why I think the retry duration should be very short (something like one minute), and the retries should stop as soon as we receive a message from the third party network.

The thing is to notify the user that the message was not bridged. For this, I see two solution:

The bridge's bot deletes the message from the matrix room, and notify the user in the admin room
The bridge's bot reply to the message (using matrix replies), to warn the user "this message was not sended" (to distinguish bot's messages, notify may be used).

The solution 1 keeps a clean room's history , but is not very user friendly and force the user to go to another room to know what happened. The second solution would be more easier to understand by the user, but will add some garbage to the room's history.

https://github.com/tulir/mautrix-whatsapp/blob/2ec6ad242c2c03961aa689e01a24ab7970fd4279/portal.go#L788

The bridge crashes when responding on Matrix

What did I do: I answered to a message on Matrix, using the "answer" (quote + link to previous message) feature.

What was expected: it produce a response message on the whatsapp side.

What happens: the bridge crashed with this:

panic: runtime error: slice bounds out of range

goroutine 1099 [running]:
maunium.net/go/mautrix-whatsapp/vendor/github.com/Rhymen/go-whatsapp.(*Conn).decryptBinaryMessage(0xc00015a1b0, 0xc000025110, 0xe, 0x10, 0xe, 0x10, 0x800a)
        /root/go/src/maunium.net/go/mautrix-whatsapp/vendor/github.com/Rhymen/go-whatsapp/conn.go:266 +0x3cc
maunium.net/go/mautrix-whatsapp/vendor/github.com/Rhymen/go-whatsapp.(*Conn).query(0xc00015a1b0, 0xa48d15, 0x7, 0xc0001b1700, 0x1b, 0xc0001b1d60, 0x14, 0xa476c1, 0x6, 0xa4655c, ...)
        /root/go/src/maunium.net/go/mautrix-whatsapp/vendor/github.com/Rhymen/go-whatsapp/contact.go:187 +0x3b0
maunium.net/go/mautrix-whatsapp/vendor/github.com/Rhymen/go-whatsapp.(*Conn).LoadMessagesBefore(0xc00015a1b0, 0xc0001b1700, 0x1b, 0xc0001b1d60, 0x14, 0x1, 0x1e, 0x97fba0, 0xc0003f43e0)
        /root/go/src/maunium.net/go/mautrix-whatsapp/vendor/github.com/Rhymen/go-whatsapp/contact.go:78 +0xd1
main.(*Portal).GetMessage(0xc00028f800, 0xc0001b1d60, 0x14, 0xc00057f310)
        /root/go/src/maunium.net/go/mautrix-whatsapp/portal.go:663 +0x6c
main.(*Portal).HandleMatrixMessage(0xc00028f800, 0xc0002ec240)
        /root/go/src/maunium.net/go/mautrix-whatsapp/portal.go:707 +0x1682
main.(*MatrixHandler).HandleMessage(0xc000173ad0, 0xc0002ec240)
        /root/go/src/maunium.net/go/mautrix-whatsapp/matrix.go:163 +0x20b
main.(*MatrixHandler).HandleMessage-fm(0xc0002ec240)
        /root/go/src/maunium.net/go/mautrix-whatsapp/matrix.go:42 +0x34
created by maunium.net/go/mautrix-whatsapp/vendor/maunium.net/go/mautrix-appservice.(*EventProcessor).Start
        /root/go/src/maunium.net/go/mautrix-whatsapp/vendor/maunium.net/go/mautrix-appservice/eventprocessor.go:56 +0x55

"login" without creating invitations for each contact

I'd like the opportunity to not have the "login" command creating loads of invitations by getting history data from whatsapp.

When a user just wants to bridge 2-3 of his important WA chats to Matrix, the initial "login" will create like 80 invitations for all his WA contacts and he needs to refuse 77 invites to have the 3 contacts he'd like to have in Matrix remaining.

In my opinion it would be good to have the login command preventing the creation of invites, just updating the database of the bridge for that user. So a user can add his wanted chats with:

login
sync
list
pm xxxxx
pm xxxxx
pm xxxxx

The history could get fetched when a user really creates a room with a WA contact (with sync --create or pm xxxx).

Replies to replies are not sent correctly

See images below for steps to reproduce & result on Matrix & WhatsApp ™️ side

Bridge looses messages which are delivered on restart

The bridge occasionally looses messages which are not delivered to Matrix. When the bridge is restarted, these missed messages are delivered (but they are not in the right order then in Riot).

Initially reported by olmari in the Matrix room, I can confirm this issue.

Leaving a group chat from Matrix should let WhatsApp leave the group chat too

Test connection to homeserver before sending events

I use a docker-compose setup which means that the mautrix-whatsapp bridge is always started just after the homeserver (using depends_on). However, once Docker thinks it's ready, the homeserver isn't fully up yet, but mautrix-whatsapp starts sending messages to the homeserver already, causing lots of 502 bad gateway errors from the nginx proxy in front of it. It doesn't retry sending the messages either, meaning lots of events go lost.

It would be nice if either mautrix-whatsapp first tries if a connection is available before starting to send messages, or retry at an interval if it receives an error like 502. Preferably it does both.

WhatsApp closing the connection is not handled correctly

[Dec  1, 2018 23:26:52] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:26:53] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:26:54] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:26:55] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:26:56] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:26:57] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:26:58] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:26:59] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Dec  1, 2018 23:27:00] [User/@Half-Shot:half-shot.uk/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)

Proper support for status updates

I wondered what this extra room with just the bot in it was... it turned out posting there created these whatsapp status update things :-).

Maybe that can be made a bit clearer.

Errors downloading media

Hi,

bridge works fine except that logs shows some media download errors. I think they appear on voice messages.

i already raised up the max file size in matrix-synapse to 128MB - made no difference.
Any idea?

Mär 14 04:41:55 h2606131.stratoserver.net mautrix-whatsapp[24633]: [Mar 14, 2019 04:41:55] [Portal/SOMENUMBERIVEHIDDDEN@[email protected]/ERROR] Failed to download media: download failed
Mär 14 04:41:55 h2606131.stratoserver.net mautrix-whatsapp[24633]: [Mar 14, 2019 04:41:55] [Portal/SOMENUMBERIVEHIDDDEN@[email protected]/ERROR] Failed to download media: download failed
Mär 14 04:41:55 h2606131.stratoserver.net mautrix-whatsapp[24633]: [Mar 14, 2019 04:41:55] [Portal/SOMENUMBERIVEHIDDDEN@[email protected]/ERROR] Failed to download media: download failed
Mär 14 04:41:55 h2606131.stratoserver.net mautrix-whatsapp[24633]: [Mar 14, 2019 04:41:55] [Portal/SOMENUMBERIVEHIDDDEN@[email protected]/ERROR] Failed to download media: download failed
Mär 14 04:41:55 h2606131.stratoserver.net mautrix-whatsapp[24633]: [Mar 14, 2019 04:41:55] [Portal/SOMENUMBERIVEHIDDDEN@[email protected]/ERROR] Failed to download media: download failed
Mär 14 04:41:55 h2606131.stratoserver.net mautrix-whatsapp[24633]: [Mar 14, 2019 04:41:55] [Portal/SOMENUMBERIVEHIDDDEN@[email protected]/ERROR] Failed to download media: download failed
Mär 14 04:41:55 h2606131.stratoserver.net mautrix-whatsapp[24633]: [Mar 14, 2019 04:41:55] [Portal/SOMENUMBERIVEHIDDDEN@[email protected]/ERROR] Failed to download media: download failed

the phone numbers i removed from the log

regards, Mario

Building doensnt work on raspberry pi

go get maunium.net/go/mautrix-whatsapp command returns:

mautrix-whatsapp/src/maunium.net/go/mautrix-whatsapp/types/types.go:20: syntax error: unexpected = in type declaration
mautrix-whatsapp/src/maunium.net/go/mautrix-whatsapp/types/types.go:23: syntax error: unexpected = in type declaration
mautrix-whatsapp/src/maunium.net/go/mautrix-whatsapp/types/types.go:26: syntax error: unexpected = in type declaration
mautrix-whatsapp/src/maunium.net/go/mautrix-whatsapp/types/types.go:29: syntax error: unexpected = in type declaration
mautrix-whatsapp/src/maunium.net/go/mautrix-whatsapp/types/types.go:32: syntax error: unexpected = in type declaration
# maunium.net/go/mautrix-whatsapp/vendor/maunium.net/go/gomatrix
mautrix-whatsapp/src/maunium.net/go/mautrix-whatsapp/vendor/maunium.net/go/gomatrix/reply.go:70: undefined: strings.Builder```

"list" sometimes returns empty list, solved by restart

The bridge sometimes gets into a state where the "list" command just returns an empty list:

A restart of the bridge is needed to solve this situation, after a restart the command works as expected.

Feature: replace hilights with name in body too

Hilights come in the for @<tel number>. The bridge should replace them with the display name of the user (if any).

Currently it's only done in formatted_body.

Mark messages read on Matrix side as read on WhatsApp

Title. Currently this doesn't happen, making it unclear which messages you have or have not read if using WhatsApp directly on your phone.

Bridge explodes if connectivity fails

[Mar 12, 2019 12:06:24] [User/@matthew:arasphere.net/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Mar 12, 2019 12:06:25] [User/@matthew:arasphere.net/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
[Mar 12, 2019 12:06:26] [User/@matthew:arasphere.net/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)
panic: repeated read on failed websocket connection

goroutine 75 [running]:
github.com/gorilla/websocket.(*Conn).NextReader(0xc000020160, 0x116dde0, 0xc000091312, 0x2, 0xc00009e000, 0xc0002ffb00)
        /home/matrix/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:967 +0x356
github.com/gorilla/websocket.(*Conn).ReadMessage(0xc000020160, 0xa135c0, 0xc000422790, 0x1, 0x1, 0xb56280, 0xc000422790)
        /home/matrix/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:1028 +0x2f
github.com/Rhymen/go-whatsapp.(*Conn).readPump(0xc00026e1a0)
        /home/matrix/go/pkg/mod/github.com/!rhymen/[email protected]/conn.go:272 +0x73
created by github.com/Rhymen/go-whatsapp.NewConn
        /home/matrix/go/pkg/mod/github.com/!rhymen/[email protected]/conn.go:129 +0x1ff

similar but not the same as #18

WhatsApp error: unexpected websocket close: websocket: close 1006 (abnormal closure): unexpected EOF

Hi,
After a few minutes of starting the service mautrix-whatsapp, I believe that whatsapp is blocking my requests.
this is de log:

error writing to socket: write tcp [XXXX:xXXXX:XXXXX:XXXX:XXX]:53642->[2a03:2880:f204:c5:face:b00c:0:167]:443: write: connection reset by peer
[Feb 22, 2019 17:06:46] [User/@user:matrix.domain.org/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1006 (abnormal closure): unexpected EOF
[Feb 22, 2019 17:06:47] [User/@user:matrix.domain.org/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1006 (abnormal closure): unexpected EOF
[Feb 22, 2019 17:06:48] [User/@user:matrix.domain.org/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1006 (abnormal closure): unexpected EOF
[Feb 22, 2019 17:06:49] [User/@user:matrix.domain.org/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1006 (abnormal closure): unexpected EOF
[Feb 22, 2019 17:06:49] [User/@user:matrix.domain.org/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1006 (abnormal closure): unexpected EOF

OS version

# cat /etc/issue
Debian GNU/Linux 9 \n \l

go version

:~# go version
go version go1.11.5 linux/amd64

mautrix-whatsapp version

:~/mautrix-whatsapp# git pull
Already up-to-date.

bridge down and not work after network fail

bridge started at virtual machine "vm1"
Whatsapp starterd in avd in virtual machine "vm2"
Network down.
vm1 and vm2 have no connect to internet ~4 hour.
After network down bridge logs: Failed to connect to WhatsApp: couldn't dial whatsapp web websocket: dial tcp 169.60.79.3:443: i/o timeout
Network up
Bridge not working (messages not sending from/to whatsapp).
I see process /opt/mautrix_whatsapp/bin/mautrix-whatsapp, but bridge not working

logs:

[Nov  1, 2018 13:12:54] [DEBUG] Initializing state store
[Nov  1, 2018 13:12:54] [DEBUG] Initializing database
[Nov  1, 2018 13:12:54] [DEBUG] Initializing Matrix event processor
[Nov  1, 2018 13:12:54] [DEBUG] Initializing Matrix event handler
[Nov  1, 2018 13:12:54] [INFO] Bridge initialization complete, starting...
[Nov  1, 2018 13:12:54] [DEBUG] Starting application service HTTP server
[Nov  1, 2018 13:12:54] [DEBUG] Starting event processor
[Nov  1, 2018 13:12:54] [INFO] Bridge started!
[Nov  1, 2018 13:12:54] [Matrix/INFO] Listening on 0.0.0.0:8081
[Nov  1, 2018 13:12:54] [DEBUG] Updating bot profile
[Nov  1, 2018 13:12:54] [User/@progserega:rsprim.ru/DEBUG] Connecting to WhatsApp
[Nov  1, 2018 13:13:14] [User/@progserega:rsprim.ru/ERROR] Failed to connect to WhatsApp: couldn't dial whatsapp web websocket: dial tcp 169.60.79.3:443: i/o timeout

systemctl status mautrix_whatsapp_bridge.service 
● mautrix_whatsapp_bridge.service - Whatsapp matrix bridge
   Loaded: loaded (/etc/systemd/system/mautrix_whatsapp_bridge.service; disabled; vendor preset: enabled)
   Active: active (running) since Thu 2018-11-01 13:12:54 +10; 1 day 3h ago
 Main PID: 22367 (mautrix-whatsap)
    Tasks: 7 (limit: 9830)
   CGroup: /system.slice/mautrix_whatsapp_bridge.service
           └─22367 /opt/mautrix_whatsapp/bin/mautrix-whatsapp

ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [DEBUG] Initializing Matrix event processor
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [DEBUG] Initializing Matrix event handler
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [INFO] Bridge initialization complete, starting...
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [DEBUG] Starting application service HTTP server
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [DEBUG] Starting event processor
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [INFO] Bridge started!
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [Matrix/INFO] Listening on 0.0.0.0:8081
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [DEBUG] Updating bot profile
ноя 01 13:12:54 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:12:54] [User/@progserega:rsprim.ru/DEBUG] Connecting to WhatsApp
ноя 01 13:13:14 bridge-im mautrix-whatsapp[22367]: [Nov  1, 2018 13:13:14] [User/@progserega:rsprim.ru/ERROR] Failed to connect to WhatsApp: couldn't dial whatsapp web websocket: dial tcp 169.60.79.3:443: i/o timeout

 ps aux|grep whatsa
mautrix+ 22367  0.0  0.2 432432 12288 ?        Ssl  ноя01   0:00 /opt/mautrix_whatsapp/bin/mautrix-whatsapp
root     28954  0.0  0.0  14004   996 pts/2    S+   17:10   0:00 grep whatsa

Bridge setup syntax error

sorry did not check the version :D

Integrate improvements into Rhymen/go-whatsapp?

HI @tulir , hi @Rhymen, how are you? Thank you for the great work you did on WhatsApp handling in Go.

I'm now working on WhatsApp bridge for Matterbridge: 42wim/matterbridge#475

It would be great to reuse the improvements / extra features @tulir you have built on top of https://github.com/Rhymen/go-whatsapp in https://github.com/tulir/mautrix-whatsapp/tree/master/whatsapp-ext.

For now I'm importing mautrix-whatsapp/whatsapp-ext, but I think it would be valuable for other reusers to keep WhatsApp related code in one library. That would require some work on @Rhymen side also, so I wonder if you had chatted about it and what do you think about it?

All the best!

New conversations from numbers not in the contacts list don't get bridged

If somebody starts a conversation with you via WhatsApp, which is not in your contact list, the invitation to join the chat doesn't get send too Matrix. This means you periodically have to actually check using the official WhatsApp client if you are missing any conversations, which is especially troubling if you use WhatsApp from a headless VM.

Add verify_ssl option

My homeserver only has a self-signed certificate which leads to this error:

[Sep 25, 2018 15:23:06] [WARN] Failed to update bot avatar: Post https://$homeserver:8448/_matrix/client/r0/register?access_token=t2amBrOienrTRDciHgs3J1AbG3hdSTBRGLmoKwPh51mlPSMm4GQZeJOwQ2PAgt0c&user_id=%40whatsappbot%3A$homeserver: x509: certificate signed by unknown authority

A bunch of (new?) errors on startup with latest docker image

[Sep  4, 2018 06:33:37] [Puppet/<phone-nr>@s.whatsapp.net/WARN] Failed to set display name: msg=Failed to POST JSON to /_matrix/client/r0/register code=400 wrapped=M_EXCLUSIVE: Invalid user localpart for this application service.

and

[Sep  4, 2018 06:33:37] [Puppet/<phone-nr>@s.whatsapp.net/ERROR] Failed to upload avatar: msg=Upload request failed: {"errcode":"M_FORBIDDEN","error":"Application service cannot masquerade as this user."} code=403 wrapped=

not migrated nicknames fro new chats

In chats user have no nicknames.

Steps to reproduce:

Whatsapp-mobile have chat with "user1" in history and some public rooms.
scan QR-code, link mautrix-bridge to whatsapp-account.
In matrix show invites for public rooms (from whatsapp) and invite for private-chat of "user1".
Nickname of "user1" success migrate to matrix.
After success init mautrix-whatsapp, "user2" type new message to my whatsapp account
In matrix I see "invite"
Accept invite and "user2" in matrix have no nickname
Also all users in public rooms have no its nicknames.

Reconnect WebSocket to WhatsApp on close (probably with some random delay)

Currently I sometimes get the following log

[Sep 18, 2018 06:01:39] [User/@test22:server/DEBUG] JSON message: ["Stream","asleep"]                                                                                                                                                
[Sep 18, 2018 07:40:36] [User/@test22:server/ERROR] WhatsApp error: unexpected websocket close: websocket: close 1000 (normal)                                                                                                       
error writing to socket: websocket: close senterror writing to socket: websocket: close sent[...]

So the device falls to sleep and after some timeout WhatsApp seems to close the socket.
The bridge currently needs a restart until it works again

Inject messages with corresponding time sent on Whatsapp

When the bridge hangs and is restarted, a flurry of messages come in, all being sent at the current time. This time doesn't match up with when they were actually sent on WhatsApp, and thus causes confusing about when these messages actually were sent.

Appservices have the ability to inject their own timestamps to solve this problem. Hopefully the bridge can make use of this as well, assuming WhatsApp tells you the time the message was actually sent.

Bridge panics after a few hours

There's several different panics after running it for a few hours:

panic: repeated read on failed websocket connection

goroutine 46 [running]:
github.com/gorilla/websocket.(*Conn).NextReader(0xc0000d4000, 0xf70de0, 0xc000281312, 0x2, 0xc00028a000, 0xc000186900)
        /go/pkg/mod/github.com/gorilla/[email protected]/conn.go:967 +0x356
github.com/gorilla/websocket.(*Conn).ReadMessage(0xc0000d4000, 0xa15100, 0xc00050d5b0, 0x1, 0x1, 0xb57dc0, 0xc00050d5b0)
        /go/pkg/mod/github.com/gorilla/[email protected]/conn.go:1028 +0x2f
github.com/Rhymen/go-whatsapp.(*Conn).readPump(0xc00015a270)
        /go/pkg/mod/github.com/!rhymen/[email protected]/conn.go:272 +0x73
created by github.com/Rhymen/go-whatsapp.NewConn
        /go/pkg/mod/github.com/!rhymen/[email protected]/conn.go:129 +0x1ff

[Feb 25, 2019 17:27:51] [Command handler/DEBUG] Importing all contacts of &{0xc000089040 0xc0004340e0 0xc0001620a0 0xc0000894a0 true true}
panic: runtime error: slice bounds out of range

goroutine 51 [running]:
github.com/Rhymen/go-whatsapp.(*Conn).decryptBinaryMessage(0xc0001955f0, 0xc000248260, 0xe, 0x10, 0xe, 0x10, 0x800a)
        /go/pkg/mod/github.com/!rhymen/[email protected]/conn.go:371 +0x3cc
github.com/Rhymen/go-whatsapp.(*Conn).query(0xc0001955f0, 0xac1480, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /go/pkg/mod/github.com/!rhymen/[email protected]/contact.go:204 +0x3b0
github.com/Rhymen/go-whatsapp.(*Conn).Contacts(0xc0001955f0, 0xc000434760, 0x2, 0x2)
        /go/pkg/mod/github.com/!rhymen/[email protected]/contact.go:128 +0x76
main.(*CommandHandler).CommandSync(0xc00018b400, 0xc0000b7ef8)
        /go/src/github.com/mautrix-whatsapp/commands.go:160 +0x112
main.(*CommandHandler).Handle(0xc00018b400, 0xc0002ee720, 0x1f, 0xc00017b860, 0xc000024ef8, 0x4)
        /go/src/github.com/mautrix-whatsapp/commands.go:87 +0x2f8
main.(*MatrixHandler).HandleMessage(0xc00017b290, 0xc00008c480)
        /go/src/github.com/mautrix-whatsapp/matrix.go:161 +0x210
main.(*MatrixHandler).HandleMessage-fm(0xc00008c480)
        /go/src/github.com/mautrix-whatsapp/matrix.go:43 +0x34
created by maunium.net/go/mautrix-appservice.(*EventProcessor).Start
        /go/pkg/mod/maunium.net/go/[email protected]/eventprocessor.go:62 +0x55

The installed version is 485acf6

Feature request: use a whatsapp user as a relaybot

In order to support bridging to other rooms, use whatsapp user as a relaybot for all incoming messages (prepend username).

Question about WhatsApp in a non-private room

Hi,
at the moment the WhatsApp Bot only can be used one to one (in a private chat/room). I have a senseful use case where i could need to put a WhatsApp Account into a group with many other people. That case is our Whatsapp business account which is used by more than one privileged person and in our case this account is not put to a mobile phone number, but to a landline call number. We like to share the output (the received messages from the people who are going to write us) to a matrix group.

Alternative would be to have some kind of bot redirect. A bot which forwards each received message to a user in the target group or something like that.

Are there any plans to deal for something like this?

kind regards, Mario

Synapse exception on talking to appservice

Hi all,

I'm having some issues getting this set up 😕 To be fair this is the first time I've set up Synapse or any appservice.

Currently my Synapse is outputting the following in its logs. Specifically the last line is what I don't understand.

2019-03-10 05:23:58,510 - synapse.http.client - 265 - INFO - as-recoverer-whatsapp-6 - Sending request PUT http://mautrixwhatsapp:8080/transactions/1?access_token=<redacted>
2019-03-10 05:23:58,516 - synapse.http.client - 286 - INFO - as-recoverer-whatsapp-6 - Received response to PUT http://mautrixwhatsapp:8080/transactions/1?access_token=<redacted>: 200
2019-03-10 05:23:58,517 - synapse.appservice.api - 249 - WARNING - as-recoverer-whatsapp-6 - push_bulk to http://mautrixwhatsapp:8080/transactions/1 threw exception Expecting value: line 1 column 1 (char 0)

I put mautrix-whatsapp as another service in my docker-compose.yml for Synapse, the service is called mautrixwhatsapp, so http://mautrixwhatsapp:8080 should be accessible from Synapse. The services are in the same network and port 8080 is also set as the appservice port in my mautrix-whatsapp config:

appservice:
  address: http://mautrixwhatsapp:8080
  hostname: 0.0.0.0
  port: 8080

This is all the log output of mautrix-whatsapp I'm seeing.

[Mar 10, 2019 05:40:12] [DEBUG] Initializing state store
[Mar 10, 2019 05:40:12] [DEBUG] Initializing database
[Mar 10, 2019 05:40:12] [DEBUG] Initializing Matrix event processor
[Mar 10, 2019 05:40:12] [DEBUG] Initializing Matrix event handler
[Mar 10, 2019 05:40:12] [INFO] Bridge initialization complete, starting...
[Mar 10, 2019 05:40:12] [DEBUG] Starting application service HTTP server
[Mar 10, 2019 05:40:12] [DEBUG] Starting event processor
[Mar 10, 2019 05:40:12] [INFO] Bridge started!
[Mar 10, 2019 05:40:12] [Matrix/INFO] Listening on 0.0.0.0:8080
[Mar 10, 2019 05:40:12] [DEBUG] Updating bot profile
[Mar 10, 2019 05:40:12] [WARN] Failed to update bot avatar: msg=Failed to PUT JSON to /_matrix/client/r0/profile/@whatsappbot:matrix.org/avatar_url code=401 wrapped=M_UNKNOWN_TOKEN: Invalid macaroon passed.
[Mar 10, 2019 05:40:12] [WARN] Failed to update bot displayname: msg=Failed to PUT JSON to /_matrix/client/r0/profile/@whatsappbot:matrix.org/displayname code=401 wrapped=M_UNKNOWN_TOKEN: Invalid macaroon passed.

Any help whatsoever would be much appreciated!

autosync

Hey, i am testing the bridge for some weeks now and recognized that sometimes i get messages to native whatsapp platform, which are sent recognized by my matrix server. I am able to retrieve those "lost" messages typing "sync --create" into the bot chat.

Is it possible to have some kind of automatic sync? If the user does not know he is missing answers from other persons, we will never get them except he enters "sync" again to get more messages.

by the way it would be good to get some bot feedback when entering "sync". So the bot could answer "syncing ... please wait" or something like that. Because syncing is sometimes slow and takes some minutes, the users wonders if it will do anything or could think its broken.

regards, Mario

Errors on startup

Some errors in the log I get on startup:

[Aug 28, 2018 10:55:05] [Database/Portal/ERROR] Database scan failed: sql: Scan error on column index 2: unsupported Scan, storing driver.Value type <nil> into type *string
[Aug 28, 2018 10:55:05] [Database/Portal/WARN] Failed to insert 358<phone-nr-1>@s.whatsapp.net->@user:domain.tld: UNIQUE constraint failed: portal.jid, portal.owner

And some more:

[Aug 28, 2018 10:55:05] [User/@user:domain.tld/Portal/358<phone-nr-1>@s.whatsapp.net/ERROR] Failed to create portal room: msg=Failed to POST JSON to /_matrix/client/r0/createRoom code=403 wrapped=M_FORBIDDEN: Application service has not registered this user
[Aug 28, 2018 10:55:05] [User/@user:domain.tld/Portal/358<phone-nr-2>-<ts?>@g.us/ERROR] Failed to download media: download failed
[Aug 28, 2018 10:55:06] [User/@user:domain.tld/Portal/358<phone-nr-3>@s.whatsapp.net/ERROR] Failed to download media: download failed

Handle contact sharing

Failed to update

Hello could you help me out here? Please
Failed to update bot avatar: msg=Failed to PUT JSON to /_matrix/client/r0/profile/@whatsappbot:matrix.org/avatar_url code=401 wrapped=M_UNKNOWN_TOKEN: Invalid macaroon passed

Support WhatsApp stickers

Currenty there is no indication on the Matrix side that a sticker has been send.

Error on go get

I get the following error when i run go get -u maunium.net/go/mautrix-whatsapp

package github.com/russross/blackfriday/v2: cannot find package "github.com/russross/blackfriday/v2" in any of: /usr/local/Cellar/go/1.11.5/libexec/src/github.com/russross/blackfriday/v2 (from $GOROOT) /Users/admin/go/src/github.com/russross/blackfriday/v2 (from $GOPATH)

Bridge crash after "list" - "pm"

Just doing a "list" command followed by a "pm xxxxxx" crashed the bridge with a panic. After the restart of the bridge the command worked. See the log part attached:

Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: [Mar 21, 2019 15:19:45] [Command handler/DEBUG] Importing [email protected] for &{0xc000226460 0xc0002d40e0 0xc00009b9a0 0xc0002277c0 false true}
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: panic: runtime error: invalid memory address or nil pointer dereference
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x788fee]
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: goroutine 2691 [running]:
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: github.com/Rhymen/go-whatsapp.(*Conn).write(0xc0001be0a0, 0x1, 0xc003ef6710, 0x10, 0xc00044e140, 0x4a, 0x50, 0xc0001f2380, 0x7f4f366eed98, 0x0)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/pkg/mod/github.com/!rhymen/[email protected]/write.go:91 +0x3e
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: github.com/Rhymen/go-whatsapp.(*Conn).writeJson(0xc0001be0a0, 0xc002ab0540, 0x3, 0x3, 0x100000000000099, 0x0, 0xa)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/pkg/mod/github.com/!rhymen/[email protected]/write.go:27 +0x2c7
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: github.com/Rhymen/go-whatsapp.(*Conn).GetProfilePicThumb(...)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/pkg/mod/github.com/!rhymen/[email protected]/contact.go:25
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: maunium.net/go/mautrix-whatsapp/whatsapp-ext.(*ExtendedConn).GetProfilePicThumb(0xc0002d40e0, 0xc002966580, 0x1b, 0x0, 0xc00010a8c0, 0xa)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/mautrix-whatsapp/whatsapp-ext/whatsapp.go:121 +0x101
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: main.(*Puppet).UpdateAvatar(0xc0035fe320, 0xc000228a80, 0x0, 0x1)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/mautrix-whatsapp/puppet.go:132 +0x71d
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: main.(*Puppet).Sync(0xc0035fe320, 0xc000228a80, 0xc00050b4a0, 0x1b, 0xc0001a237a, 0x5, 0x0, 0x0, 0x0, 0x0)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/mautrix-whatsapp/puppet.go:194 +0x290
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: main.(*CommandHandler).CommandPM(0xc0001a5ea0, 0xc003451ef8)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/mautrix-whatsapp/commands.go:277 +0x41d
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: main.(*CommandHandler).Handle(0xc0001a5ea0, 0xc002954880, 0x1e, 0xc000228a80, 0xc003d723e0, 0xf)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/mautrix-whatsapp/commands.go:101 +0x20b
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: main.(*MatrixHandler).HandleMessage(0xc0001accc0, 0xc000590240)
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/mautrix-whatsapp/matrix.go:161 +0x203
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: created by maunium.net/go/mautrix-appservice.(*EventProcessor).Start
Mar 21 15:19:45 fblchat mautrix-whatsapp[14915]: #011/opt/golang/pkg/mod/maunium.net/go/[email protected]/eventprocessor.go:62 +0x55
Mar 21 15:19:45 fblchat systemd[1]: whatsapp-bridge.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Mar 21 15:19:45 fblchat systemd[1]: whatsapp-bridge.service: Unit entered failed state.
Mar 21 15:19:45 fblchat systemd[1]: whatsapp-bridge.service: Failed with result 'exit-code'.

bot will not respond in a docker-compose environment

Setup:
Docker-Compose file with synapse and traefik as load balancer.

version: '3.7'

services:
  synapse:
    image: docker.io/matrixdotorg/synapse:latest
    restart: unless-stopped
    environment:
      - SYNAPSE_SERVER_NAME=[DOMAIN]
      - SYNAPSE_REPORT_STATS=no
      - SYNAPSE_ENABLE_REGISTRATION=yes
      - SYNAPSE_LOG_LEVEL=INFO
      - SYNAPSE_NO_TLS=1
      - POSTGRES_DB=synapse
      - POSTGRES_USER=synapse
      - POSTGRES_HOST=sdb
      - POSTGRES_PASSWORD=changeme
    volumes:
      - ${CONFIG_DIR}/synapse/app:/data
    depends_on:
      - sdb
    ports:
      - 8008
    labels:
      - "traefik.enable=true"
      - "traefik.basic.frontend.rule=Host:[DOMAIN]"
      - "traefik.basic.port=8008"
      - "traefik.basic.protocol=http"

  whatsapp:
    image: tulir/mautrix-whatsapp
    restart: unless-stopped
    volumes:
      - ${CONFIG_DIR}/synapse/bridges/mautrix-whatsapp:/data
    ports:
      - 8080
    depends_on:
      - synapse

  sdb:
    image: docker.io/postgres:10-alpine
    environment:
      - POSTGRES_DB=synapse
      - POSTGRES_USER=synapse
      - POSTGRES_PASSWORD=changeme
    volumes:
      - ${CONFIG_DIR}/synapse/db:/var/lib/postgresql/data

I'm using following config file:

homeserver:
  address: http://synapse:8008
  domain: [DOMAIN]
appservice:
  address: http://whatsapp:8080
  hostname: 0.0.0.0
  port: 8080
  database:
    type: sqlite3
    uri: mautrix-whatsapp.db
  state_store_path: ./mx-state.json
  id: whatsapp
  bot:
    username: whatsappbot
    displayname: WhatsApp bridge bot
    avatar: mxc://maunium.net/NeXNQarUbrlYBiPCpprYsRqr
  as_token: [STRIPPED]
  hs_token: [STRIPPED]
bridge:
  username_template: whatsapp_{{.}}
  displayname_template: '{{if .Notify}}{{.Notify}}{{else}}{{.Jid}}{{end}} (WA)'
  command_prefix: '!wa'
  permissions:
    '@admin:example.com': 100
    example.com: 10
logging:
  directory: ./logs
  file_name_format: '{{.Date}}-{{.Index}}.log'
  file_date_format: "2006-01-02"
  file_mode: 384
  timestamp_format: Jan _2, 2006 15:04:05
  print_level: debug

The log file puts out a warning about the avatar, but nothing else:

Registration generated. Add the path to the registration to your Synapse config restart it, then start the bridge.                                                                            
Didn't find a registration file.
Generated one for you.
Copy that over to synapses app service directory.
[Mar 28, 2019 10:02:47] [DEBUG] Initializing state store
[Mar 28, 2019 10:02:47] [DEBUG] Initializing database
[Mar 28, 2019 10:02:47] [DEBUG] Initializing Matrix event processor
[Mar 28, 2019 10:02:47] [DEBUG] Initializing Matrix event handler
[Mar 28, 2019 10:02:47] [INFO] Bridge initialization complete, starting...
[Mar 28, 2019 10:02:47] [DEBUG] Starting application service HTTP server
[Mar 28, 2019 10:02:47] [DEBUG] Starting event processor
[Mar 28, 2019 10:02:47] [Matrix/INFO] Listening on 0.0.0.0:8080
[Mar 28, 2019 10:02:47] [INFO] Bridge started!
[Mar 28, 2019 10:02:47] [DEBUG] Updating bot profile
[Mar 28, 2019 10:02:47] [DEBUG] Starting users
[Mar 28, 2019 10:02:47] [WARN] Failed to update bot avatar: msg=Failed to PUT JSON to /_matrix/client/r0/profile/@whatsappbot:[DOMAIN]/avatar_url code=401 wrapped=M_UNKNOWN_TOKEN: Invalid macaroon passed.
[Mar 28, 2019 10:02:47] [WARN] Failed to update bot displayname: msg=Failed to PUT JSON to /_matrix/client/r0/profile/@whatsappbot:[DOMAIN]/displayname code=401 wrapped=M_UNKNOWN_TOKEN: Invalid macaroon passed.

In the webclient, the bot doesnt answer with the expected "This room has been registered as your bridge management/status room."