Giter VIP home page Giter VIP logo

matrix2051's Introduction

Matrix2051

Join Matrix from your favorite IRC client

Matrix2051 (or M51 for short) is an IRC server backed by Matrix. You can also see it as an IRC bouncer that connects to Matrix homeservers instead of IRC servers. In other words:

         IRC client
  (eg. weechat or hexchat)
             |
             |     IRC protocol
             v
         Matrix2051
             |
             |     Matrix protocol
             v
     Your Homeserver
     (eg. matrix.org)

Goals:

  1. Make it easy for IRC users to join Matrix seamlessly
  2. Support existing relay bots, to allows relays that behave better on IRC than existing IRC/Matrix bridges
  3. Bleeding-edge IRCv3 implementation
  4. Very easy to install. This means:
    1. as little configuration and database as possible (ideally zero)
    2. small set of depenencies.

Non-goals:

  1. Being a hosted service (it would require spam countermeasures, and that's a lot of work).
  2. TLS support (see previous point). Just run it on localhost. If you really need it to be remote, access it via a VPN or a reverse proxy.
  3. Connecting to multiple accounts per IRC connection or to other protocols (ร  la Bitlbee). This conflicts with goals 1 and 4.
  4. Implementing any features not natively by both protocols (ie. no need for service bots that you interract with using PRIVMSG)

Major features

  • Registration and password authentication
  • Joining rooms
  • Sending and receiving messages (supports formatting, multiline, highlights, replying, reacting to messages)
  • Partial IRCv3 ChatHistory support; enough for Gamja to work. open chathistory issues
  • Partial display name support

Shortcomings

  • Direct chats are shown as regular channels, with random names
  • Does not "feel" like a real IRC network (yet?)
  • User IDs and room names are uncomfortably long
  • Loading the nick list of huge rooms like #matrix:matrix.org overloads some IRC clients
  • IRC clients without hex color support will see some garbage instead of colors. (Though colored text seems very uncommon on Matrix)
  • IRC clients without advanced IRCv3 support work miss out on many features: quote replies, reacts, display names.

Screenshot

screenshot of #synapse:matrix.org with Element and IRCCloud side-by-side

Two notes on this screenshot:

  • Message edits are rendered with a fallback, as message edits are not yet supported by IRC,
  • Replies on IRCCloud are rendered with colored icons, and clicking these icons opens a column showing the whole thread. Other clients may render replies differently.

Usage

  • Install system dependencies. For example, on Debian: sudo apt install elixir erlang erlang-dev erlang-inets erlang-xmerl
  • Install Elixir dependencies: mix deps.get
  • Run tests to make sure everything is working: mix test
  • Run: mix run matrix2051.exs
  • Connect a client to localhost:2051, with the following config:
    • no SSL/TLS
    • SASL username: your full matrix ID (user:homeserver.example.org)
    • SASL password: your matrix password

See below for extra instructions to work with web clients.

See INSTALL.md for a more production-oriented guide.

Architecture

  • matrix2051.exs starts M51.Application, which starts M51.Supervisor, which supervises:
    • config.ex: global config agent
    • irc_server.ex: a DynamicSupervisor that receives connections from IRC clients.

Every time irc_server.ex receives a connection, it spawns irc_conn/supervisor.ex, which supervises:

  • irc_conn/state.ex: stores the state of the connection
  • irc_conn/writer.ex: genserver holding the socket and allowing to write lines to it (and batches of lines in the future)
  • irc_conn/handler.ex: task busy-waiting on the incoming commands from the reader, answers to the simple ones, and dispatches more complex commands
  • matrix_client/state.ex: keeps the state of the connection to a Matrix homeserver
  • matrix_client/client.ex: handles one connection to a Matrix homeserver, as a single user
  • matrix_client/sender.ex: sends events to the Matrix homeserver and with retries on failure
  • matrix_client/poller.ex: repeatedly asks the Matrix homeserver for new events (including the initial sync)
  • irc_conn/reader.ex: task busy-waiting on the incoming lines, and sends them to the handler

Utilities:

  • matrix/raw_client.ex: low-level Matrix client / thin wrapper around HTTP requests
  • irc/command.ex: IRC line manipulation, including "downgrading" them for clients that don't support some capabilities.
  • irc/word_wrap.ex: generic line wrapping
  • format/: Convert between IRC's formatting and org.matrix.custom.html
  • matrix_client/chat_history.ex: fetches message history from Matrix, when requested by the IRC client

Questions

Why?

There are many great IRC clients, but I can't find a Matrix client I like. Yet, some communities are moving from IRC to Matrix, so I wrote this so I can join them with a comfortable client.

This is also a way to prototype the latest IRCv3 features easily, and for me to learn the Matrix protocol.

What IRC clients are supported?

In theory, any IRC client should work. In particular, I test it with Gamja, IRCCloud, The Lounge, and WeeChat.

Please open an issue if your client has any issue.

What Matrix homeservers are supported?

In theory, any, as I wrote this by reading the Matrix specs. In practice, this is only tested with Synapse.

A notable exception is registration, which uses a Synapse-specific API as Matrix itself does not specify registration.

Please open an issue if you have any issue with your homeserver (a dummy login/password I can use to connect to it would be appreciated).

Are you planning to support features X, Y, ...?

At the time of writing, if both Matrix and IRC/IRCv3 support them, Matrix2051 likely will. Take a look at the list of open 'enhancement' issues.

A notable exception is direct messages, because Matrix's model differs significantly from IRC's.

Can I connect with a web client?

To connect web clients, you need a websocket gateway. Matrix2051 was tested with KiwiIRC's webircgateway (try this patch if you need to run it on old Go versions).

Here is how you can configure it to connect to Matrix2051 with Gamja:

[fileserving]
enabled = true
webroot = "/path/to/gamja"


[upstream.1]
hostname = "localhost"
port = 2051
tls = false
# Connection timeout in seconds
timeout = 20
# Throttle the lines being written by X per second
throttle = 100
webirc = ""
serverpassword = ""

What's with the name?

This is a reference to xkcd 1782:

2004: Our team stays in touch over IRC. 2010: Our team mainly uses Skype, but some of us prefer to stick to IRC. 2017: We've got almost everyone on Slack, But three people refuse to quit IRC and connect via gateway. 2051: All consciousnesses have merged with the Galactic Singularity, Except for one guy who insists on joining through his IRC client. "I just have it set up the way I want, okay?!" Sigh

I still have a question, how can I contact you?

Join #matrix2051 at irc.interlinked.me. (No I am not eating my own dogfood, I still prefer "native" IRC.)

matrix2051's People

Contributors

jlu5 avatar mk-fg avatar progval avatar thaodan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

matrix2051's Issues

Display names in the nick list

IRC only supports two kinds of display names:

We use the latter, but it's not great, as it does not show up in the nick list (and doing so would be out of scope for the spec)

Send server name as PONG parameter

The spec says:

Servers MUST send a parameter, and clients SHOULD ignore it. It exists for historical reasons, and indicates the name of the server sending the PONG. Clients MUST NOT send a parameter.

But Matrix2051 does not actually do it, and only sends the cookie/token.

Option to use display name rather than mxid

For native Matrix users, the mxid is usually at least somewhat descriptive, but for users bridged in from other protocols the mxid is almost universally useless. For example, any matrix channel that bridges in users from discord is going to have a lot of users that look fine in (say) Element but show up as "discord_1234567890:example.com" in M51.

It looks like M51 sends this in the draft/display-name tag, but weechat doesn't support this.

I suspect that doing this as a quick hack wouldn't be particularly hard, and I may prototype it locally -- write a mangler that flattens Matrix display names into something IRC can compass, change the m.room.message handler and send_multiline_batch so that use that instead of nick2nuh(mxid) as the sender. Making it an option (not necessarily stateful, it can be a command line flag or envar or so) will be more annoying, making it robust (so that, e.g., @- or :-mentions of the munged name on the IRC side get translated into @-mentions of the correct mxid matrix-side) will, I think, take it well past "quick hack" status -- but I may accept an inability to reliably @ people if the alternative is not being able to tell who's talking at all, which is the status quo.

Packaging

Hey,

I have a few questions regarding packaging this app.

  • Why does mix try to install hex and rebar even thou it is already already installed with elixir?
  • Can I make mix deps.get stop asking questions?

Various issues with circe

I noticed this issue when connecting with emacs-circe:

Arguments are either of the two:

:<server> 333 <target> <channel> <nick> 1434996762
:<server> 333 <target> <channel> <nick>!<user>@<host> 1434996803" (let ((channel-buffer (circe-server-get-chat-buffer channel)) (topic-time (string-to-number topic-time))) (save-current-buffer (set-buffer (or channel-buffer (circe-server-last-active-buffer))) (circe-display (if channel-buffer 'circe-format-server-topic-time 'circe-format-server-topic-time-for-channel) :nick target :channel channel :setter (irc-userstring-nick setter) :setter-userhost (or (irc-userstring-userhost setter) "(unknown)") :topic-time topic-time :topic-date (current-time-string (seconds-to-time topic-time)) :topic-ago (circe-duration-string (- (float-time) topic-time)))))) 6) (args were (#<process  bnc.server.de<1>> "333" "server." "matrixserver" "channel" "0"))
Error running event nil handler circe--irc-display-event: (wrong-number-of-arguments ((circe-query-mode-abbrev-table circe-query-mode-syntax-table circe-channel-mode-abbrev-table circe-channel-mode-syntax-table circe-chat-mode-abbrev-table circe-chat-mode-syntax-table circe-server-mode-abbrev-table circe-server-mode-syntax-table circe-mode-abbrev-table circe-mode-syntax-table t) (_server ignored _numeric target channel setter topic-time) "Show a 333 numeric (RPL_TOPICWHOTIME).

I think this is what makes circe disconnect periodically.

Is there something with RPL_TOPICWHOTIME?

Add meaningful logging

Let's fill this stdout with some meaningful information instead of just the startup IO.inspect and crash stack traces

Send "rich" media types to Matrix

When IRC users post a link, it would be nice if we could detect whether it's an image/sound/... and set m.image/m.audio/... messages to Matrix instead of just the link as m.text

Rooms joined a while ago are ignored when they don't have a canonical alias

This typically happens for long-lived PMs.

I cannot figure out how to solve this.

This solves the issue, but causes rooms with a canonical alias to be shown as two channels (one named after the room ids and one after the canonical alias):

diff --git a/lib/matrix_client/poller.ex b/lib/matrix_client/poller.ex
index 11137ce..20c3d5b 100644
--- a/lib/matrix_client/poller.ex
+++ b/lib/matrix_client/poller.ex
@@ -357,6 +370,8 @@ defmodule M51.MatrixClient.Poller do
             %M51.Matrix.RoomMember{display_name: displayname}
           )
 
+        my_nick = M51.IrcConn.State.nick(irc_state)
         if !state_event and !was_already_member do
           my_nick = M51.IrcConn.State.nick(irc_state)
 
@@ -382,6 +397,15 @@ defmodule M51.MatrixClient.Poller do
           end
         end
 
+        if state_event and !was_already_member do
+          sender = nil
+          old_canonical_alias = nil
+          # make handle_joined_room() send channel welcome
+          {room_id, {sender, old_canonical_alias}}
+        else
+          nil
+        end
+
       "leave" ->
         params_tail =
           case event do
@@ -409,6 +433,8 @@ defmodule M51.MatrixClient.Poller do
           end
         end
 
+        nil
+
       "ban" ->
         if !state_event do
           send.(%M51.Irc.Command{
@@ -419,6 +445,8 @@ defmodule M51.MatrixClient.Poller do
           })
         end
 
+        nil
+
       "invite" ->
         if !state_event do
           send.(%M51.Irc.Command{
@@ -429,15 +457,17 @@ defmodule M51.MatrixClient.Poller do
           })
         end
 
+        nil
+
       _ ->
         send.(%M51.Irc.Command{
           tags: %{"account" => sender},
           command: "NOTICE",
           params: [channel, "Unexpected m.room.member event: " <> Kernel.inspect(event)]
         })
-    end
 
-    nil
+        nil
+    end
   end
 
   def handle_event(

M51 doesn't follow redirects when locating the homeserver

My homeserver runs on matrix.ancilla.ca. In order to use it without the leading matrix., the webserver at ancilla.ca serves redirects on the .well-known/matrix directory:

$ curl -i https://ancilla.ca/.well-known/matrix/client
HTTP/2 301 
location: https://matrix.ancilla.ca/.well-known/matrix/client

This works fine for Element, but M51 doesn't follow the redirect (linebreaks added and HTML removed for clarity):

10:06:59.150 [warning] Well-known request for https://ancilla.ca/.well-known/matrix/client returned
  %HTTPoison.Response{
    status_code: 301,
    body: "[elided]",
    headers: [{"Date", "Tue, 23 Jan 2024 15:06:59 GMT"}, {"Server", "Apache"}, {"Access-Control-Allow-Origin", "*"}, {"Location", "https://matrix.ancilla.ca/.well-known/matrix/client"}, {"Cache-Control", "max-age=600"}, {"Expires", "Tue, 23 Jan 2024 15:16:59 GMT"}, {"Content-Length", "259"}, {"Content-Type", "text/html; charset=iso-8859-1"}],
    request_url: "https://ancilla.ca/.well-known/matrix/client",
    request: %HTTPoison.Request{method: :get, url: "https://ancilla.ca/.well-known/matrix/client", headers: [], body: "", params: %{}, options: []}}.
  Falling back to https://ancilla.ca

Which obviously doesn't work. I've tried it with both 301 and 302 without success.

I can fix it temporarily by copying the .well-known from matrix.ancilla.ca to ancilla.ca and turning off the redirect, but M51 should handle redirects itself. I'm pretty rusty on Erlang but I'm pretty sure this is just a matter of setting follow_redirect to true when making the initial request via httpoison.

You're not a channel operator

I recently gave matrix2051 a try by cloning from github. My client is irssi. Two issues:

  1. After I join a channel (autojoin'd to channels I'm already in), I get the notice "You're not a channel operator". I'm not issuing any commands.
  2. I get lots of server restarts. How can I get some more debugging details here?
[...]
16:38:23.804 [info] Incoming connection from ::ffff:127.0.0.1:43934

16:39:01.847 [info] Incoming connection from ::ffff:127.0.0.1:41598

16:40:43.879 [warning] Server connection error [http-server-error], retrying after 1s

16:42:24.972 [warning] Server connection error [http-server-error], retrying after 2s

16:43:01.124 [info] Incoming connection from ::ffff:127.0.0.1:43418

16:43:23.150 [info] Incoming connection from ::ffff:127.0.0.1:46198

16:44:01.198 [info] Incoming connection from ::ffff:127.0.0.1:36036
[...]

Support direct messages

it's really hard, in Matrix they are actual rooms you can invite people to.

So currently, they are shown like a regular channel with a random name

Casemapping (RFC 3454) does not work for channel names

Hi,

Trying to use matrix2051 with ZNC-ERC bouncer-client combination, immediately bumped into error notice like following when sending message from client in a channel listed as !RrkYlRwPmZqHGhPFvf:foss.wtf:
[11:13:07] -server.- Error while sending message: {:room_not_found, "!rrkylrwpmzqhghpfvf:foss.wtf"}

I think the issue might be lack of casemapping for channel names, which client expects, but server does not seem to implement, since afaik on IRC !RrkYlRwPmZqHGhPFvf:foss.wtf and !rrkylrwpmzqhghpfvf:foss.wtf channel/user names should generally be equivalent.

matrix2051 advertises CASEMAPPING=rfc3454 in particular, which explicitly includes ASCII letters (listed at the start of B2/B3 appendixes there) and I think should be backwards-compatible in this case, even if client reads this as CASEMAPPING=ascii.

I'm not sure if there is an option to not support any case-mapping on IRC, as at least in ERC client, any unknown/missing CASEMAPPING values fallback to RFC 1459, which maps ascii letters + some non-letter characters, and there's no "none" option supported.

To report it more strictly:

  • What I did: tried sending message (any content) to a matrix channel !RrkYlRwPmZqHGhPFvf:foss.wtf, with ascii upper/lower-case letters in its name, from an ERC client (built into emacs text editor).

  • Expected result: message gets dispatched over Matrix protocol with no errors and can be seen in the channel there.

  • Actual result: [11:13:07] -server.- Error while sending message: {:room_not_found, "!rrkylrwpmzqhghpfvf:foss.wtf"} notice in response, message cannot be seen in Matrix channel.

  • Steps to reproduce:

    • Create any non-encrypted private channel, its name will almost certainly contain mixed-case letters in matrix2051 representation.

    • Use client that maps IRC names internally to some "canonical" representation using casemapping and resulting mapped names, or otherwise send PRIVMSG IRC command with channel name in all-upper/lower-case.

      For example, following command can be used in clients for testing: /msg <lowercase-chan-name> test

    • Note error notice in response instead of successful message send.

Wasn't able to find this reported under Issues from a quick look and search.
(if it is, maybe can include more keywords/info in the title)

Thanks.

(Jason.DecodeError) unexpected end of input at position 0

I noticed while testing the client with circe I get these random erros while lurking in #[email protected]:

00:30:54.864 [error] Task #PID<0.1237.0> started from #PID<0.1211.0> terminating
** (Jason.DecodeError) unexpected end of input at position 0
    (jason 1.3.0) lib/jason.ex:92: Jason.decode!/2
    (matrix2051 0.1.0) lib/matrix/raw_client.ex:45: M51.Matrix.RawClient.get/4
    (matrix2051 0.1.0) lib/matrix_client/poller.ex:83: M51.MatrixClient.Poller.poll_one/4
    (matrix2051 0.1.0) lib/matrix_client/poller.ex:59: M51.MatrixClient.Poller.loop_poll/3
    (elixir 1.13.2) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (stdlib 3.17.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Function: &M51.MatrixClient.Poller.poll/1
    Args: [{#PID<0.1211.0>}]

Re-add support for rooms with no canonical alias

For a little while now (since f515c97?), M51 waits for a canonical_alias event before sending any message from a room. This means that rooms without one are never shown.

Instead it should probably remember somehow which rooms were not announced, and do it at the end of each poll.

However, fixing this bug will have the downside of showing rooms that are unlikely to be interesting on the IRC side:

  • DM rooms (which are often encrypted)
  • spaces

and if a DM room is not encrypted, then there is a workaround: manually add a canonical alias (which also avoids showing the random room id). So is this really desirable?

Extra rooms created

When I joined one one of my accounts with pre-existing conversations, I got all my channels (good), but also a lot of 1:1 conversations with people I never directly talked to (bad). Those were composed of messages they sent to one of the rooms I'm subscribed to.

I'm connecting to Pantalaimon.

Implement CHATHISTORY BETWEEN

Currently not implemented because the only primitive we have is event context (somewhat equivalent to CHATHISTORY AROUND), and implementing CHATHISTORY BETWEEN on top of it can be very wasteful (or possibly loop indefinitely if we're unlucky).

So we need to be careful in implementing this.

Implement NAMES command

It should be a simple matter of re-using this code in lib/irc/handler.ex:

# send RPL_NAMREPLY
overhead = make_numeric.("353", ["=", channel, ""]) |> M51.Irc.Command.format() |> byte_size()
# note for later: if we ever implement prefixes, make sure to add them
# *after* calling nick2nuh; we don't want to have prefixes in the username part.
M51.MatrixClient.State.room_members(state, room_id)
|> Enum.map(fn {user_id, _member} ->
nuh = nick2nuh(user_id)
# M51.Irc.Command does not escape " " in trailing
String.replace(nuh, " ", "\\s") <> " "
end)
|> Enum.sort()
|> M51.Irc.WordWrap.join_tokens(512 - overhead)
|> Enum.map(fn line ->
line = line |> String.trim_trailing()
if line != "" do
# RPL_NAMREPLY
send_numeric.("353", ["=", channel, line])
end
end)
# RPL_ENDOFNAMES
send_numeric.("366", [channel, "End of /NAMES list"])

Suggested by @Thaodan in #14 (comment)

Test more clients

  • gamja
  • irccloud
  • weechat
  • the lounge
  • hexchat
  • limnoria
  • irssi
  • kiwi
  • quassel
  • ...

Rewrite M51.MatrixClient.Client.get_base_url to use cache

This function is now called on every mxc:// URL, ie. on about every image; and it needs to send a network request

It should be rewritten (as a GenServer?) to have some caching with expiry. A couple minutes should be good enough.

Encryption

Hey,

The bridge looks nice so far I wonder does it support encrypted rooms?

Support encrypted rooms

This will have to wait for libolm (or an alternative) to be documented; ideally with an Elixir/Erlang binding.

Error with resolving hex dependencies !

root@srv112118-206152:~/matrix2051# elixir --version
Erlang/OTP 22 [erts-10.6.4] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1]

Elixir 1.9.1 (compiled with Erlang/OTP 22)
root@srv112118-206152:~/matrix2051# MIX_ENV=prod mix deps.get
Resolving Hex dependencies...
** (UndefinedFunctionError) function Logger.should_log/2 is undefined or private. Did you mean one of:

  * __should_log__/1

(logger) Logger.__should_log__(:debug, Hex.Solver.Solver)
(hex) lib/hex/solver/solver.ex:169: Hex.Solver.Solver.add_incompatibility/2
(hex) lib/hex/solver/solver.ex:19: Hex.Solver.Solver.run/4
(hex) lib/hex/solver.ex:53: Hex.Solver.run/5
(hex) lib/hex/remote_converger.ex:114: Hex.RemoteConverger.run_solver/5
(mix) lib/mix/dep/converger.ex:95: Mix.Dep.Converger.all/4
(mix) lib/mix/dep/converger.ex:51: Mix.Dep.Converger.converge/4
(mix) lib/mix/dep/fetcher.ex:16: Mix.Dep.Fetcher.all/3

root@srv112118-206152:~/matrix2051#

Redactions

I would suggest that redactions could be handled as in other similar clients such as purple-slack.
E.g.
Chat:

<user> <data> bar bar

Reduction:

Reducted:  
<user> <data> bar bar
Reason:
bar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.