what process should I do on Debian 11? about rtorrent HOT 1 OPEN

If libtorrent wasn't installed, you wouldn't be able to start up rtorrent in the first place, let alone get that message (you can run apt list --installed | grep libtorrent to prove it if you want). That message is coming from the tracker of that specific torrent, so you'll need to contact an admin or find whichever wiki they're talking about in order to see what clients are allowed. It's odd that rTorrent isn't, since it's a very popular and stable client, but trackers can ban whatever clients they wish.

Just a heads up, there have been reports that Debian doesn't compile rTorrent with asynchronous DNS by default, so UDP trackers (primarily used for public torrents) may slow down a vanilla rtorrent installation on Debian.

from rtorrent.