Comments (6)
Nice graphs! I think I will forward small messages (<4096 bytes) directly and big messages by parts. Also, for big messages it is possible to implement speed throttling on tcp level.
I tested the official proxy and I managed it to occupy a 1MB per client. The interesting thing, when I tried to test with about 300 clients, the official client always crashes:
I sent the program which predictably crashes a proxy by its address to the official proxy authors, hope they will fix soon.
from mtprotoproxy.
Why do you need it? Does the scheme with multiplexing give some advantages?
from mtprotoproxy.
I personally don't need this, I have my own implementation now =) But talking about advantages: it's resource consumption / utilization + latency.
Telegram tcp socket connections in general are not very active. You spend more time doing nothing, waiting for events to be sent / received. So, you can have a lots of connections and relatively small amount of network traffic. For example, on one of my nodes I have ~17k client connections and just 6MiB/s traffic. But each socket connection occupies some memory in kernel and userspace buffers and structs (on this node it uses more than 1Gb of RAM).
With current schema we have 2x number of open sockets to number of client connections. In multiplexed scheme we could have 1x + Const
where const will be equal to number of datacenter IPs or a bit more, so this will help us to lower memory consumption a lot by the price of code complexity.
Another reason is connection setup time. Each time TG client connects to our proxy, it waits for us to establish a connection to Telegram server. It takes some time (not that much, but still infinity times worse than 0 in case when connection was already established).
FYI, here is a heatmap chart of how long does it take to connect to Telegram server from my proxy nodes:
from mtprotoproxy.
I was having an another idea how to get lower latency and memory consumption. Now, all proxy-servers with advertising, including an official one, read a full message from a one side of connection and then forward it to the other side. The messages can have up to 1MByte size. The proxy stores it into memory until it is fully read, so in the worst case we can have a big memory consumption (1MByte per client + socket buffers) and big latency.
The idea is to send a message as early as possible. For example, when we got the first 10KBytes of 1MByte message, we can reformat and forward that bytes to the connection and wait for more data.
In the nearest future I am planning to stabilize the current program, release the first version and write about it on some site like habr.com. After that I plan to work on memory consumption and lattency.
Now I set up test proxies on Digital Ocean droplets and trying to advertise them by telegram channels. I have only about 1250 users connected, and the cpu and memory consumption are very low:
from mtprotoproxy.
The idea is to send a message as early as possible.
Yep, that's how it's done in official proxy, as far as I understand: https://github.com/TelegramMessenger/MTProxy/blob/master/mtproto/mtproto-proxy.c#L1814 and it also prevents from potential DoS attack.
But, FYI, I have monitoring for telegram protocol packet sizes and it's relatively small for majority of cases (UP is upstream == telegram client, Down is downstream == Telegram server), so, left chart is what telegram client sends to server and right one - what TG server sends to client:
from mtprotoproxy.
The messages can have up to 1MByte size.
Isn't it 64Mb?
https://github.com/alexbers/mtprotoproxy/blob/master/mtprotoproxy.py#L304
3 bytes = 24bits and you multiply it by 4, so: 2**24 * 4 / 1024 / 1024 = 64
from mtprotoproxy.
Related Issues (20)
- How to set limit total bandwidth usage per secret
- Ydh
- proxy
- unknown rpc ans type: b'\xdf\xa20W' HOT 1
- MTProxy Admin Bot dont accept my secret cause python script make it more than 32 hex. Please, help!
- . HOT 2
- .
- . HOT 1
- When open telegam app , it takes too long time wait for connecting to the proxy HOT 2
- Proxy works on Telegram Desktop (Windows) but not on Android Telegram
- Error in source code
- Does MTProto allow to route all a device internet traffic to the proxy? Like a VPN? HOT 1
- .
- IPV6 dont work when i add users
- IP white list method - amazon changer & checker - local server tunnel
- Installing Python MTProxy on CentOS 8 with Python update HOT 5
- port change doesnt work in config.py HOT 1
- Dokerfile for debian 12 HOT 1
- ERROR: Service 'mtprotoproxy' failed to build: The command '/bin/sh -c apt-get update && apt-get install --no-install-recommends -y python3 python3-uvloop python3-cryptography python3-socks libcap2-bin ca-certificates && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100 HOT 3
- It only works for a few minutes HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mtprotoproxy.