namuyan / srudp Goto Github PK

Secure Reliable User Datagram Protocol implemented by pure Python

License: MIT License

Python 99.89% Batchfile 0.11%

rudp python nat-traversal udp-hole-punching asyncio udp nat

srudp's Introduction

Secure Reliable UDP

This repository provides Secure Reliable Data Stream that works like TCP.
My purpose is enable users create P2P connection between clients in closed NAT.

Features

Pure Python
Usage like normal socket object
Protocol similar to RUDP
UDP hole punching
high performance (4Mbps/s Up&Down when 10mb)
ipv4/ipv6
optional asyncio

Requirement

Python3.6+
requirements.txt

Installation

tutorial for users "cannot work on my condition"

pip3 install --user srudp

Tests

git clone https://github.com/namuyan/srudp
cd srudp
pip3 install --user -r requirements.txt
python3 -m mypy --config-file=mypy.ini srudp
python3 -m unittest discover

Demo

Prepare two independent PCs.

from srudp import SecureReliableSocket
from time import sleep, time
 
sock = SecureReliableSocket()
sock.connect(("<remote host 1>", 12345))
 
while not sock.is_closed:
    sock.sendall(b'hello ' + str(time()).encode())
    sleep(3)
print("closed", sock)

A side, send message hello once in a 3 sec.

from srudp import SecureReliableSocket
from time import time
 
sock = SecureReliableSocket()
sock.connect(("<remote host 2>", 12345))
 
while not sock.is_closed:
    data = sock.recv(1024)
    if not data:
        break
    print(time(), data)
print("closed", sock)

Another side, receive the message and show immediately.

asyncio usage

from srudp import SecureReliableSocket
import asyncio

# Get a reference to the current event loop
loop = asyncio.get_event_loop()

# create a socket
sock = SecureReliableSocket()

# connect() on another thread because block event loop
address = ("example.com", 3000)
await loop.run_in_executor(None, sock.connect, (address,))

# Register the open socket to wait for data
reader, writer = await asyncio.open_connection(sock=sock)

# read
data = await reader.read(1024)

# write
writer.write(b"hello")
writer.write(b"world")
await writer.drain()

# close
writer.close()

You can do just like a normal TCP socket. But if you don't intend, like HTTP protocol which requires a lot of connections, you don't have to use async method.

to avoid troubles

Do not think always success connection establish. Web-RTC detect UDP-hole-punching success, or use alternative way when failed, you need to implement it.
UDP (and some TCP) is often blocked on public network like airport free wifi and university local LAN etc. Addition to it, some router and network adapter sometimes block.
I designed this simple TCP like socket object. This don't have signaling function, haven't data specialized because I will use this as one of low-layer libraries for P2P.

Note: Why make this?

These days, PC is located in a local environment protected by NAT. It is difficult to transfer data between two outsides. In order to solve this problem, connection is realized by UDP hole punching without using UPnP.

UDP is a socket protocol with minimum functions for connecting applications. Therefore, there is no connection state, data may not be reachable, spoofing the source is easy. This is why, you cannot substitute it as TCP.

With this program, you can treat it just like TCP without worrying about the above problems. In other words, it has a connection state, guarantees data reachability, and is difficult to forge.

Author

@namuyan

Licence

MIT

srudp's People

Contributors

Stargazers

Watchers

Forkers

yumoqing krishnendu wonabru mrsipan aorizondo tuskaw forestj2 aashishkalra19 aishwaryasalunke23 youcannotburnmyshadow scott91e1 wahello ckninja2

srudp's Issues

Sending buffer bug after network fail

Hey, very cool project! Today i was testing the code in a long run, and this happend:

[DEBUG ] [MainThread] [2023-01-28 03:57:24,939 ] try to communicate addr=('192.168.0.30', 50001) bind=['', 50001]
[DEBUG ] [MainThread] [2023-01-28 03:57:24,940 ] select curve NIST256p (static)
[DEBUG ] [MainThread] [2023-01-28 03:57:25,234 ] success getting shared key
[DEBUG ] [MainThread] [2023-01-28 03:57:25,250 ] success hand shaking
[DEBUG ] [MainThread] [2023-01-28 03:57:25,498 ] success get MUT size 1472b
[DEBUG ] [MainThread] [2023-01-28 03:57:25,500 ] connect success! mtu=1472
[DEBUG ] [Thread-2  ] [2023-01-28 03:57:25,523 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:57:25,524 ] send now! loss=0 time=1674878245
[DEBUG ] [SRUDP     ] [2023-01-28 03:57:25,528 ] reached end of chunk seq=3
[DEBUG ] [SRUDP     ] [2023-01-28 03:57:25,532 ] allow sending operation again seq=3
[DEBUG ] [Thread-1  ] [2023-01-28 03:57:25,563 ] received! 4110b loss=0 79.25kb/s

[DEBUG ] [SRUDP     ] [2023-01-28 03:57:45,519 ] reached end of chunk seq=6
[DEBUG ] [Thread-2  ] [2023-01-28 03:57:45,558 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:57:45,559 ] send now! loss=0 time=1674878265
[DEBUG ] [Thread-2  ] [2023-01-28 03:57:45,562 ] send broadcast!
[DEBUG ] [Thread-1  ] [2023-01-28 03:57:45,564 ] received! 4110b loss=0 75.75kb/s

[DEBUG ] [SRUDP     ] [2023-01-28 03:57:45,567 ] allow sending operation again seq=6
[DEBUG ] [SRUDP     ] [2023-01-28 03:58:05,523 ] reached end of chunk seq=9
[DEBUG ] [SRUDP     ] [2023-01-28 03:58:05,526 ] find you!!! (Packet(BCT seq:0 retry:0 time:1674878285.51 data:27b))
[DEBUG ] [Thread-1  ] [2023-01-28 03:58:05,563 ] received! 4110b loss=0 83.46kb/s

[DEBUG ] [Thread-2  ] [2023-01-28 03:58:05,596 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:58:05,596 ] send now! loss=0 time=1674878285
[DEBUG ] [Thread-2  ] [2023-01-28 03:58:05,599 ] send broadcast!
[DEBUG ] [SRUDP     ] [2023-01-28 03:58:05,603 ] allow sending operation again seq=9
Exception in thread SRUDP:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/pi/srudp/srudp/__init__.py", line 449, in _backend
    self.sendto(self._encrypt(packet2bin(p)), self.address)
  File "/home/pi/srudp/srudp/__init__.py", line 667, in sendto
    return self.receiver_socket.sendto(data, address)
OSError: [Errno 101] Network is unreachable
[DEBUG ] [Thread-2  ] [2023-01-28 03:58:25,631 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:58:25,632 ] send now! loss=0 time=1674878305
[DEBUG ] [Thread-2  ] [2023-01-28 03:58:45,664 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:58:45,664 ] send now! loss=0 time=1674878325
[DEBUG ] [Thread-2  ] [2023-01-28 03:59:05,696 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:59:05,697 ] send now! loss=0 time=1674878345
[DEBUG ] [Thread-2  ] [2023-01-28 03:59:25,729 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:59:25,730 ] send now! loss=0 time=1674878365
[DEBUG ] [Thread-2  ] [2023-01-28 03:59:45,762 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 03:59:45,762 ] send now! loss=0 time=1674878385
[DEBUG ] [Thread-2  ] [2023-01-28 04:00:05,795 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:00:05,795 ] send now! loss=0 time=1674878405
[DEBUG ] [Thread-2  ] [2023-01-28 04:00:25,827 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:00:25,828 ] send now! loss=0 time=1674878425
[DEBUG ] [Thread-2  ] [2023-01-28 04:00:25,831 ] send broadcast!
[DEBUG ] [Thread-2  ] [2023-01-28 04:00:45,863 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:00:45,864 ] send now! loss=0 time=1674878445
[DEBUG ] [Thread-2  ] [2023-01-28 04:01:05,906 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:01:05,907 ] send now! loss=0 time=1674878465
[DEBUG ] [Thread-2  ] [2023-01-28 04:01:25,939 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:01:25,940 ] send now! loss=0 time=1674878485
[DEBUG ] [Thread-2  ] [2023-01-28 04:01:25,943 ] send broadcast!

The "Network is unreachable" it is intentional, i was testing how the code would behave in the case of that happening.
But the wired thing comes after:

[DEBUG ] [Thread-2  ] [2023-01-28 04:17:47,605 ] send now! loss=0 time=1674879467
[DEBUG ] [Thread-2  ] [2023-01-28 04:17:47,608 ] send broadcast!
[DEBUG ] [Thread-2  ] [2023-01-28 04:18:07,641 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:18:07,642 ] send now! loss=0 time=1674879487
[DEBUG ] [Thread-2  ] [2023-01-28 04:18:07,645 ] send broadcast!
[DEBUG ] [Thread-2  ] [2023-01-28 04:18:27,678 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:18:27,679 ] send now! loss=0 time=1674879507
[DEBUG ] [Thread-2  ] [2023-01-28 04:18:47,713 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:18:47,713 ] send now! loss=0 time=1674879527
[DEBUG ] [Thread-2  ] [2023-01-28 04:19:07,747 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:19:07,748 ] send now! loss=0 time=1674879547
[DEBUG ] [Thread-2  ] [2023-01-28 04:19:27,781 ] send operation success 4110b
[DEBUG ] [Thread-2  ] [2023-01-28 04:19:27,782 ] send now! loss=0 time=1674879567
[DEBUG ] [Thread-2  ] [2023-01-28 04:19:47,808 ] buffer is full and wait for signaled
[DEBUG ] [Thread-2  ] [2023-01-28 04:20:08,810 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:20:29,811 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:20:50,813 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:21:11,814 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:21:32,816 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:21:53,817 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:22:14,819 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:22:35,820 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:22:56,822 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:23:17,823 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:23:38,825 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:23:59,826 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:24:20,827 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:24:41,829 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:25:02,830 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:25:23,832 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:25:44,833 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:26:05,835 ] waiting for sending buffer have space..
[DEBUG ] [Thread-2  ] [2023-01-28 04:26:26,836 ] waiting for sending buffer have space..
(...)

Something wierd happend in the middle and i cant figure what, any idea?
Again, very good project, very good code structure, i will upload some PRs to contribute.

~ uint69_t

ConnectionError: timeout on hand shaking

Hey, if I am trying to get the demo working.

I get this Error:

Traceback (most recent call last):
  File "C:\Users\flipg\Desktop\sender.py", line 5, in <module>
    sock.connect(("149.224.169.189", 5050))
  File "C:\Users\flipg\AppData\Roaming\Python\Python37\site-packages\srudp\__init__.py", line 190, in connect
    self.bind(tuple(address_copy))
OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted

I replaced the <remote host 1,2> with the Ipv4 adress of each pc.
I want to connect to. Can you tell me what I am doing wrong ?

Clarification on how to proceed with UDP Hole Punching

So I've spent about a week trying to use your library(and yes I do know how hole punching works xD), but I can't seem to get it working. This is however probably an oversight on my end.

The code without srudp mirrors this and works perfectly.

But could you clarify if I need to use the connect() only on the two peers, or do I need to use it for establishing a connection with the control server as well? Basically a tutorial/example of sorts to implement hole punching would be great!

Thanks!

cannot send specific size of packet

Abstract

Some specific size of packet is always blocked depends on OS or Eth.

Detail

I find this problem when connect Windows10 to Ubuntu16.04 with expand MTU size big, but I don't understand the conditions under which the problem occurs. I think this may often occur on some condition because no one have checked if the UDP is working properly.

Fix

Minimum size of MUT is reliable and don't change the size, remove CONTROL_MTU future.

なんやねん

Don’t work on MacOS

sock = SecureReliableSocket()

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/srudp/__init__.py", line 201, in __init__
    super().connect(self_address)
OSError: [Errno 22] Invalid argument

asyncio is not working!

I am using it on windows 10. this is from the test...

await loop.run_in_executor(self.executor, sock1.connect, ("127.0.0.1", self.port))
await loop.run_in_executor(self.executor, sock2.connect, ("127.0.0.1", self.port))

reader1, writer1 = await asyncio.open_connection(sock=sock1)
reader2, writer2 = await asyncio.open_connection(sock=sock2)

now this writes without error...
writer1.write(b"nice world")
await writer1.drain()

but the data is not arriving on this... it just waits indefinitely
received = await reader2.read(1024)

and when I do this,... well I found out 😒 data is just sitting here and circling around. it is not being sent.
received = await reader1.read(1024)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.