Giter VIP home page Giter VIP logo

rpcudp's Introduction

RPCUDP : RPC over UDP in Python

Build Status

RPC over UDP may seem like a silly idea, but things like the DHT Kademlia require it. This project is specifically designed for asynchronous Python 3 code to accept and send remote proceedure calls.

Because of the use of UDP, you will not always know whether or not a procedure call was successfully received. This isn't considered an exception state in the library, though you will know if a response isn't received by the server in a configurable amount of time.

Installation

pip install rpcudp

Usage

This assumes you have a working familiarity with asyncio.

First, let's make a server that accepts a remote procedure call and spin it up.

import asyncio
from rpcudp.protocol import RPCProtocol

class RPCServer(RPCProtocol):
    # Any methods starting with "rpc_" are available to clients.
    def rpc_sayhi(self, sender, name):
        return "Hello %s you live at %s:%i" % (name, sender[0], sender[1])

    # You can also define a coroutine
    async def rpc_sayhi_slowly(self, sender, name):
        await some_awaitable()
        return "Hello %s you live at %s:%i" % (name, sender[0], sender[1])

# start a server on UDP port 1234
loop = asyncio.get_event_loop()
listen = loop.create_datagram_endpoint(RPCServer, local_addr=('127.0.0.1', 1234))
transport, protocol = loop.run_until_complete(listen)
loop.run_forever()

Now, let's make a client script. Note that we do need to specify a port for the client as well, since it needs to listen for responses to RPC calls on a UDP port.

import asyncio
from rpcudp.protocol import RPCProtocol

async def sayhi(protocol, address):
    # result will be a tuple - first arg is a boolean indicating whether a response
    # was received, and the second argument is the response if one was received.
    result = await protocol.sayhi(address, "Snake Plissken")
    print(result[1] if result[0] else "No response received.")

# Start local UDP server to be able to handle responses
loop = asyncio.get_event_loop()
listen = loop.create_datagram_endpoint(RPCProtocol, local_addr=('127.0.0.1', 4567))
transport, protocol = loop.run_until_complete(listen)

# Call remote UDP server to say hi
func = sayhi(protocol, ('127.0.0.1', 1234))
loop.run_until_complete(func)
loop.run_forever()

You can run this example in the examples folder (client.py and server.py).

Logging

This library uses the standard Python logging library. To see debut output printed to STDOUT, for instance, use:

import logging

log = logging.getLogger('rpcudp')
log.setLevel(logging.DEBUG)
log.addHandler(logging.StreamHandler())

Implementation Details

The protocol is designed to be as small and fast as possible. Python objects are serialized using MsgPack. All calls must fit within 8K (generally small enough to fit in one datagram packet).

rpcudp's People

Contributors

bashkirtsevich avatar bmuller avatar csunny avatar glindstedt avatar jayvdb avatar justheuristic avatar kigawas avatar wangjunstf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

rpcudp's Issues

Python 3.11: AttributeError: module 'asyncio' has no attribute 'coroutine'

/site-packages/rpcudp/protocol.py:71> exception=AttributeError("module 'asyncio' has no attribute 'coroutine'")>
Traceback (most recent call last):
File "[...]lib/python3.11/site-packages/rpcudp/protocol.py", line 83, in _accept_request
func = asyncio.coroutine(func)
^^^^^^^^^^^^^^^^^
AttributeError: module 'asyncio' has no attribute 'coroutine'

Support for generator-based coroutines is deprecated and is removed in Python 3.11.
( https://docs.python.org/3.10/library/asyncio-task.html#generator-based-coroutines )

As explored in the pull-request #20 ,
one could wrap the plain function in a coroutine
or call the function directly, without await.

Refer: #20 (comment)

if not asyncio.iscoroutinefunction(func):
    response = func(address, *args)
else:
    response = await func(address, *args)

Invalid python 3 modulo operator usage for byte string.

2016-01-16 14:42:22-0500 [-] Traceback (most recent call last):
2016-01-16 14:42:22-0500 [-]   File "example.py", line 28, in <module>
2016-01-16 14:42:22-0500 [-]     client.sayhi(('127.0.0.1', 1234), "Snake Plissken").addCallback(client.handleResult)
2016-01-16 14:42:22-0500 [-]   File "/home/f483/dev/upstream/rpcudp/rpcudp/protocol.py", line 99, in func
2016-01-16 14:42:22-0500 [-]     txdata = b'\x00%s%s' % (msgID, data)
2016-01-16 14:42:22-0500 [-] TypeError: unsupported operand type(s) for %: 'bytes' and 'tuple'

How to understand RPCProtocol.__getattr__

Hey @bmuller. As you can probably already tell, I've been reading a fair bit of your work in the past few days. :-)

Right now I'm trying to understand (and maybe help you to write a docstring for) RPCProtocol.getattr.

If I understand correctly, it behaves normally for attrs which begin with "_" or "rpc_".

For others, it behaves normally if they exist but otherwise, instead of raising AttributeError, it returns a closure that will attempt to call the function on a remote node?

Add authentication and end to end encryption.

Currently packets are blindly trusted, leaving authentication up to the user and the same for encryption. In a post Snowden world this is no longer acceptable.

Good options for encryption and authentication now exist for memory constrained protocols, as shown by bitcoin. Which authentication/encryption schema to use is however subject to much debate.

PS: Since this would break compatibility, I'm also adding additional issues that also would require a break. This should keep everyone on the same page as to a possible future direction.

Remove unnecessary 20 byte message overhead.

Currently every packet includes a 20 byte message id overhead which is costly in terms of reliability. It could be entirely removed if the hash of the packet were used as the message id.

This does burden the user to ensure each packet contains enough entropy, as to avoid a collision. But this can be done by the user with a much smaller nonce, if the data is expected to not contain enough entropy.

To maintain some level of backwards compatibility additional datagram prefixes could be used, b'\x02' b'\x03'. Though it is probably with breaking compatibility altogether.

PS: I plan eventually adding end to end encryption to rpcudp, or forking if it makes more sense. Since this would break compatibility. I'm adding additional issues that also would require a break. This should keep everyone on the same page as to a possible future direction.

Enable fire and forget rpc calls.

Currently the protocol assumes that every request is to receive a response. Since the protocol is for UDP where lost packets are to be expected, users cannot rely on getting a response and may design there usage so that it is not required.

For this case it can be of an advantage to enable fire and forget calls that do not require a response and thus reduce traffic. To enable this an additional datagram prefix could be used.

PS: I plan eventually adding end to end encryption to rpcudp, or forking if it makes more sense. Since this would break compatibility. I'm adding additional issues that also would require a break. This should keep everyone on the same page as to a possible future direction.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.