Giter VIP home page Giter VIP logo

aiocontextvars's Introduction

aiocontextvars

image

image

IMPORTANT: This package will be deprecated after contextvars asyncio backport is fixed. Before then, this library experimentally provides the missing asyncio support for the contextvars backport library. Please read more in Python 3.7 contextvars documentation.

Compatibility

In Python 3.7 this package is 100% contextvars.

In Python 3.5 and 3.6, this package added asyncio support to the PEP-567 backport package also named contextvars, in a very different way than Python 3.7 contextvars implementation:

  1. call_soon() and family methods.

Python 3.7 added keyword argument context to call_soon() and its family methods. By default those methods will copy (inherit) the current context and run the given method in that context. But aiocontextvars won't touch the loop, so in order to achieve the same effect, you'll need to:

loop.call_soon(copy_context().run, my_meth)
  1. Task local.

Python 3.7 used above keyword argument context in Task to make sure that each step of a coroutine is ran in the same context inherited at the time its driving task was created. Meanwhile, aiocontextvars uses Task.current_task() to achieve similar effect: it hacks asyncio and attaches a copied context to the task on its creation, and replaces thread local with current task instance to share the context. This behaves identically to Python 3.7 in most times. What you need to do is to import aiocontextvars before creating loops.

  1. Custom tasks and loops.

Because above hack is done by replacing asyncio.get_event_loop and loop.create_task, therefore tasks and loops created by custom/private API won't behave correctly as expected, e.g. uvloop.new_event_loop() or asyncio.Task(). Also, event loops created before importing aiocontextvars are not patched either. So over all, you should import aiocontextvars at the beginning before creating event loops, and always use asyncio.* to operate loops/policies, and public asyncio API to create tasks.

Credits

Fantix King is the author and maintainer of this library. This library is open source software under BSD license.

aiocontextvars's People

Contributors

decaz avatar fantix avatar jjjacksn avatar pyup-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

aiocontextvars's Issues

Is uvloop compatible with this library?

"Because above hack is done by replacing asyncio.get_event_loop and loop.create_task, therefore tasks and loops created by custom/private API won't behave correctly as expected, e.g. uvloop.new_event_loop() or asyncio.Task(). Also, event loops created before importing aiocontextvars are not patched either. So over all, you should import aiocontextvars at the beginning before creating event loops, and always use asyncio.* to operate loops/policies, and public asyncio API to create tasks."

Does this mean I can't use uvloop at all, or just don't use the uvloop.new_event_loop()?

Loosen the version constraint of contextvars to avoid dependency conflicts

Hi, aiocontextvars locked the version constraint of contextvars as contextvars ==2.4, which leads a troubling scenario to its direct downstream project gumbelcodes which has dependency contextvars.

What makes the situation worse is that the downstream projects [gumbelcodes, flexget, pysoa] of aiocontextvars are also have dependency contextvars.

Could you please loosen the version constraint of contextvars?
Benefit of this is that users using both of aiocontextvars and contextvars can upgrade their third party libraries in a timely manner to reduce technical debts.

Solution

The dependency trees of your project and affected downstream projects are shown as follows.
Taking the version constraints of upstream and downstream projects into comprehensive consideration, you can

  1. Loosen click to be contextvars >=2.4.

@pyup-bot May I submit a PR to this issue?

Thanks for your attention.
Best,
Neolith

Minimal Documentation

It would be useful to at least have some minimal usage examples in the read me file. Of course, documentation would even be better.

when i execute python setup.py install,returned no results

  • aiocontextvars version:
  • Python version:3.6
  • Operating System: Linux

Description

Describe what you were trying to get done.
Tell us what happened, what went wrong, and what you expected to happen.

What I Did

python setup.py install

when i execute python setup.py install,returned no results
image

Make context variable namespaced

  • aiocontextvars version:0.2.2
  • Python version: 3.6
  • Operating System: macos

Description

Describe what you were trying to get done.
Tell us what happened, what went wrong, and what you expected to happen.

We are using encode/databases, which uses aiocontextvars.

The thread local state should be type checked. For now, the lookup will tend to return context if other library also sets a same named thread local variable.
So the name should be namespaced to provide less chances of error.

def _get_context():
    ctx = getattr(_state, 'context', None)
   # ctx may not be null, but of other type.
    if ctx is None:
        ctx = Context()
        _state.context = ctx
    return ctx

_state = threading.local()

What I Did

One of our library sets the context as dictionary.

  File "/Users/ramandeepsingh/.pyenv/versions/3.6.10/lib/python3.6/site-packages/contextvars/__init__.py", line 110, in set
    data = ctx._data
AttributeError: 'dict' object has no attribute '_data'

aiocontextvars ContextVar value not kept independently for the different async functions that set it

  • aiocontextvars version: 0.2.2
  • Python version: 3.5.0
  • Operating System: Ubuntu 18.04.3

Description

Mismatching results when running an asyncio script with Python 3.5.0 and aiocontextvars vs. Python 3.7.5rc1 and native contextvars package.

What I Did

I created a virtual environment with Python 3.5.0 and installed aiocontextvars via pip3. I activated the virtual environment and ran the script below. I ran the same script with Python 3.7.5rc1 and got different results. The results that I got from running with Python 3.7.5rc1 are the ones I expected. When I run the script with aiocontextvars it seems like the ContextVar value is not kept independently for the different async functions that set its value.

(3-5-0) ocelotl@harrison:~$ cat async_contextvar.py 
from asyncio import sleep, gather, get_event_loop
from contextvars import ContextVar

local_name = ContextVar("async_name")


async def waiting(async_name, first_sleep, second_sleep):

    await sleep(first_sleep)

    local_name.set(async_name)

    print("async_name:\t{}".format(async_name))
    print("local_name:\t{}".format(local_name.get()))
    print()

    await sleep(second_sleep)

    print("async_name:\t{}".format(async_name))
    print("local_name:\t{}".format(local_name.get()))
    print()


async def main():
    await gather(waiting("A", 0, 1), waiting("B", 0.1, 3), waiting("C", 2, 1))

loop = get_event_loop()
loop.run_until_complete(main())
loop.close()
(3-5-0) ocelotl@harrison:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.3 LTS
Release:        18.04
Codename:       bionic
(3-5-0) ocelotl@harrison:~$ python3 --version
Python 3.5.0
(3-5-0) ocelotl@harrison:~$ pip3 freeze
aiocontextvars==0.2.2
contextvars==2.4
immutables==0.11
You are using pip version 7.1.2, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(3-5-0) ocelotl@harrison:~$ python3 async_contextvar.py 
async_name:     A
local_name:     A

async_name:     B
local_name:     B

async_name:     A
local_name:     B

async_name:     C
local_name:     C

async_name:     C
local_name:     C

async_name:     B
local_name:     C

(3-5-0) ocelotl@harrison:~$ 
ocelotl@harrison:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.3 LTS
Release:        18.04
Codename:       bionic
ocelotl@harrison:~$ python3 --version
Python 3.7.5rc1
ocelotl@harrison:~$ python3 async_contextvar.py 
async_name:     A
local_name:     A

async_name:     B
local_name:     B

async_name:     A
local_name:     A

async_name:     C
local_name:     C

async_name:     C
local_name:     C

async_name:     B
local_name:     B

ocelotl@harrison:~$ 

Patching process throws error when setting event loop to None

  • aiocontextvars version: 0.2.0 (master @ b36af92)
  • Python version: 3.6.6
  • Operating System: macOS High Sierra (10.13.6)

Description

The patching process introduced by aiocontextvars is not correctly handling setting the event loop to None and raises an AttributeError. I discovered this while adding aiocontextvars to my project which also uses aiohttp and their test utilities. aiohttp's test utils set the event loop to None as part of test teardown (see here).

What I Did

Minimally reproducing code:

>>> import asyncio
>>> import aiocontextvars
>>> asyncio.set_event_loop(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jdot/code/vendor/aiocontextvars/aiocontextvars/__init__.py", line 67, in set_event_loop
    return _set_event_loop(_patch_loop(loop))
  File "/Users/jdot/code/vendor/aiocontextvars/aiocontextvars/__init__.py", line 57, in _patch_loop
    loop._orig_create_task = loop.create_task
AttributeError: 'NoneType' object has no attribute 'create_task'

I believe this can be fixed with a one line change.

Some early versions of Python 3.5 don't work with aiocontextvars

  • aiocontextvars version: 0.2.2
  • Python version: 3.5.2
  • Ubuntu Xenial LTS

Original bug report: matrix-org/synapse#5199

Description

On Ubuntu Xenial supported Python 3.5 is 3.5.2.

On Python 3.5.2 asyncio doesn't have asyncio._get_running_loop.

Example traceback (copied from itamarst/eliot#418 (comment)):

Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import eliot
>>> eliot.current_action()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/synapse/env3/lib/python3.5/site-packages/eliot/_action.py", line 53, in current_action
    return _ACTION_CONTEXT.get(None)
  File "/home/synapse/env3/lib/python3.5/site-packages/contextvars/__init__.py", line 94, in get
    ctx = _get_context()
  File "/home/synapse/env3/lib/python3.5/site-packages/aiocontextvars.py", line 19, in _get_context
    state = _get_state()
  File "/home/synapse/env3/lib/python3.5/site-packages/aiocontextvars.py", line 33, in _get_state
    loop = asyncio._get_running_loop()
AttributeError: module 'asyncio' has no attribute '_get_running_loop'

Solution

Haven't tried this yet, but:

if sys.version_info < (3, 5, 3):
    import asyncio
    from asyncio import events
    asyncio._get_running_loop = events._get_running_loop
    asyncio._set_running_loop = events._set_running_loop

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.