psycopg / psycogreen Goto Github PK

View Code? Open in Web Editor NEW

98.0 98.0 3.0 30 KB

Integration of psycopg2 with coroutine libraries

License: Other

Python 94.09% Shell 5.91%

psycogreen's Introduction

Psycopg 3 -- PostgreSQL database adapter for Python

Psycopg 3 is a modern implementation of a PostgreSQL adapter for Python.

Installation

Quick version:

pip install --upgrade pip               # upgrade pip to at least 20.3
pip install "psycopg[binary,pool]"      # install binary dependencies

For further information about installation please check the documentation.

Hacking

In order to work on the Psycopg source code you need to have the libpq PostgreSQL client library installed in the system. For instance, on Debian systems, you can obtain it by running:

sudo apt install libpq5

After which you can clone this repository:

git clone https://github.com/psycopg/psycopg.git
cd psycopg

Please note that the repository contains the source code of several Python packages: that's why you don't see a setup.py here. The packages may have different requirements:

The psycopg directory contains the pure python implementation of psycopg. The package has only a runtime dependency on the libpq, the PostgreSQL client library, which should be installed in your system.
The psycopg_c directory contains an optimization module written in C/Cython. In order to build it you will need a few development tools: please look at Local installation in the docs for the details.
The psycopg_pool directory contains the connection pools implementations. This is kept as a separate package to allow a different release cycle.

You can create a local virtualenv and install the packages in development mode, together with their development and testing requirements:

python -m venv .venv
source .venv/bin/activate
pip install -e "./psycopg[dev,test]"    # for the base Python package
pip install -e ./psycopg_pool           # for the connection pool
pip install ./psycopg_c                 # for the C speedup module

Please add --config-settings editable_mode=strict to the pip install -e above if you experience editable mode broken.

Now hack away! You can run the tests using:

psql -c 'create database psycopg_test'
export PSYCOPG_TEST_DSN="dbname=psycopg_test"
pytest

psycogreen's People

Contributors

Stargazers

Watchers

Forkers

isgasho uxio0 mfmarche

psycogreen's Issues

Question: which Python 3 version does psycogreen support?

Hi 👋

I'm about to upgrade an existing project running on Python 3.5 to a more recent version of Python (initially Python 3.6 but the plan is to migrate to the latest at later time).

Since psycogreen is one of the project's dependencies, I was wondering which versions of Python 3 are currently supported. I could not find any specific references in the setup.py nor in any other configuration file.

Thanks 🙌

psycogreen breaks connect_timeout option

python version 3.8.6

library versions:
gevent==20.9.0
greenlet==0.4.17
psycogreen==1.0.2
psycopg2==2.8.6

minimal example:

import gevent
import gevent.monkey

gevent.monkey.patch_all()  # noqa

import psycogreen.gevent

psycogreen.gevent.patch_psycopg()  # noqa

import psycopg2


def fetch(num, secs):
    print("connecting to db")
    conn = psycopg2.connect("dbname=postgres connect_timeout=1")
    print("connected")


jobs = [gevent.spawn(fetch, 3, 2)]
gevent.joinall(jobs)

This will print "connecting to db" and hang. If you comment out line psycogreen.gevent.patch_psycopg() # noqa code will fail with OperationalError in couple seconds.

question: integration with django / examples

as mentioned here:
https://serverfault.com/questions/635100/django-conn-max-age-persists-connections-but-doesnt-reuse-them-with-postgresq

I am a bit lost, and wonder if there are any examples, showing an django intergration with psycogreen. As I interpret, it could be a drop in solution, that could make for example pgbouncer no more necessary, when using geven/eventlet. The key question for me: Where to do the monkey patching? in the .wsgi file? Other places?

Sqlachemy = 2.0
Gevent= 22.10.0
psycogreen =1.0.2
psycopg2=2.9.5
grrenlet=2.0.1

File "/usr/local/lib/python3.9/site-packages/psycogreen/gevent.py", line 36, in gevent_wait_callback
    wait_read(conn.fileno(), timeout=timeout)
  File "src/gevent/_hub_primitives.py", line 353, in gevent._gevent_c_hub_primitives.wait_read
  File "src/gevent/_hub_primitives.py", line 367, in gevent._gevent_c_hub_primitives.wait_read
  File "src/gevent/_hub_primitives.py", line 350, in gevent._gevent_c_hub_primitives.wait_on_watcher
  File "src/gevent/_hub_primitives.py", line 304, in gevent._gevent_c_hub_primitives._primitive_wait
  File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_hub_primitives.py", line 57, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
gevent.exceptions.InvalidSwitchError: Invalid switch into <Greenlet at 0x7f8e26b51c20: _handle_and_close_when_done(functools.partial(<bound method GeventWorker.handl, <bound method StreamServer.do_close of <StreamServ, (<gevent._socket3.socket at 0x7f8e269fc3a0 object,)>: got None (expected <gevent._gevent_c_waiter.Waiter object at 0x7f8e26a379f0>; waiting on <Hub '' at 0x7f8e2b765810 epoll default pending=0 ref=15 fileno=6 resolver=<gevent.resolver.thread.Resolver at 0x7f8e2872d1f0 pool=<ThreadPool at 0x7f8e287aa190 tasks=0 size=0 maxsize=10 hub=<Hub at 0x7f8e2b765810 thread_ident=0x7f8e2d59bb48>>> threadpool=<ThreadPool at 0x7f8e287aa190 tasks=0 size=0 maxsize=10 hub=<Hub at 0x7f8e2b765810 thread_ident=0x7f8e2d59bb48>> thread_ident=0x7f8e2d59bb48> with <io at 0x7f8e26b4de40 native=0x7f8e26b4de80 fd=25 events=READ active callback=<bound method Waiter.switch of <gevent._gevent_c_waiter.Waiter object at 0x7f8e26a379f0>> args=(<gevent._gevent_c_waiter.Waiter object at 0x7f8e26a379f0>,)>)

Add support for psycopg 3

Psycopg 3 requires some "psycogreen" support starting from 3.1.5 when the c or binary extensions are used.