Giter VIP home page Giter VIP logo

greentor's Introduction

greentor

greentor is a fork of gTornado

greentor通过给pymysql打补丁,使pymysql在Tornado中的运行过程变为异步IO,相比于其它支持Tornado的mysql驱动,greentor有以下不同

  1. 同步pymysql的写法
  2. 理论上可以支持各种ORM的调用异步

感谢@alex8224和他的gTornado

感谢@snower,参考他的TorMySQL优化了IOStream的读写性能

安装

pip install git+https://github.com/zhu327/greentor.git

使用

# coding: utf-8

from greentor import green
green.enable_debug()
from greentor import mysql
mysql.patch_pymysql()
  1. green.enable_debug() 非必须,开启greenlet调试模式,可以打印greenlet switch过程
  2. mysql.patch_pymysql() 给pymysql打异步补丁,异步的pymysql依赖于Tornado,在Tornado的IOLoop start后才能正常使用

RequestHandler中使用

涉及到pymysql的调用都需要运行在greenlet中,提供了3种方式实现同步代码转异步

from greentor import green
from greentor import mysql
mysql.patch_pymysql()
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    @green.green
    def get(self):
        connect = MySQLdb.connect(user='root',
                                  passwd='',
                                  db='test',
                                  host='localhost',
                                  port=3306)
        cursor = connect.cursor()
        cursor.execute('SELECT * FROM app_blog LIMIT 1')
        result = cursor.fetchone()
        cursor.close()
        connect.close()
        self.finish(result[2])

通过green.green装饰器使整个get方法都运行在greenlet中,这样是最方便的使用pymysql的方式

from greentor import green
from greentor import mysql
mysql.patch_pymysql()
import tornado.web
import tornado.gen

@green.green
def test_mysql():
    connect = MySQLdb.connect(user='root',
                              passwd='',
                              db='test',
                              host='localhost',
                              port=3306)
    cursor = connect.cursor()
    cursor.execute('SELECT * FROM app_blog LIMIT 1')
    result = cursor.fetchone()
    cursor.close()
    connect.close()
    return result


class MainHandler(tornado.web.RequestHandler):
    @tornado.gen.coroutine
    def get(self):
        result = yield test_mysql()
        self.finish(result[2])

通过green.green装饰器包装的函数会返回Future对象,可以在Tornado的协程中使用

from greentor import green
from greentor import mysql
mysql.patch_pymysql()
import tornado.web
import tornado.gen

def test_mysql():
    connect = MySQLdb.connect(user='root',
                              passwd='',
                              db='test',
                              host='localhost',
                              port=3306)
    cursor = connect.cursor()
    cursor.execute('SELECT * FROM app_blog LIMIT 1')
    result = cursor.fetchone()
    cursor.close()
    connect.close()
    return result


class MainHandler(tornado.web.RequestHandler):
    @tornado.gen.coroutine
    def get(self):
        result = yield green.spawn(test_mysql)
        self.finish(result[2])

green.spawn(callable_obj, *arg, **kwargs)的调用与green.green一致

实例

在tests目录下有一个使用纯pymysql调用的实例

demo目录下有一个完整的 Tornado + Django ORM 的环境,具体可以查看demo目录下的README

greentor's People

Contributors

alex8224 avatar zhu327 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

greentor's Issues

当使用tornado并发访问时,会出现AssertionError: Already reading错误

配合peewee使用时,会出现错误
"SELECT * FROM sysconf_systemconfig WHERE system='maps' AND module='interface'").fetchone() File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/playhouse/shortcuts.py", line 210, in execute_sql sql, params, require_commit) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/peewee.py", line 3669, in execute_sql cursor.execute(sql, params or ()) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/pymysql/cursors.py", line 167, in execute result = self._query(query) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/pymysql/cursors.py", line 323, in _query conn.query(q) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/pymysql/connections.py", line 836, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/pymysql/connections.py", line 1020, in _read_query_result result.read() File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/pymysql/connections.py", line 1303, in read first_packet = self.connection._read_packet() File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/pymysql/connections.py", line 962, in _read_packet packet_header = self._read_bytes(4) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/greentor/mysql.py", line 73, in _read_bytes data = self._rfile.read(num_bytes) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/greentor/green.py", line 125, in _sync_call return main.switch() File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/tornado/gen.py", line 282, in wrapper yielded = next(result) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/greentor/green.py", line 247, in read buff = yield self._iostream.read_bytes(nbytes, partial=partial) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/tornado/iostream.py", line 311, in read_bytes future = self._set_read_callback(callback) File "/home/qingyun.meng/pydev/lib/python2.7/site-packages/tornado/iostream.py", line 662, in _set_read_callback assert self._read_future is None, "Already reading"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.