aio-libs / aiohttp-devtools Goto Github PK

View Code? Open in Web Editor NEW

249.0 249.0 41.0 710 KB

dev tools for aiohttp

License: MIT License

Makefile 0.53% Python 99.47%

aiohttp asyncio cookiecutter developer-tools development devtools reload

aiohttp-devtools's People

Contributors

Stargazers

Watchers

aiohttp-devtools's Issues

'check_port_open' was never awaited

OS X 10.12.6, Python 3.6.6, aiohttp 3.3, aiohttp-devtools 0.10.1

runserver raises an exception on ctrl-C:

pikeas@27d8f65ec422:~$ adev runserver foo/__main__.py
======== Running on http://0.0.0.0:8080 ========
(Press CTRL+C to quit)
^C
Traceback (most recent call last):
  File "/app/.venv/bin/adev", line 11, in <module>
    sys.exit(cli())
  File "/app/.venv/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/app/.venv/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/app/.venv/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/.venv/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/app/.venv/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/app/.venv/lib/python3.6/site-packages/aiohttp_devtools/cli.py", line 86, in runserver
    run_app(*_runserver(**active_config))
  File "/app/.venv/lib/python3.6/site-packages/aiohttp_devtools/runserver/main.py", line 47, in runserver
    loop.run_until_complete(check_port_open(config.main_port, loop))
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 444, in run_until_complete
    self._check_closed()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 358, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
sys:1: RuntimeWarning: coroutine 'check_port_open' was never awaited

Not able to choose ip address

I have 2 local PC.

I would like to see live reload working on a second PC, but I have errors.

On a second PC injected livereload script path looks like "http://localhost:8001/livereload.js"></script>"

It doesn't make any sense because the second PC doesn't have aiohttp server running, only first PC.

I think it would be great to have such feature like

adev runserver . -ip 192.168.70.1

see HISTORY.rst is not very informative.
Could you please add a link to release notes at least?
Ideally change notes from the release should be copied to text area but the link is required at least.

cannot set port

I get following error:

$ AIO_PORT=8080 adev runserver --app-factory="main"
  File ".../lib/python3.6/site-packages/aiohttp_devtools/runserver/config.py", line 63, in __init__
    self.aux_port = aux_port or (main_port) + 1
TypeError: must be str, not int

or $ adev runserver --port=8080

app.loop is not defined during pre-check

Hello,

I noticed a difference between running my application directly and using aiohttp-devtools.

The web.run_app of aiohttp fires on_startup after the call to make_handler which defines the loop on the app instance. In aiohttp-devtools, the pre-check fires on_startup and on_cleanup directly on the Application instance it finds in a module without setting the loop.

So if on a on_startup handler you create a task (using the loop) it will not work because loop is None at that point. The official example of aiohttp on background task is a good example:

http://pythonhosted.org/aiohttp/web.html#background-tasks

async def listen_to_redis(app):
    try:
        sub = await aioredis.create_redis(('localhost', 6379), loop=app.loop)
        ch, *_ = await sub.subscribe('news')
        async for msg in ch.iter(encoding='utf-8'):
            # Forward message to all connected websockets:
            for ws in app['websockets']:
                ws.send_str('{}: {}'.format(ch.name, msg))
    except asyncio.CancelledError:
        pass
    finally:
        await sub.unsubscribe(ch.name)
        await sub.quit()


async def start_background_tasks(app):
    app['redis_listener'] = app.loop.create_task(listen_to_redis(app)) # <--------- HERE


async def cleanup_background_tasks(app):
    app['redis_listener'].cancel()
    await app['redis_listener']


app = web.Application()
app.on_startup.append(start_background_tasks)
app.on_cleanup.append(cleanup_background_tasks)
web.run_app(app)

The only solution I have at this point is deactivating the pre-checks...

Webfonts wasn't rendered in web page that use css and font file serving through runserver command

aiohttp-devtools version:
0.6.4
aiohttp version:
2.3.7
python version:
Python 3.6.3
Platform:
macOS

Issue Summary

Text in web page was rendered using system fonts instead of webfonts that I have specified in css file.

I have checked that all files can be served correctly (got http status 200 or 304).

If i use webfonts library that was served from CDN, text was rendered with webfonts perfectly. This issue was happened only when files was served through adev runserver command.

While digging into this issue, I found that http response headers is different between webfonts file that was served from devtools and from CDN. The main difference is the lacking of Cross Origin Resource Sharing headers in files from devtools. Although I'm not 100% sure that this is the real culprit. I still haven't found a way to inject these headers to response to test my assumption.

allow function execution on reload

Eg. to let me reload react yarn start when the server reloads by touching js/public/index.html.

Autoreload doesn't work

I've setup a project with:

aiohttp devtools: 0.6.4
aiohttp version: 2.3.6
python version: 3.6
Platform: Ubuntu 17.10 Artful

The autoreload doesn't work, it doesn't detect the changes of the backend side. I use the following adev command:

$ DEBUG=DEBUG adev runserver --no-debug-toolbar datalibrary/api.py
[12:19:17] pre-check enabled, checking app factory
2017-12-19 12:19:17,078 [6652] INFO     datalibrary.api: boot datalibrary-api server
2017-12-19 12:19:17,079 [6652] DEBUG    datalibrary.api: setup asyncpg, using postgres://datalibrary:datalibrary@localhost:5432/datalibrary
[12:19:17] Starting aux server at http://localhost:8001 ◆
2017-12-19 12:19:17,093 [6652] INFO     adev.server.dft: Starting aux server at http://localhost:8001 ◆
[12:19:17] Starting dev server at http://localhost:8000 ●
2017-12-19 12:19:17,094 [6652] INFO     adev.server.dft: Starting dev server at http://localhost:8000 ●
2017-12-19 12:19:17,102 [6652] DEBUG    watchgod.main: time=1ms files=40 changes=0
2017-12-19 12:19:17,502 [6652] DEBUG    watchgod.main: time=1ms files=40 changes=0
2017-12-19 12:19:17,768 [6678] INFO     datalibrary.api: boot datalibrary-api server
2017-12-19 12:19:17,770 [6678] DEBUG    datalibrary.api: setup asyncpg, using postgres://datalibrary:datalibrary@localhost:5432/datalibrary
2017-12-19 12:19:17,904 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
2017-12-19 12:19:18,305 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
2017-12-19 12:19:18,706 [6652] DEBUG    watchgod.main: time=3ms files=40 changes=0
2017-12-19 12:19:19,107 [6652] DEBUG    watchgod.main: time=3ms files=40 changes=0
2017-12-19 12:19:19,507 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
2017-12-19 12:19:19,909 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
2017-12-19 12:19:20,309 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
2017-12-19 12:19:20,711 [6652] DEBUG    watchgod.main: time=3ms files=40 changes=0
2017-12-19 12:19:21,111 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
2017-12-19 12:19:21,512 [6652] DEBUG    watchgod.main: time=3ms files=40 changes=0
2017-12-19 12:19:21,913 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
2017-12-19 12:19:22,314 [6652] DEBUG    watchgod.main: time=3ms files=40 changes=0
2017-12-19 12:19:22,715 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0

The watchgod is kicking and then when a file changes it stops...

Here is the traceback when I hit Ctrl+C:

2017-12-19 12:19:22,715 [6652] DEBUG    watchgod.main: time=2ms files=40 changes=0
^C[12:20:23] shutting down server...
2017-12-19 12:20:23,377 [6652] INFO     adev.server.dft: shutting down server...
2017-12-19 12:20:23,464 [6652] CRITICAL root: Traceback (most recent call last):
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/bin/adev", line 11, in <module>
    sys.exit(cli())
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp_devtools/cli.py", line 88, in runserver
    run_app(*_runserver(**active_config))
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp_devtools/runserver/main.py", line 26, in run_app
    loop.run_until_complete(app.shutdown())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 467, in run_until_complete
    return future.result()
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp/web.py", line 272, in shutdown
    yield from self.on_shutdown.send(self)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp/signals.py", line 51, in send
    yield from self._send(*args, **kwargs)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp/signals.py", line 17, in _send
    yield from res
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py", line 108, in close
    await super().close()
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py", line 32, in close
    self._task.result()
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py", line 48, in _run
    async for changes in self._awatch:
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/watchgod/main.py", line 89, in __anext__
    changes = await self._loop.run_in_executor(None, self._w.check)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/watchgod/watcher.py", line 43, in check
    self._walk(str(self.root_path), changes, new_files)
  File "/home/amirouche/.local/share/virtualenvs/west2-bifHViav/lib/python3.6/site-packages/watchgod/watcher.py", line 32, in _walk
    mtime = entry.stat().st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/home/amirouche/src/namr/datalibrary/west2/datalibrary/.#api.py'

2017-12-19 12:20:23,497 [6652] ERROR    asyncio: Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7ff6416b6a58>
(west2-bifHViav)

devtools can not work with session[aioredis]

aiohttp-devtools version:
0.6.4
aiohttp version:
2.3.4
python version:
3.6.3
Platform:
OSX

Issue Summary

Starting with adev runserver simple.py will raise RuntimeError: This event loop is already running but it works for python simple.py

Code Example

#simple.py
from aiohttp import web
import asyncio
import aioredis
from aiohttp_session import setup, get_session
from aiohttp_session.redis_storage import RedisStorage


async def create_redis_pool():
    return await aioredis.create_pool(('127.0.0.1', 6379))


def setup_sessions(app, pool):
    storage = RedisStorage(pool)
    setup(app, storage)


async def index(request):
    sess = await get_session(request)
    last_visit = sess['last_visit'] if 'last_visit' in sess else "never"
    sess['last_visit'] = "yes"
    return web.Response(text=last_visit)


def create_app():
    loop = asyncio.get_event_loop()
    pool = loop.run_until_complete(create_redis_pool())
    app = web.Application()
    setup_sessions(app, pool)
    app.router.add_get("/", index)
    return app


if __name__ == '__main__':
    web.run_app(create_app(), port=8000)

Steps to reproduce

adev runserver simple.py.
check out error message.

sqlite support

hello, i'm using the latest version of python, aiohttp and aiohttp-devtools (as of now). I would like to work with sqlite .. what can i do to the code to make that happen?

db settings missing without message-board

https://github.com/aio-libs/aiohttp-devtools/blob/master/aiohttp_devtools/start/template/app/settings.py#L24

StringPayload append error

  File ".../aiohttp_devtools/runserver/serve.py", line 50, in on_prepare
    response.body += lr_snippet.encode()
TypeError: unsupported operand type(s) for +=: 'StringPayload' and 'bytes'

Can't import correctly when `import module.submodule`

aiohttp-devtools version: 0.9
aiohttp version: 3.2.1
python version: 3.6.2
Platform: windows 10 and ubuntu 16.04

Issue Summary

There are 2 bugs.

First:

In windows, deep_reload try to reload standard package

if not m_file or 'lib/python' in m_file or m_file in already_loaded: try to filter lib/python in m_file.
But in Windows, m_file could be .virtualenvs\{venv_name}\lib\site-packages\aiohttp and ~\AppData\Local\Programs\Python\Python36\lib\typing.py

this did't happend on ubuntu,but in windows, when trying import standard lib ssl, the second bug happens

Second

I'm useing motor as my mongodb driver,and I init a client right in top level of my app.py

# app.py
import motor.motor_asyncio as aiomotor

mongo_uri = "mongodb://127.0.0.1:27017"
conn = aiomotor.AsyncIOMotorClient(
        mongo_uri,
        maxPoolSize=20,
        io_loop=loop)
db = conn['db_name']

But when i try to start my dev, it stuchs in yield from find_modules(import_module(module_name))

I found it try to import motor_asyncio in function import_module, instead of motor.motor_asyncio it should raise a ModuleNotFoundError in import_module, but in fact it didn't and freeze. And i have no idea how to fix it.

Steps to reproduce

I reproduce the second bug on ubuntu with python3.6

sudo python3.6 -m pip install aiohttp_devtools aiohttp motor
mkdir src
cd src

src/app.py

import asyncio
import pathlib
from os import path

import aiohttp
import motor.motor_asyncio
from aiohttp import web

base_dir = pathlib.Path(path.dirname(__file__))

loop = asyncio.get_event_loop()
mongo_uri = "mongodb://127.0.0.1:27017"
conn = motor.motor_asyncio.AsyncIOMotorClient(
        mongo_uri,
        maxPoolSize=20,
        io_loop=loop)
db = conn['gbf']


async def index(request: aiohttp.web.BaseRequest):
    return web.Response(body='hello world')


def create_app(loop=None):
    app = web.Application(loop=loop)
    app.add_routes([
        web.get('/', index),
    ])
    print('create app', flush=True)
    return app


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    web.run_app(create_app(loop=loop), port=6001)

adev runserver app.py -v --app-factory create_app -p 6001

curl http://localhost:6001/
# curl: (7) Failed to connect to localhost port 6001: Connection refused

I try from motor.motor_asyncio import AsyncIOMotorClient, same error'

Sorry for my poor English.

RuntimeWarning: coroutine 'PyCodeEventHandler.src_reload_when_live' was never awaited

I've got some exception.
Can you help please understand why its happen.

[04:08:32] Restarting dev server at http://localhost:8000 ●
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/watchdog/observers/api.py", line 199, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/watchdog/observers/api.py", line 368, in dispatch_events
    handler.dispatch(event)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py", line 67, in dispatch
    self.on_event(event)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py", line 88, in on_event
    self._loop.create_task(self.src_reload_when_live())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 284, in create_task
    task = tasks.Task(coro, loop=self)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 576, in call_soon
    self._check_thread()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 615, in _check_thread
    "Non-thread-safe operation invoked on an event loop other "
RuntimeError: Non-thread-safe operation invoked on an event loop other than the current one

Task was destroyed but it is pending!
source_traceback: Object created at (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/watchdog/observers/api.py", line 199, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/watchdog/observers/api.py", line 368, in dispatch_events
    handler.dispatch(event)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py", line 67, in dispatch
    self.on_event(event)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py", line 88, in on_event
    self._loop.create_task(self.src_reload_when_live())
  File "/usr/lib/python3.6/asyncio/base_events.py", line 284, in create_task
    task = tasks.Task(coro, loop=self)
task: <Task pending coro=<PyCodeEventHandler.src_reload_when_live() running at /home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py:90> created at /usr/lib/python3.6/asyncio/base_events.py:284>
/usr/lib/python3.6/threading.py:949: RuntimeWarning: coroutine 'PyCodeEventHandler.src_reload_when_live' was never awaited
  del exc_type, exc_value, exc_tb

optional multiprocessing for reload

from v0.8 live reloading server stop using multiprocessing, in my project new approach with module reloading not working at all, and i don't know how to reorganize my project for this approach, so will be better to support two methods of reloading, multiprocessing or modules reload and select what method to use by flags.

Expose command line interface in init.py

python3 -m aiohttp_devtools <args> should work in the same way as adev <args>.

cannot inject middlewares

In an attempt to give aiohttp-devtools a try, I ran this command: aiohttp-runserver main.py run
(run is the app_factory)

opening the homepage, I get the following error:

[21:03:09] Starting dev server at http://localhost:8000 ●
[21:03:09] Starting aux server at http://localhost:8001 ◆
Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/aiohttp/server.py", line 265, in start
    yield from self.handle_request(message, payload)
  File "/usr/local/lib/python3.5/dist-packages/aiohttp/web.py", line 96, in handle_request
    resp = yield from handler(request)
  File "/usr/local/lib/python3.5/dist-packages/aiohttp_session/__init__.py", line 129, in middleware
    response = yield from handler(request)
  File "/usr/mms/middlewares.py", line 35, in middleware
    request.db = app.db
AttributeError: 'Application' object has no attribute 'db'

the app_factory is as follows:

def run(loop):
	app = aiohttp.web.Application(loop=loop)
	app.on_startup.append(middlewares.init_pg)
	app.on_cleanup.append(middlewares.close_pg)
	return app

and eventually init_pg is:

async def init_pg(app):
	engine = await aiopg.sa.create_engine(
		database=config.DB_NAME,
		user=config.DB_USER,
		password=config.DB_PASS,
		host=config.DB_HOST,
		port=config.DB_PORT,
		loop=app.loop)
	app.db = engine

the problem is that runserver does not call this function (init_pg)

OS: Ubuntu 16.04
Python: 3.5.2

any help would be greatly appreciated

New release

Currently aiohttp-devtools doesn't work with aiohttp 2.3.0 so we need to fix this ASAP and do a new release.

The only definite problem I know about is finish_connections vs shutdown which is fixed in #144.

It would also be good to merge #145, that's quite simple but it would be good to be sure it fixed the problem and also that it doesn't do anything terrible on OSX / windows - I have no idea how terminal stdin file descriptors work there.

@asvetlov do you want to review either of those PRs or just code before release?

ImportError: cannot import name 'unquote'

aiohttp-devtools version: 0.6.4
aiohttp version: 2.3.8
python version: 3.5.2
Platform: Ubuntu 16.04.3

Issue Summary

Not fixed, as in Fix #2662: Properly unquote path parts

Steps to reproduce

$ adev runserver

empty response body error

Happens on something to do with websockets, need to make sure the livereload stuff doesn't occur on websockets.

  File "/home/samuel/code/chataio/env/lib/python3.6/site-packages/aiohttp/signals.py", line 48, in send
    yield from self._send(*args, **kwargs)
  File "/home/samuel/code/chataio/env/lib/python3.6/site-packages/aiohttp/signals.py", line 16, in _send
    yield from res
  File "/home/samuel/code/chataio/env/lib/python3.6/site-packages/aiohttp_devtools/runserver/serve.py", line 41, in on_prepare
    response.body += livereload_snippet
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'bytes'

Removed method load_settings() in prepare_database()

make reset-database returned:

from .main import load_settings, pg_dsn
ImportError: cannot import name 'load_settings'

OSX 10.12.4 local trouble with click

Hello. I've got a problem at the first-start of aiohttp-devtools:

adev runserver api/app.py 
Traceback (most recent call last):
  File "/Users/netstuff/Sites/egora/venv/bin/adev", line 11, in <module>
    sys.exit(cli())
  File "/Users/netstuff/Sites/egora/venv/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/netstuff/Sites/egora/venv/lib/python3.6/site-packages/click/core.py", line 676, in main
    _verify_python3_env()
  File "/Users/netstuff/Sites/egora/venv/lib/python3.6/site-packages/click/_unicodefun.py", line 118, in _verify_python3_env
    'for mitigation steps.' + extra)
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment.  Consult http://click.pocoo.org/python3/for mitigation steps.

This system lists a couple of UTF-8 supporting locales that
you can pick from.  The following suitable locales where
discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8

Click discovered that you exported a UTF-8 locale
but the locale system could not pick up from it because
it does not exist.  The exported locale is "ru_RU.utf8" but it
is not supported

python --version
Python 3.6.0

locale
LANG="ru_RU.utf8"
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

What can I do?

Can't set a non-default event loop when running `adev runserver`

This is the command that I'm running

adev runserver -v butler/app.py --app-factory create_app --pre-check

..And this is the traceback I get

Process Process-1:
Traceback (most recent call last):
  File "/home/username/.pyenv/versions/3.6.3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/username/.pyenv/versions/3.6.3/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/username/work/project/project-venv/lib/python3.6/site-packages/aiohttp_devtools/runserver/serve.py", line 127, in serve_main_app
    loop=loop,
  File "/home/username/work/project/project-venv/lib/python3.6/site-packages/aiohttp/web.py", line 250, in make_handler
    self._set_loop(loop)
  File "/home/username/work/project/project-venv/lib/python3.6/site-packages/aiohttp/web.py", line 128, in _set_loop
    "web.Application instance initialized with different loop")
RuntimeError: web.Application instance initialized with different loop

The problem is that I am instantiating my app like this: app = aiohttp.web.Application(loop=uvloop).
I have no way of making this change the default ioloop policy before a default ioloop is being instantiated.

This is the function that's killing my buzz:

def serve_main_app(config: Config, tty_path: Optional[str], loop: asyncio.AbstractEventLoop=None):
    with set_tty(tty_path):
        setup_logging(config.verbose)

        # THIS IS THE BUZZ-KILLING LINE
        # If only this line was AFTER the current line - but that is circular and can't be done currently.
        # However, if my factory function was loaded before this line was executed, then I'd be very happy :D
        # That would give me the chance to do `asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())`
        loop = loop or asyncio.get_event_loop()

        app = config.load_app(loop)

        modify_main_app(app, config)

        loop.run_until_complete(check_port_open(config.main_port, loop))
        handler = app.make_handler(
            logger=dft_logger,
            access_log_format='%r %s %b',
            loop=loop,
        )

Do you need help with this? I'd gladly import the factory function before retrieving the loop. That could probably make a few people happy.

Thanks!

runserver is confusing

This isn't particularly a bug but I found it confusing to use runserver and a bit foreign for typical aiohttp apps.

First of all, it seems standard to expose aiohttp.web.Application instance as "app" instead of creating a factory (a practice came from flask, I suppose). Application has call method for compatibility but it doesn't accept any other arguments so it doesn't work with create_main_app() which tries to pass loop to the factory.

Ok, let's assume I made a factory that accepts loop parameter, I'm trying to run it, aaaand stuff breaks because you explicitly create a loop https://github.com/samuelcolvin/aiohttp-devtools/blob/c06eb95d052200b2f3a2d93ca55470048e9d7a20/aiohttp_devtools/runserver/serve.py#L72

Why not use default loop (or at least set it)? some functions like asyncio.ensure_task relies on it.

Thanks!

aiohttp-tools: do we need it?

There's quite a bit of code required to deploy an aiohttp app which will be duplicated between almost all apps.

This code probably doesn't want to end up in aiohttp itself, but also can't be included in aiohttp-devtools as it will be required in production and testing where aiohttp-devtools shouldn't be installed.

There are therefore two options:

include this code when generating a new project with with start
create a third library perhaps aiohttp-tools which has minimal requirements and takes care of this logic

Examples of such code:
1 config parsing eg. clayman74/wallet config, see #2 for example of needs
2 creating, resetting, migration dbs
3 executing management commands - this is probably an easier way of covering 2.

The more I think about this the more I think it's necessary and could causing more confusion than it solves.

It will however require a bit of code to be created in all started projects while could go out of date and wouldn't be easily updated across the population.

Server reload, custom port and host

aiohttp-devtools version: 0.10.1
aiohttp version: 3.4.2
python version: 3.5.4rc1
Platform: MacOS

Issue Summary

Cannot run server on the specified host and port. Also server does not reload self after changes.

Steps to reproduce

adev runserver --port=8000 or adev runserver --port 8000 (the same with host).

Does runserver work with uvloop?

I tried different ways but always got attached to a different loop exception

Hot reloading not working on FreeBSD

**aiohttp-devtools version: 0.8
**aiohttp version: 3.0.7
**python version: 3.6.4
**Platform: FreeBSD 11.1

Issue Summary

After having followed the instructions several times using both virtualenv and pipenv I cannot get hot reloading to work. I had it working in a previous version about 6 months ago so am experienced in working with aiohttp as well as aiohttp-devtools. The server log states that it reloads the main.py file however any changes are only visible after stopping the server completely and then starting it again.

Steps to reproduce

virtualenv -p python3 venv36
cd venv36
source bin/activate
pip install aiohttp-devtools
adev start test1
cd test1
source ./activate-settings.sh
adev runserver test1 --host 192.168.0.1

How to use pdb inside an app running with adev?

I am using adev in my aiohttp application. I've setup a pdb inside the app, when I get the following error:

Traceback (most recent call last):
  File "/home/amirouche/.virtualenvs/socialite/lib/python3.5/site-packages/aiohttp/web_protocol.py", line 417, in start
    resp = yield from self._request_handler(request)
  File "/home/amirouche/.virtualenvs/socialite/lib/python3.5/site-packages/aiohttp/web.py", line 289, in _handle
    resp = yield from handler(request)
  File "/home/amirouche/.virtualenvs/socialite/lib/python3.5/site-packages/aiohttp_devtools/runserver/serve.py", line 62, in _handler
    return await handler(request)
  File "/home/amirouche/src/python/socialite/socialite/api.py", line 34, in status
    try:
  File "/home/amirouche/src/python/socialite/socialite/api.py", line 34, in status
    try:
  File "/usr/lib/python3.5/bdb.py", line 48, in trace_dispatch
    return self.dispatch_line(frame)
  File "/usr/lib/python3.5/bdb.py", line 67, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit

Is it possible to use both adev and pdb (or better aiomonitor)

Update all `secret: ` values in `.travis.yml`

Do like this:

REPO_NAME={{ YOUR_REPO_NAME }}
travis encrypt -r "aio-libs/${REPO_NAME}" --api-endpoint 'https://api.travis-ci.com/'

Ref: https://github.com/orgs/aio-libs/teams/admins/discussions/9

Extract autoreload facility into separate library

During EuroPython people asked me about a tiny and simple library with autoreload.
aiohttp-devtools supports it already but also provides a lot of other functionality.

The idea is: move reload-related code to separate project and reuse this project by aiohttp-devtools itself.

@samuelcolvin what do you think about?

Documentation how to add livereload by api

Please add example how to add livereload from code, without cli usage.
I have runserver.py script that up and running aiohttp web server

"""
This script runs the natrix application using a development server.
"""
from os import environ
from myapp import run_app

if __name__ == '__main__':
    HOST = environ.get('SERVER_HOST', '0.0.0.0')
    try:
        PORT = int(environ.get('SERVER_PORT', '8888'))
    except ValueError:
        PORT = 5555
    run_app(HOST, PORT) # create web.Application, setup middlewares and routes and call web.run_app

How can I add livereload here?
Will be good to add some utility function which will do all the work for attach live reload
as example:

from aiohttp_devtools import patch_web_app
patch_web_app(web_app)

Application logs get swallowed by aiohttp-devtools

aiohttp-devtools version: 0.10.1
aiohttp version: 3.3.2
python version: 3.7
Platform: Mac OS

Issue Summary

My application logs get swallowed by the dev tools.

Steps to reproduce

I am running the server inside my own manage.py file. So I am not using the adev tool directly.

This is how I am running the server:

from aiohttp_devtools import runserver as devtools

        devtools.run_app(
            *devtools.runserver(
                app_path=os.path.join(BASE_DIR, 'app'),
                app_factory_name='create_app',
                host=host or devtools.INFER_HOST,
                main_port=int(port),
                livereload=True,
                verbose=True,
            ))

This is how I am setting up logging:

def setup_logging(verbose: bool = False):
    """
    setup logging config
    """
    log_level = 'DEBUG' if verbose else 'INFO'
    config = {
        'version': 1,
        'disable_existing_loggers': False,
        'formatters': {
            'default': {
                'format': '%(levelname)s %(name)s %(message)s',
            },
        },
        'handlers': {
            'console': {
                'class': 'logging.StreamHandler',
                'level': log_level,
                'formatter': 'default',
                # 'stream': 'ext://sys.stdout',
            },
        },
        'loggers': {
            '': {
                'level': log_level,
                'handlers': ['console'],
                'propagate': True,
            },
        },
    }
    logging.config.dictConfig(config)

And this is how I am logging logs:

logger = getLogger()

print('**************************************************')
logger.info('Hello')

The print statement work but logger does not display anything on the screen. However, when I use the aiohttp runserver directly, everything works as expected.

python 3.4 - surely not

Does this package need to support python 3.4?

I would suggest not. 3.5 is in wide usage and is available on all major platforms I've seen.

Also keeping support for 3.4 would make the code (both the library and the generated start project) significantly uglier.

@asvetlov ?

reset-database don't create tables

I try example app based on template.

make reset-database don't create tables.

>>> Base.metadata
MetaData(bind=None)
>>> Base.metadata.tables
immutabledict({})

Runserver loop parameter

Hi.

If I use uvloop for my project as

asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

and run runserver command and press ctrl+c, then I get an error

RuntimeError: Task <Task pending coro=<AppTask._run() running at /../lib/python3.6/site-packages/aiohttp_devtools/runserver/watch.py:48>> got Future <Future pending cb=[_chain_future.<locals>._call_check_cancel()]> attached to a different loop

Also, the server does not restart if I change the code.

Maybe, runserver command should take --loop parameter (for example --loop=uvloop) ?
Or already there is a solution to this problem?

Thanks.

disable the aux server when neither reload nor static server are required

Will be good to provide an option to disable static serving for people who use aiohttp only as a API server and not need a static file serving from devtools livereload server.
In one of my project I use aiohttp as rest api server and when I use livereload server from aiohttp devtools and when server is restarting during to changes, bootstrap of my app occurs twice (one for server and one for static files server).
I will try to implement this, and make a pr.

Debug toolbar don't work by default.

What we need to solve in debugtoolbar for removing this todo?

aiohttp-devtools/aiohttp_devtools/runserver/config.py

Line 38 in ec1da74

debug_toolbar: bool=False, # TODO set True once debug toolbar is fixed

I wanna solve that but i don't know what 😅

Problem with library yarl v1.0

aiohttp-devtools version:
0.6.4
aiohttp version:
2.3.8
yarl library version:
1.0
python version:
3.6.1
Platform:
Linux Ubuntu 16.04

Issue Summary

aiohttp-devtools don't work with yarl library version 1.0.
Reason -

aiohttp-devtools/aiohttp_devtools/runserver/serve.py

Line 15 in 33ef2dc

from yarl import unquote

(at least). I don't check all files, maybe there are any other places with similar error.

can't pickle Loop objects with uvloop

TypeError: can't pickle Loop objects, see #149 for more details

aux-port default

aux-port should default to port + 1 not 8001.

modify_main_app dont setup aiohttp_debugtoolbar for nested apps

Reproduce:

nested_app = Application()
nested_app['name'] = 'api'

app = Application()
app['name'] = 'main_app'
app.add_subapp('/api', nested_app)

adev runserver .

500 Internal Server Error


Traceback:

Traceback (most recent call last):
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp/web_server.py", line 62, in handle_request
    resp = yield from self._handler(request)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp/web.py", line 270, in _handle
    resp = yield from handler(request)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_debugtoolbar/middlewares.py", line 59, in toolbar_middleware
    toolbar = DebugToolbar(request, panel_classes, global_panel_classes)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_debugtoolbar/toolbar.py", line 24, in __init__
    panel_inst = panel_class(request)
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp_debugtoolbar/panels/traceback.py", line 21, in __init__
    self.exc_history = request.app[APP_KEY]['exc_history']
  File "/home/anton/virtualenvs/extremedialer/lib/python3.6/site-packages/aiohttp/web.py", line 71, in __getitem__
    return self._state[key]
KeyError: 'aiohttp_debugtoolbar'

If manual setup for nested app it works OK

nested_app = Application()
nested_app['name'] = 'api'
aiohttp_debugtoolbar.setup(nested_app)

Is it possible add cheching nested apps and auto setup debug_toolbar for it?

Logging option

aiohttp-devtools version:
Newest
aiohttp version:
3.3.2
python version:
3.6
Platform:
MacOS

Issue Summary

Question about logging, not issue.

Detail

First, thanks for good tool. It has been a help.

This is not in document, but found while looking on cli.py.
In runserver method, -v seems setting up the log, by true/false.

@click.option('-v', '--verbose', is_flag=True, help=verbose_help)

But in log.py method, there are level for this.

LOG_FORMATS = {
    logging.DEBUG: {'fg': 'white', 'dim': True},
    logging.INFO: {'fg': 'green'},
    logging.WARN: {'fg': 'yellow'},
}

How about making log option as level name, instead of true/false?
If I'm thinking wrong, or if you have other plan, please let me know.

Thanks.

models.py.Message created even with message app disabled

when pg is enabled but "message-board" is disabled.

runserver doesn't work with async Application factory

aiohttp-devtools version: 0.9
aiohttp version: 3.2.1
python version: 3.6.5
Platform: Ubuntu 18.04

Issue Summary

Unable to run aiohttp application defined as async Application factory

Steps to reproduce

Create an app like so:

async def my_web_app():
    app = web.Application()
    app.router.add_get('/', index)
    return app

$ adev runserver --app-factory my_web_app
Note: runserver works if I remove async from definition

I've tried to add this check to runserver/main.py:run_app

 if asyncio.iscoroutine(app):
     app = loop.run_until_complete(app)

but it didn't work (maybe because auxiliary_app is not coroutine)

Version of aiohttp in requirements.txt is old after creating project

I'm using aiohttp-devtools==0.3, after setup project, adev runserver returned:

ImportError: cannot import name 'FileResponse'

steps in tutorial doesnt work on Windows

aiohttp-devtools version:
aiodns (1.1.1)
aiohttp-debugtoolbar (0.5.0)
aiohttp-devtools (0.8)
aiohttp-jinja2 (0.15.0)
aiohttp-session (2.2.0)
aiohttp version:

aiohttp (3.0.1)

python version:
Python 3.6.0 :: Anaconda 4.3.1 (64-bit)
Platform:
Windows 10

Issue Summary

The steps in the tutorial doesnt launch and gives this error

C:\Users\tyoc213\my_new_app>adev runserver
Error: unable to find a recognised default file ("app.py" or "main.py") in the directory "."

after that I added the variables in activate.settings.sh like

setx AIO_APP_PATH "./app/"
setx AIO_STATIC_PATH "./static/"

(I also did try without the ./)

And now I get

C:\Users\tyoc213\my_new_app>adev runserver
Error: error importing "app\main" from "C:\Users\tyoc213\my_new_app": No module named 'app\\main'

There is a main.py inside

Steps to reproduce

Follow the steps in the tutorial without changing to virtual env because Im already running on anaconda.

Configuration files: ini, json, yaml or python

The project created by the start sub-command will need to employ some kind of config file. While people can of course change it, the default used will influence what people use and should therefore make sense.

It should be noted that most of the cleanest "12-factor" deployment pipelines (eg. heroku, Docker) use environment variables to pass sensitive values to apps, so the configuration logic will either need to allow overriding of variables from environment variables or allow environment variables to be referenced from the configuration.

There are a number of options:

ini

Advantages:

pythonic - .ini parsing is included in the python stdlib and used in setup.cfg etc.
no extra requirements

Disadvantages:

not as widely used as other formats and therefore less well understood, hard to describe complex objects with
can't access environment variables

json

Advantages:

widely understood
no extra requirements - included in python stdlib

Disadvantages:

comments not possible
easy to make syntax mistakes: trailing comma, double quotes vs single quotes
can't access environment variables

yaml

Advantages:

widely used

Disadvantages:

extra requirement, also pyyaml also often requires compiling unless you're lucky and can use the manylinux binary
not well documented, the pyyaml docs are confusing and it's not always clear which other docs you should use
can't access environment variables
for those who haven't used it yaml is yet another thing to learn, and not always that intuitive - partly because it's so relaxed and partly as there are multiple ways to do things (eg. lists).

The aiohttp docs discuss this issue and recommend yaml.

python

Advantages:

arguable pythonic
no extra requirements
allows arbitrary logic while configuring
allows access to environment variables inside configuration
used by django and an option in any other project so fairly well understood

disadvantages:

allows arbitrary logic while configuring: logic vs config can get confused

(lets ignore the likes of toml and xml for brevity)

It would in theory be possible to support multiple formats however I think that would lead to a cognitive burden both in development and usage.

I'm happy with either yaml or python but I would suggest python as it's the most "vanilla" and easy way to go. Also if people really want to use another config file they can easy load it inside the python configuration.

@asvetlov ?

Expose version in init.py

Looks like it is standard de-facto for python packaging.

No module named 'aiohttp_devtools.runserver'

Trace:

>> adev runserver server.py make_app
Traceback (most recent call last):
  File "/Users/<user>/.virtualenvs/<project>/bin/adev", line 7, in <module>
    from aiohttp_devtools.cli import cli
  File "/Users/<user>/.virtualenvs/<project>/lib/python3.5/site-packages/aiohttp_devtools/cli.py", line 3, in <module>
    from .runserver.logs import setup_logging
ImportError: No module named 'aiohttp_devtools.runserver'

aio-libs / aiohttp-devtools Goto Github PK

aiohttp-devtools's People

Contributors

Stargazers

Watchers

Forkers

aiohttp-devtools's Issues

Issue Summary

Issue Summary

Code Example

Steps to reproduce

Issue Summary

First:

Second

Steps to reproduce

Issue Summary

Steps to reproduce

Issue Summary

Steps to reproduce

Issue Summary

Steps to reproduce

Issue Summary

Steps to reproduce

Issue Summary

Issue Summary

Detail

Issue Summary

Steps to reproduce

Issue Summary

Steps to reproduce

ini

json

yaml

python

Recommend Projects

Recommend Topics

Recommend Org