Related: pytest-dev/pytest-cov#139

gevent is an infamous Python library of doing all sorts of monkey patching within the stdlib to get async support for stdlib modules, basically rewriting threading et al. See one of related issues and good explanation.

After upgrading Pyramid 1.7 -> Pyramid 1.8 I cannot run pserve anymore, despite doing in gevent monkey patches as early as possible in the launch script:

"""Wrapper for pserve."""
import sys
from pkg_resources import load_entry_point

try:
    import gevent.monkey
    gevent.monkey.patch_all()
except ImportError:
    pass

def main():
    sys.exit(
        load_entry_point('pyramid', 'console_scripts', 'pserve')()
    )

I get exception on the startup:

Starting monitor for PID 49731.
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/Users/mikko/code/x/venv/lib/python3.5/site-packages/hupper/worker.py", line 98, in run
    while self.pipe.recv_bytes():  # pragma: nocover
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
BlockingIOError: [Errno 35] Resource temporarily unavailable

I do not have a solution yet; I leave issue open here to raise some awareness and for further investigation if there are other gevent users out there.

What I did:

• Using hupper = 1.10.3 on MacOS 12.3.1 -> works like a charm
• Afterwards installed watchman == 2022.01.17.00_0 using MacPorts.

What happened:

• Hupper no longer restarts the process on file changes.
• Stopping the hupper using ^C no longer work, i have to force kill it.
• Sorry, no log messages nor anything else which could guide to the source of the problem. It just hangs and does nothing.
• Uninstalling watchman fixed the problem, so it seems to be the reason behind.

I don't know if its a bug, but it could be.

1. start pserve --reload
2. run pip install -e newpackage
3. pserve does not reload
4. add import newpackage to the project code
5. pserve reloads, but gets ImportError
6. stop pserve; start pserve. Now everything is fine

Looks like some package cache (?) could be involved, which is only cleared on fresh start

Hi,

I have a problem starting a pyramid project with --reload on Windows7

When I type hupper -m http.server I get:

λ hupper -m http.server
Traceback (most recent call last):
  File "C:\Tools\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Tools\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Devel\pyramid\quick_tutorial\env\Scripts\hupper.exe\__main__.py", line 9, in <module>
  File "c:\devel\pyramid\quick_tutorial\env\lib\site-packages\hupper\cli.py", line 21, in main
    worker_kwargs={"alter_sys": True, "run_name": "__main__"})
  File "c:\devel\pyramid\quick_tutorial\env\lib\site-packages\hupper\reloader.py", line 290, in start_reloader
    return reloader.run()
  File "c:\devel\pyramid\quick_tutorial\env\lib\site-packages\hupper\reloader.py", line 111, in run
    if not self._run_worker():
  File "c:\devel\pyramid\quick_tutorial\env\lib\site-packages\hupper\reloader.py", line 144, in _run_worker
    self.worker.start()
  File "c:\devel\pyramid\quick_tutorial\env\lib\site-packages\hupper\worker.py", line 129, in start
    os.set_inheritable(self.stdin_fd, True)
OSError: [WinError 87] Falscher Parameter

I get the same error when starting the pyramid project with --reload

Do you have any idea what I am doing wrong?
Thanks you very much
Regards
Seyhbold

See Pylons/pyramid#2920

If we can figure out what error is being raised in watchdog.py when the inotify limit is reached we can stop squashing it. It would be useful output for a user.

See Pylons/pyramid#3043

The worker is unable to import hupper. The issue is that buildout enhances the sys.path of the python process and this does not survive the fork+exec. This belongs in the preparation data in ipc.py.

Something like hupper -s -- run-some-app to monitor a binary named run-some-app and run it using subprocess. Combine this with the -w option proposed in #18 to support a pretty nice CLI file monitor.

When using pserve the number of inotify instances explodes. Lots of these errors:

The user limit on the total number of inotify instances has been reached; increase the 
 fs.inotify.max_user_instances sysctl

I am running watchman. It turns out that the watchlist gets huge, and contains the path '/usr/lib/python2.7', which means that effectively all python libs are being watched.

I am running inside a virtualenv. Is there maybe something wrong in my configuration?

For now, I downgraded to hupper 1.2 to solve the problem.

I'm using hupper to run a Flask server in a VS Code devcontainer with debugging (using debugpy). Generally all is working well while the server is running - reloading is working perfectly, as is debugging.

The only problem is that the way VS Code runs the app with debugging means that hupper is run within the debugger. Usually this doesn't cause any problems, but when the process ends (e.g. with with ctrl-c) hupper exits with sys.exit(1). The VS Code debugger then picks this up as an unhandled SystemExit exception.

Is there a reason to return a non-zero exit code? Could this be changed to a 0 success exit code? Conceptually I think the process has ended normally so 0 would be an acceptable exit code anyway.

I am using hupper to hot-reload a Dash application running in waitress. hupper is set to run with reload_interval=0 and it works fine on Windows but appears to behave badly by consuming 100% CPU in Linux.

Steps to reproduce:

1. Run a python docker container:

   docker run -it --cap-add SYS_PTRACE python:3.9.13-buster /bin/bash
   

2. In the container install py-spy: pip install py-spy
3. Install poetry: curl -sSL https://install.python-poetry.org | python3 -
4. Add poetry to path: export PATH="/root/.local/bin:$PATH"
5. Clone the app into the container: git clone https://github.com/entropy-lab/entropy.git
6. Checkout to a revision where hupper is enabled: git checkout tags/v0.11.0
7. cd entropy && poetry install
8. Run the app with poetry run entropy serve .

Actual results:

Run top and you will see that the app consumes ~100% CPU.
Run py-spy --pid <process id> with the process id of the app as it is running and you will see that hupper's polling mechanism is the consumer:

  %Own   %Total  OwnTime  TotalTime  Function (filename)
 92.00%  92.00%   717.0s    717.0s   get_mtime (hupper/polling.py)

Expected results:

hupper should consume a reasonably small amount of CPU when running in the app.

Miscellaneous :

You can see how hupper is used by the app here: https://github.com/entropy-lab/entropy/blob/85536ef62f724290d35eea0eb7e7c47134ab05f5/entropylab/dashboard/__init__.py#L61

Thanks,
urig

We need to write some tests!

There are 2 (probably related) errors right now.

1. When the worker process starts "things are fine" (things work if you don't need stdin). However when it restarts there is a IOError Bad File Descriptor emitted in the worker process.

2. When it starts the first time and "things are fine", if you set a pdb breakpoint in your code things will go haywire when you hit it. The process hangs and a windows dialog pops up and you can't actually use the stdin pipe to talk to pdb.

I have a Zope based application and want to run waitress instead of paster as wsgi server, but keep the reload feature for development. Unfortunately, when running
./bin/hupper -m my.entry.point ./paste.ini
A reload is triggered when a file in the chameleon-cache ./var/chameleon-cache changes. Is there a possibility to exclude this directory from the monitored files?

This issue occurred on current (Arch-)Linux and MacOS with
hupper 1.4
Zope 4.0b7
Chameleon 3.5
and there respective dependencies.

Thanks for this nice tool!

Watchdog is basically dead and unmaintained and has caused a couple irritating bugs for people such as Pylons/pyramid#2920

One option is to fork watchdog into watchdog-lite that focuses on our simplified use case and doesn't ship with the cli runner that depends on pyyaml (if we did this I would consider adding it as a direct dependency of hupper). Another is to evaluate less cross-platform options such as recommending installing inotify on some systems, etc. This would be a last resort, however, because watchdog already does a pretty good job.

Right now hupper runs the command immediately and then kills / restarts it when files change. Some interesting use-cases open up if it has an option to only run the command when files change. For example:

hupper --wait-for-changes -- docker restart app

If the command exits then hupper already will wait for files to change and run it again. I think this issue is just about not running it at startup.

This is likely contingent on changes in #19

Add an appending -w option to hupper to watch extra file patterns.

hupper -w foo.ini -w 'media/**' -m myapp

Hupper tries to avoid monitoring all system code loaded into a process due to #40. We do want to monitor user code however. The four ways I'm aware of right now to develop "user" code:

• ensure folder is in PYTHONPATH
• pip install -e . installs a .pth file in site-packages which means the code continues to live properly wherever it is checked out.
• developing inside a virtualenv - some people do this so they can use bin/python
• symlinking the folder containing the code into site-packages which is done manually or using flit install --symlink and I imagine poetry install will also do this.

We want to try to count all of these scenarios as user code that is monitored and only ignore stuff that is third party installed directly into site-packages or stuff that is part of the virtualenv / system python.

related: #40 #41 #42 #60 #61

Warning comes from unconditional import of imp module in hupper.compat

First off, thanks for hupper!

My usecase is to re-run a test suite each time I make a change to my project. I'm using hupper rather than an off the shelf solution like pytest-watcher because I wanted to wrap the test runner in a little repl that allows interacting with test filtering and allows interrogating the results.

When someone pressed ^C or types quit in the repl, I'd like to tell the monitoring proces to also exit, but I couldn't find any way to do that (short of hijacking the proxy's pipe to send an unrecognized message anyway, but that seems... inelegant).

Would you consider adding a new proxy command, exit() (or perhaps exit(code) for this purpose?

If you think the feature is worthwhile but don't have time to add it yourself, I'd be happy to contribute.

Thanks for your consideration!
Josh

Currently I can kill the pserve process and there are still extra python processes sticking around. This is supposed to be handled by watching the pipe but it doesn't seem to work on windows.

flit is a new package build tool that’s I find nicer than distutils/setuptools for simple projects (issues with missing package data are gone!). The develop install equivalent is symlink install that creates and installs dist-info dir + a symlink from site-packages to project dir so that imports work. With that setup, editing a file does not cause hupper (used via pserve) to reload the app.

Our product runs in a gevent.pywsi+Pyramid server with monkey.patch_all() enabled. For the most part this works great. However, the --reload flag to pserve does not work out of the box. Our application fails to start up entirely and source file changes do not trigger a reload. I did manage to get it working, but it required some tweaks. I will detail what our solution was in case you want to add support for this use case or if anyone else out there is curious how to get this working.

I created a pserve wrapper command called gserve:

#!/usr/bin/python3.6
from gevent import monkey; monkey.patch_all()
from ourproduct import monkey; monkey.patch_hupper()
from pyramid.scripts.pserve import main; main()

This ensures that gevent has a chance to patch everything before hupper or pyramid has a chance to run. This wrapper script serves as a drop-in replacement for pserve.

The ourproduct.monkey.patch_hupper() function looks like this:

def patch_hupper():
    # Patch hupper to work with gevent so gserve --reload works!
    from gevent import os
    from hupper.ipc import Connection
    from mock import patch
    _recv_packet = Connection._recv_packet
    @patch('os.read', side_effect=os.tp_read)
    def _patched_recv_packet(self, tp_read):
        return _recv_packet(self)
    Connection._recv_packet = _patched_recv_packet

gevent does not patch os.read or os.write (see gevent.os). I use mock.patch to ensure my replacement of os.read only affects the _recv_packet method. I tried gevent.os.nb_read first, which did not work, then switched to gevent.os.tp_read.

This was enough for hupper to spawn the server in a subprocess and for basic reload functionality to work. However, the server spawned in the subprocess was no longer gevent-patched, as it is not spawned using gserve. This prevented the server from starting up completely as many calls were now blocking that weren't before.

The hupper.ipc module starts a new interpreter and then uses hupper.ipc.spawn_main to import and run a python function (in this case pyramid.scripts.pserve.main). To make sure the subprocess runs both patch_all() and patch_hupper(), I made the following change to hupper/ipc.py:

--- hupper/ipc.py	2018-06-14 01:25:35.000000000 -0500
+++ hupper/ipc.py	2018-06-21 14:14:56.226000000 -0500
@@ -250,7 +250,7 @@


 def get_command_line(**kwds):
-    prog = 'from hupper.ipc import spawn_main; spawn_main(%s)'
+    prog = 'from gevent import monkey; monkey.patch_all(); from ourproduct import monkey; monkey.patch_hupper(); from hupper.ipc import spawn_main; spawn_main(%s)'
     prog %= ', '.join('%s=%r' % item for item in kwds.items())
     opts = args_from_interpreter_flags()
     args = [sys.executable] + opts + ['-c', prog]

After that hack was in place, the server remained patched after being spawned in a subprocess by hupper and would successfully reload after source changes. We're now running with that patched version of hupper, but an official solution would be nice.

One possible fix would be for us to supply hupper with code that we want executed before anything else in the subprocess. We would then be able to give it the patch statements we would like executed and it would blindly execute them prior to from hupper.ipc import spawn_main; spawn_main(%s).

Another fix might be to spawn the server in yet another subprocess using the exact sys.argv used originally. This would cause the server to be spawned again using the gserve wrapper instead of hupper being provided with a module entry point (i.e. pyramid.scripts.pserve.main) to load and run, which ends up skipping gserve entirely.

Thoughts and comments appreciated! Hopefully this at least helps some people using patch_all() also utilize the --reload flag to pserve.

Problem

The cc-by-nc-sa-3.0-us license on the docs subdirectory is much less permissive than the MIT license on everything else. As far as I can tell the license on the docs applies to any usage of hupper, so the real top-level license is not MIT but MIT AND cc-by-nc-sa-3.0-us

Solution

Pick a license and stick to it.

Context

Dual-licensing is usually one or the other, pick the one you like. In this case there are two licenses on pretty much the same code.

I'm not entirely sure what the specific root cause of this issue, but I feel like the behavior here is not quite right.

So basic setup, there is a virtualenv located at /opt/warehouse, and inside of that a directory called /opt/warehouse/src which has a git checkout of the Warehouse code, which is added to sys.path using PYTHONPATH.

Now, it just so happens that when inside of a virtual environment, the root of the virtual environment is set to be the "data" path (sysconfig.get_paths()["data"]). This means that /opt/warehouse gets detected as a "system path" by #40, and because the project files are in an unrelated directory within that directory, it detects the entire project as an installed project and doesn't monitor it.

It seems to me, the easiest fix would be to not use all of the paths returned by sysconfig.get_paths(). There are 8 keys that can exist in the dictionary returned by that API:

• data
• include
• platinclude
• platlib
• platstdlib
• purelib
• scripts
• stdlib

It seems to me, like installed packages are not going to exist in data, include, platinclude, and maybe scripts, so it would make sense to exclude those directories from the paths you get from sysconfig.get_paths().

Another option would be, instead of doing a simple startswith, is to figure out which sys.path entry this file is imported from (by comparing the filename to every entry in sys.path, and making it "owned" by whichever sys.pathentry is longer *and* is a prefix match), and then only ignore files whosesys.path`` entry matches on the system paths.

Hupper seems to avoid forwarding SIGTERM/SIGINT to the child process. It would be quite nice if it forward them. Especially SIGTERM as hupper does not seem to anything at all in response to that signal.

This need comes from developing my applications responses to said signals. As the default behavior for hupper on SIGINT seems to be to exit, an optional flag to simply forward it would be pretty handy.

I added a new module to my code base, and as soon as I updated my Pyramid projects __init__.py for config.scan all hell broke loose:

Serving on http://0.0.0.0:6543
Killing server with PID 83267.
Traceback (most recent call last):
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/reloader.py", line 107, in run
    if not self._run_worker():
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/reloader.py", line 161, in _run_worker
    self.monitor.add_path(path)
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/reloader.py", line 43, in add_path
    self.monitor.add_path(p)
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/watchdog.py", line 34, in add_path
    if dirpath not in self.dirpaths:
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/observers/fsevents.py", line 172, in schedule
    return BaseObserver.schedule(self, event_handler, path, recursive)
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/observers/api.py", line 290, in schedule
    timeout=self.timeout)
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/observers/fsevents.py", line 73, in __init__
    self.snapshot = DirectorySnapshot(watch.path, watch.is_recursive)
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/utils/dirsnapshot.py", line 207, in __init__
    st = stat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/xistence/Projects/fulgent/fulgent/subscribers/__pycache__'

During handling of the above exception, another exception occurred:

ValueError: PyCapsule_GetPointer called with invalid PyCapsule object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/xistence/.ve/fulgent/bin/pserve", line 11, in <module>
    load_entry_point('pyramid==1.8.dev0', 'console_scripts', 'pserve')()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 37, in main
    return command.run()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/pyramid/scripts/pserve.py", line 154, in run
    verbose=self.options.verbose,
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/reloader.py", line 273, in start_reloader
    return reloader.run()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/reloader.py", line 113, in run
    self._stop_monitor()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/reloader.py", line 194, in _stop_monitor
    self.monitor.stop()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/hupper/reloader.py", line 49, in stop
    self.monitor.stop()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/utils/__init__.py", line 99, in stop
    self.on_thread_stop()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/observers/api.py", line 357, in on_thread_stop
    self.unschedule_all()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/observers/api.py", line 353, in unschedule_all
    self._clear_emitters()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/observers/api.py", line 231, in _clear_emitters
    emitter.stop()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/utils/__init__.py", line 99, in stop
    self.on_thread_stop()
  File "/Users/xistence/.ve/fulgent/lib/python3.5/site-packages/watchdog/observers/fsevents.py", line 77, in on_thread_stop
    _fsevents.stop(self)
SystemError: <built-in function stop> returned a result with an error set

Steps:

1. Start pserve with --reload
2. mkdir subscribers && touch subscribers/__init__.py
   This didn't cause the reloader to do anything, because it didn't know about this path
3. Modify __init__.py to add config.scan('.subscribers')
4. Reloader reloads because of change to existing file
5. ???
6. Boom

I have a Pyramid project where content is authored as Markdown on disk. I want treat pserve as an authoring server: you edit Markdown content, the browser reloads.

Thus I need something that watches for a recursive glob and hands changes to a callback I register. I saw the WatchmanFileMonitor discusses a callback. But I'm not sure:

• How in Pyramid to get the in-use hupper to make a registration
• Will it replace the callback already in use

On a related note, I see the callback is called with each file in a changeset. Does this mean, when a bunch of files are copied in, a bunch of callbacks are fired? (Would be nice to batch things.)

When I run my application with hupper -m and hit a pdb.set_trace(), the prompt appears and I can interact with the debugger, but neither cursors nor tab-completion work as expected. Instead they emit their raw escape sequence, for example this is what happens when I press the cursor up key:

(Pdb) l
…
 10  	def foo():
 11  	    import pdb; pdb.set_trace()
 12  ->	    pass
[EOF]
(Pdb) ^[[A

Running it without hupper does not exhibit the problem.

This is on GNU/Linux (Debian sid), with Python 3.6.

If watchman is installed on the system, hupper fails to monitor files.

watchman's binary version : 20210409.063814.0
hupper version : hupper==1.10.2

Steps to reproduce (using a Pyramid application)

mkvirtualenv test_hupper_reload
pip install pyramid cookiecutter 
cookiecutter gh:Pylons/pyramid-cookiecutter-starter
# Add an app test_pyramid_app
cd test_pyramid_app
python setup.py develop
pserve development.ini --reload

Generates the following errors (returned by the watchman service)

[ERROR] watchman error=RootResolveError: RootResolveError: unable to resolve root /home/gas/test/test_pyramid_app: RootResolveError: directory /home/gas/test/test_pyramid_app is not watched
Starting server in PID 12261.
2021-05-11 11:33:29,793 INFO  [waitress:485][MainThread] Serving on http://[::1]:6543
2021-05-11 11:33:29,793 INFO  [waitress:485][MainThread] Serving on http://127.0.0.1:6543
[ERROR] watchman error=RootResolveError: RootResolveError: unable to resolve root /home/gas/test/test_pyramid_app/test_pyramid_app: RootResolveError: directory /home/gas/test/test_pyramid_app/test_pyramid_app is not watched
[ERROR] watchman error=RootResolveError: RootResolveError: unable to resolve root /home/gas/test/test_pyramid_app/test_pyramid_app/models: RootResolveError: directory /home/gas/test/test_pyramid_app/test_pyramid_app/models is not watched
[ERROR] watchman error=RootResolveError: RootResolveError: unable to resolve root /home/gas/test/test_pyramid_app/test_pyramid_app/scripts: RootResolveError: directory /home/gas/test/test_pyramid_app/test_pyramid_app/scripts is not watched

Modifications to any file (.py or .ini file) won't fire any reload action.

While hupper 1.10.3 works great with watchdog 2.1.3, with recently released 2.1.4 I get the following traceback:

Traceback (most recent call last):
  File "/usr/local/bin/pserve", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/pyramid/scripts/pserve.py", line 34, in main
    return command.run()
  File "/usr/local/lib/python3.9/site-packages/pyramid/scripts/pserve.py", line 245, in run
    hupper.start_reloader(
  File "/usr/local/lib/python3.9/site-packages/hupper/reloader.py", line 489, in start_reloader
    return reloader.run()
  File "/usr/local/lib/python3.9/site-packages/hupper/reloader.py", line 138, in run
    with self._setup_runtime():
  File "/usr/local/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.9/site-packages/hupper/reloader.py", line 179, in _setup_runtime
    with self._start_monitor():
  File "/usr/local/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.9/site-packages/hupper/reloader.py", line 209, in _start_monitor
    self.monitor.start()
  File "/usr/local/lib/python3.9/site-packages/hupper/reloader.py", line 56, in start
    self.monitor.start()
  File "/usr/local/lib/python3.9/site-packages/watchdog/observers/api.py", line 254, in start
    for emitter in self._emitters.copy():
AttributeError: 'WatchdogFileMonitor' object has no attribute '_emitters'

Hi,

I’d like to use Twisted as a WSGI container for a Pyramid app and use hupper in dev.

Sadly there seems to be something broken about that setup when cleaning that I suspect is related to threads:

$ hupper -m twisted web --path=.
Starting monitor for PID 78003.
2017-01-11T17:46:46+0100 [-] Site starting on 8080
2017-01-11T17:46:46+0100 [twisted.web.server.Site#info] Starting factory <twisted.web.server.Site instance at 0x1021e5a28>
2017-01-11T17:46:46+0100 [twisted.application.runner._runner.Runner#info] Starting reactor...
^CKilling server with PID 78003.
2017-01-11T17:46:49+0100 [-] Received SIGTERM, shutting down.
2017-01-11T17:46:49+0100 [-] Received SIGINT, shutting down.
2017-01-11T17:46:49+0100 [-] Unhandled Error
	Traceback (most recent call last):
	  File "/Users/hynek/.virtualenvs/tempenv-1c44245683697/lib/python2.7/site-packages/twisted/application/runner/_runner.py", line 63, in run
	    self.startReactor()
	  File "/Users/hynek/.virtualenvs/tempenv-1c44245683697/lib/python2.7/site-packages/twisted/application/runner/_runner.py", line 166, in startReactor
	    reactor.run()
	  File "/Users/hynek/.virtualenvs/tempenv-1c44245683697/lib/python2.7/site-packages/twisted/internet/base.py", line 1199, in run
	    self.mainLoop()
	  File "/Users/hynek/.virtualenvs/tempenv-1c44245683697/lib/python2.7/site-packages/twisted/internet/base.py", line 1208, in mainLoop
	    self.runUntilCurrent()
	--- <exception caught here> ---
	  File "/Users/hynek/.virtualenvs/tempenv-1c44245683697/lib/python2.7/site-packages/twisted/internet/base.py", line 801, in runUntilCurrent
	    f(*a, **kw)
	  File "/Users/hynek/.virtualenvs/tempenv-1c44245683697/lib/python2.7/site-packages/twisted/internet/base.py", line 584, in stop
	    "Can't stop reactor that isn't running.")
	twisted.internet.error.ReactorNotRunning: Can't stop reactor that isn't running.

2017-01-11T17:46:49+0100 [-] (TCP Port 8080 Closed)
2017-01-11T17:46:49+0100 [twisted.web.server.Site#info] Stopping factory <twisted.web.server.Site instance at 0x1021e5a28>
2017-01-11T17:46:49+0100 [-] Main loop terminated.

Please note I’m pressing Ctrl-C there, it’s not exploding or something.

To reproduce simply install twisted and hupper and run the command line from my example.

Any ideas how to fix that? Since Twisted doesn’t have own means for reloading, it would be great if hupper had first class support.

P.S. Tested on macOS Sierra with both Python 3.5.2 & 2.7.13.

I tried to use hupper on a process running inside a windows service. In order to use the win32serviceutil framework, I have to run my process in a separate thread, so that the main thread can listen for service control messages.

Unfortunately, hupper does not work in this configuration, complaining:

Traceback (most recent call last):
  File "AppSrvService.py", line 73, in run
    WaitressServer.main(self.waitress_args)  # blocking
  File "c:\ProgramData\conda\envs\myenv\lib\site-packages\webware\Scripts\WaitressServer.py", line 181, in main
    serve(args)
  File "c:\ProgramData\conda\envs\myenv\lib\site-packages\webware\Scripts\WaitressServer.py", line 44, in serve
    hupper.start_reloader(
  File "c:\ProgramData\conda\envs\myenv\lib\site-packages\hupper\reloader.py", line 489, in start_reloader
    return reloader.run()
  File "c:\ProgramData\conda\envs\myenv\lib\site-packages\hupper\reloader.py", line 138, in run
    with self._setup_runtime():
  File "c:\ProgramData\conda\envs\myenv\lib\contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "c:\ProgramData\conda\envs\myenv\lib\site-packages\hupper\reloader.py", line 180, in _setup_runtime
    with self._capture_signals():
  File "c:\ProgramData\conda\envs\myenv\lib\contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "c:\ProgramData\conda\envs\myenv\lib\site-packages\hupper\reloader.py", line 239, in _capture_signals
    psig = signal.signal(signum, handler)
  File "c:\ProgramData\conda\envs\myenv\lib\signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread of the main interpreter

The Python documentation states:

Python signal handlers are always executed in the main Python thread, even if the signal was received in another thread. This means that signals can’t be used as a means of inter-thread communication. You can use the synchronization primitives from the threading module instead.

Would it be possible to make hupper use threading module primitives instead of signals?

when running hupper.cli I sometimes get:

/home/graingert/projects/platform/pricing/models.py changed; reloading ...
Gracefully killing the server.

worker: Warm shutdown (MainProcess)
Server did not exit, forcefully killing.
Exception in thread Thread-16 (_read_loop):
Traceback (most recent call last):
  File "/home/graingert/anaconda3/envs/coiled/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/home/graingert/anaconda3/envs/coiled/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
Starting monitor for PID 1043314.
  File "/home/graingert/anaconda3/envs/coiled/lib/python3.11/site-packages/hupper/ipc.py", line 169, in _read_loop
    self.on_recv(None)
TypeError: 'NoneType' object is not callable

Hi,

Thanks for hupper, it's awesome !

I'm hitting a weird behaviour, similar to #47 . I'm using hupper in https://github.com/dalibo/temboard, using hupper API: https://github.com/dalibo/temboard/blob/c35895bf79a65758270de6fa5ea73869f17cd82f/agent/temboardagent/cli/app.py#L196-L210

In a debian container, Python python3.9, I enable hupper like this:

root@d303b92213a0:/var/lib/temboard-agent# sudo -Eu postgres DEBUG=y python3 -m  temboardagent
13:29:05 temboardagent[1239] INFO:  app: Starting temboard-agent 9.0.dev0.
13:29:05 temboardagent[1239] DEBUG:  taskmanager: Register worker discover
...
Starting monitor for PID 1243.
...
13:29:05 temboardagent[1243] INFO:  app: Starting temboard-agent 9.0.dev0.
...

Hot reloading works well. Here is the logs when I hit Ctrl-C:

^CReceived SIGINT, waiting for server to exit ...
13:29:08 temboardagent[1243] INFO:  services: Interrupted.
13:29:08 temboardagent[1257] INFO:  services: Interrupted.
13:29:08 temboardagent[1258] INFO:  services: Interrupted.
13:29:08 temboardagent[1258] INFO:  taskmanager: Aborting jobs.
13:29:08 temboardagent[1258] DEBUG:  services: Done. service=worker pool
13:29:08 temboardagent[1243] DEBUG:  services: Terminating background service. service=scheduler pid=1257
13:29:08 temboardagent[1243] DEBUG:  services: Terminating background service. service=worker pool pid=1258
13:29:09 temboardagent[1243] DEBUG:  services: Waiting background services.
13:29:09 temboardagent[1257] DEBUG:  services: Done. service=scheduler
Server did not exit, forcefully killing.

Then hupper hangs forever. I need to type Ctrl-C a second time.
I tried to call graceful_shutdown() from a sigint_handler without success.

Do you have a clue on something wrong I do ?

Regards,
Étienne

Several interesting issues have arisen in my many hours fighting multiprocessing. The most obvious is the issue with forwarding stdin to the subprocess which requires terrible hacks, especially on windows.

However another much more subtle issue is this... on unix multiprocessing by default does not fork+exec. It only forks. This is interesting because it uses the original python interpreter. The issue is that any libraries that were already imported are not re-imported into the child process because they are already found in memory. Instead we would prefer to fork+exec a new subprocess which uses its own interpreter to get a fresh system. To be more clear, if you import a library prior to hupper.start_reloader and then you try to develop on that library, it will not be reloaded even though hupper says it restarted.

Currently patterns are expanded prior to passing files to the monitors. It would be nice to add pattern support directly to the monitors and have them re-evaluate the list of watched files to find anything new. Both watchdog and polling can be updated to see new files and folders and decide what to do with them.

Hello all, first off thanks for hupper 🌟

The warehouse team uses hupper in our development stack but recently we started noticing the it failed to restart on code changes. I believe this is because we invoke hupper in a Docker Compose environment causes the input call to fail with an EOF.

We've worked around it by pinning the hupper version to 1.7 but ideally there'd be a CLI option to prevent reading from stdin. Would that be acceptable?

Some worker processes could ignore SIGTERM signals. In such cases terminating reloader process ends with reloader process waiting for worker termination forever (in self.worker.join()).

Ugly result is that killing reloader in this state makes worker process orphan.

It seems reasonable that reloader should wait for worker termination with some (short) timeout and then kill worker with SIGKILL.

https://facebook.github.io/watchman/

I do not want hupper to depend on watchdog but if it's available we should support it. The reason is that watchdog has a few tricky and unnecessary dependencies and seems to be not very actively maintained right now so a hard dependency would make hupper unavailable on certain systems.

This basically means writing an alternate to the PollingFileMonitor and having start_reloader attempt to use it if watchdog is available. Finally we could add an extra requirement such as hupper[watchdog] which depends on watchdog and update the documents as such.

reloader.watch_files(...) should be able to accept glob patterns like assets/** or assets/**/*.png. This feature should be doable by simply adding support for this to the watchdog and polling file monitors.