Comments (9)
I'll have to take a look a it later today. This is a dictionary of all
current watches (using the "wd" a descriptor returned by the kernel for
each). It looks like the watch is being cleaned-up due to some other event.
On Sat, Jun 13, 2015 at 5:25 AM, backbord [email protected] wrote:
Hi, I've recently started to use your nice little project in version 0.2.3
(I used pyinotify before).
However, I stumbled across the following error and hope that you can help
me.Traceback (most recent call last):
File "test.py", line 88, in
sys.exit(main())
File "test.py", line 82, in main
handle_events(path)
File "test.py", line 49, in handle_events
for event in notifier.event_gen():
File "/foo/bar/lib/python2.7/site-packages/inotify/adapters.py", line 193, in event_gen
for event in self.__i.event_gen():
File "/foo/bar/lib/python2.7/site-packages/inotify/adapters.py", line 148, in event_gen
in self.__handle_inotify_event(fd, event_type):
File "/foo/bar/lib/python2.7/site-packages/inotify/adapters.py", line 134, in __handle_inotify_event
path = self.__watches_r[header.wd]
KeyError: 226The number varies. I've seen all from 9 to 10 and from 220 to 229.
Have you seen this error before and can advice?The function handle_events(path) is intended to announce all files under
a path whenever they are closed after writing. It is defined as follows.def handle_events(path):
# we intend to react on files being closed after writing and directories being created mask = inotify.constants.IN_CLOSE_WRITE | inotify.constants.IN_CREATE # block_duration_s is the pause [s] between calls to the kernel # cycle: (poll for events, yield events sequentially, yield None, sleep for block_duration_s, loop) notifier = inotify.adapters.InotifyTree(path=path, mask=mask, block_duration_s=1) for event in notifier.event_gen(): if event is None: continue (header, _type_names, watch_path, filename) = event # InotifyTree gives IN_ISDIR, IN_CREATE, IN_DELETE regardless of the actual mask for technical reasons # we ignore everything not in our mask if not header.mask & mask: continue # a file has been closed after writing. something to announce. if header.mask & inotify.constants.IN_CLOSE_WRITE: fpath = os.path.join(watch_path, filename) announce_file(fpath) # inotify adds a watch for the newly created path automatically (and removes it if the path is deleted) # but there might already be some files in the folder before the watch is active. let's make sure to announce them if header.mask & inotify.constants.IN_CREATE & inotify.constants.IN_ISDIR: announce_all(watch_path)
Best regards,
Tim—
Reply to this email directly or view it on GitHub
#1.
from pyinotify.
At closer inspection, the error pops up regularly after my cleanup process removed old files and empty subdirectories from the watched directory $HOME/mirror
.
find $HOME/mirror -mmin +2880 -type f -exec rm -f {} \;
find $HOME/mirror -depth -type d -empty -exec rmdir {} \;
Could it be the case that the IN_DELETE events for files and directories which inotify creates are out of order (or read out of order by PyInotify) so that the watch for a subdir is removed (with del self.__watches_r[wd]
in line 71 of adapters.py
) before the IN_DELETE for a file in that subdir is processed?
Please let me know if I can help to find the cause. :-)
from pyinotify.
Hm, the watched $HOME/mirror
gives PyInotify some trouble and as event_gen()
is a generator, I'm having trouble handling exceptions from the outside.
From the traceback in my first post and the following one, I identified two places where removed watches appear to cause problems.
Traceback (most recent call last):
File \"test.py\", line 88, in <module>
sys.exit(main())
File \"test.py\", line 82, in main
handle_events(path)
File \"test.py\", line 49, in handle_events
for event in notifier.event_gen():
File \"/foo/bar/lib/python2.7/site-packages/inotify/adapters.py\", line 215, in event_gen
self.__i.remove_watch(full_path, superficial=True)
File \"/foo/bar/lib/python2.7/site-packages/inotify/adapters.py\", line 68, in remove_watch
wd = self.__watches[path]
KeyError: 'some/deleted/directory'
I'll let my program run with this patch to PyInotify and will get back to you if I see anything interesting.
--- inotify/adapters.py
+++ inotify/adapters.py
@@ -65,7 +65,9 @@
our tracking since inotify already cleans-up the watch.
"""
- wd = self.__watches[path]
+ wd = self.__watches.get(path)
+ if wd is None:
+ return
del self.__watches[path]
del self.__watches_r[wd]
@@ -131,7 +133,9 @@
self.__buffer = self.__buffer[event_length:]
- path = self.__watches_r[header.wd]
+ path = self.__watches_r.get(header.wd)
+ if path is None:
+ break
yield (header, type_names, path, filename)
buffer_length = len(self.__buffer)
Again, thanks for putting together such a nice straightforward code!
Best regards,
Tim
from pyinotify.
Hmm. I prefer very strict code. However, as you mentioned before, things
might be coming in out-of-order (child events being received after the
child subdirectory was already removed).
On Jun 16, 2015 3:13 AM, "backbord" [email protected] wrote:
Hm, the watched $HOME/mirror gives PyInotify some trouble and as
event_gen() is a generator, I'm having trouble handling exceptions from
the outside.From the traceback in my first post and the following one, I identified
two places where removed watches appear to cause problems.Traceback (most recent call last):
File "test.py", line 88, in
sys.exit(main())
File "test.py", line 82, in main
handle_events(path)
File "test.py", line 49, in handle_events
for event in notifier.event_gen():
File "/foo/bar/lib/python2.7/site-packages/inotify/adapters.py", line 215, in event_gen
self.__i.remove_watch(full_path, superficial=True)
File "/foo/bar/lib/python2.7/site-packages/inotify/adapters.py", line 68, in remove_watch
wd = self.__watches[path]
KeyError: 'some/deleted/directory'I'll let my program run with this patch to PyInotify and will get back to
you if I see anything interesting.--- inotify/adapters.py+++ inotify/adapters.py@@ -65,7 +65,9 @@
our tracking since inotify already cleans-up the watch.
"""
wd = self.__watches[path]+ wd = self.__watches.get(path)+ if wd is None:+ return del self.__watches[path] del self.__watches_r[wd]@@ -131,7 +133,9 @@ self.__buffer = self.__buffer[event_length:]
path = self.__watches_r[header.wd]+ path = self.__watches_r.get(header.wd)+ if path is None:+ break yield (header, type_names, path, filename) buffer_length = len(self.__buffer)
Again, thanks for putting together such a nice straightforward code!
Best regards,
Tim—
Reply to this email directly or view it on GitHub
#1 (comment).
from pyinotify.
Hi Dustin,
I can neither catch nor work around an exception thrown from the generator. (See for example http://stackoverflow.com/a/11366139/3985756)
However, I agree, that strict code is the way to go -- do you have an idea on how to cope with events being handled out-of-order?
My python is a bit too limited to come up with a straight-up answer. :-)
Best regards,
Tim
from pyinotify.
I think your way is already the best way for IDs that aren't going to be
reappearing.
Dustin
On Jun 16, 2015 3:59 AM, "backbord" [email protected] wrote:
Hi Dustin,
I can neither catch nor work around an exception thrown from the
generator. (See for example http://stackoverflow.com/a/11366139/3985756)
However, I agree, that strict code is the way to go -- do you have an idea
on how to cope with events being handled out-of-order?
My python is a bit too limited to come up with a straight-up answer. :-)Best regards,
Tim—
Reply to this email directly or view it on GitHub
#1 (comment).
from pyinotify.
If this seems to work, please submit a PR.
Dustin
On Tue, Jun 16, 2015 at 4:06 AM, Dustin Oprea [email protected]
wrote:
I think your way is already the best way for IDs that aren't going to be
reappearing.Dustin
On Jun 16, 2015 3:59 AM, "backbord" [email protected] wrote:Hi Dustin,
I can neither catch nor work around an exception thrown from the
generator. (See for example http://stackoverflow.com/a/11366139/3985756)
However, I agree, that strict code is the way to go -- do you have an
idea on how to cope with events being handled out-of-order?
My python is a bit too limited to come up with a straight-up answer. :-)Best regards,
Tim—
Reply to this email directly or view it on GitHub
#1 (comment).
from pyinotify.
Ok, my first pull request ever. Hope it's to your liking. :-)
Thanks and regards,
Tim
from pyinotify.
I've performed a release of 0.2.4. This includes the fix.
from pyinotify.
Related Issues (20)
- Provide test.sh within release tarballs
- Exception during shutdown because of __del__ HOT 2
- Feature: add timestamp to events.
- Feature: InotifyTree depth, or path mask
- [Improvement] Adapters: Remove duplicate code
- Question about fd monitoring
- InotifyTree does not pick up file events if the directories created HOT 3
- crashes when encountering inaccessible directories (mode 0700)
- pytest Failed under CentOS7/Python3.8 HOT 1
- `superficial` parameter not forwarded to remove_watch_with_id() HOT 1
- Does this project have any active maintainers? HOT 3
- Create Dockerfile to support unit-test debugging/development from local and fix current CI issues
- Issue running with socketio HOT 1
- Package seems to be broken HOT 2
- AttributeError: 'Inotify' object has no attribute '_Inotify__inotify_fd' HOT 1
- Problem in SunOS/Solaris
- test.sh fails
- Release a new version with recent fixes?
- Moves are not handled properly HOT 2
- Superficial remove prevents later remove
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyinotify.