jordansissel / eventmachine-tail Goto Github PK

Ran into this exception and wanted to put it somewhere:

Unhandled EventMachine Exception: NoMethodError: undefined method `ino' for nil:NilClass:
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-tail-0.6.3/lib/em/filetail.rb:395:in `handle_fstat'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-tail-0.6.3/lib/em/filetail.rb:333:in `handle_eof'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-tail-0.6.3/lib/em/filetail.rb:370:in `read_file_metadata'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-tail-0.6.3/lib/em/filetail.rb:332:in `handle_eof'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-tail-0.6.3/lib/em/filetail.rb:302:in `schedule_eof'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `call'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `run_deferred_callbacks'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `each'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `run_deferred_callbacks'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'
    /usr/lib64/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'

Reported by jeffreyhorn:

We create a new log file every hour and then update a sym link to point to the most current one.

When I use EM::FileTail to tail these files via EM::file_tail('/path/to/symlink',SubClassofEM::FileTail)

it seems to keep tailing the file that is below the symlink at the time the script starts running, and when the sym link changes, no more lines are produced in the tail.

File::Tail (which is the module I'm converting from) will reopen the file if the symlink changes. Is there any way I can get EM::FileTail to do this as well?

Running the example script results in an error:

/Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-0.12.10/lib/eventmachine.rb:1266:in `watch_filename': Inappropriate ioctl for device - must enable kqueue (Errno::ENOTTY)
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-0.12.10/lib/eventmachine.rb:1266:in `watch_file'
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-tail-0.2.20100517011408/lib/em/filetail.rb:126:in `watch'
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-tail-0.2.20100517011408/lib/em/filetail.rb:61:in `initialize'
from emtail.rb:13:in `initialize'
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-tail-0.2.20100517011408/lib/em/filetail.rb:225:in `new'
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-tail-0.2.20100517011408/lib/em/filetail.rb:225:in `file_tail'
from emtail.rb:33:in `block (2 levels) in main'
from emtail.rb:32:in `each'
from emtail.rb:32:in `block in main'
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `call'
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'
from /Users/harm/ruby1.9p129/lib/ruby1.9/gems/1.9.1/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'
from emtail.rb:31:in `main'
from emtail.rb:38:in `<main>'

I'm using Ruby 1.9p129, OSX 10.5, eventmachine-tail 0.2.20100517011408 and eventmachine 0.12.10

Hi, I'm wondering if this project works on Windows?

Hello :)

I am trying to tail the same file twice but there seems to be a problem, here is a gist describing it http://gist.github.com/458403

Any ideas?

It complains that you haven't implemented the file_found when you're missing file_deleted

gaiottino@34bd543

In my application I am using "logging" instead of "logger" because it offers more features. What I would like is to be able to feed the logger I have already defined into eventmachine-tail.

I have modify "globwatcher.rb" and "filetail.rb" and this works fine as long as the interface of "logging" keep the same methods name as "logger". By doing this I can get all messages in the same logfile.

Thanks for this nice package.
René

Hi

I suspect (from looking at http://code.google.com/p/semicomplete/) that this project is licensed under the new BSD license, but since there's no LICENSE file I can't be 100% sure. Can a license be added?

Thanks

I don't know exactly why but apparently @file gets nil here:
@position = @file.sysseek(0, IO::SEEK_END)

and its raising an error

EventMachine error: undefined method sysseek' for nil:NilClass - /usr/lib64/ruby/gems/1.8/gems/eventmachine-tail-0.6.3/lib/em/filetail.rb:106:in initialize'

I think it has to do with a reopening of a file

maybe check if @file.nil?

Hi,

I'm using em-tail to tail apache log files but I'm running into problems when the files are rotated by Apache. I get the following exception

Tailing started
Tailer(/mnt/var/log/apache2/access-20101218130000.log) @ pos:
Found: /mnt/var/log/apache2/access-20101218130000.log
Tailer(/mnt/var/log/apache2/access-20101218131500.log) @ pos:
Found: /mnt/var/log/apache2/access-20101218131500.log

/home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-tail-0.5.20101204110840/lib/em/filetail.rb:178:in `initialize': No such file or directory - /mnt/var/log/apache2/access-20101218130000.log (Errno::ENOENT)
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-tail-0.5.20101204110840/lib/em/filetail.rb:178:in `open'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-tail-0.5.20101204110840/lib/em/filetail.rb:178:in `open'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-tail-0.5.20101204110840/lib/em/filetail.rb:280:in `block in schedule_reopen'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `call'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `block in run_deferred_callbacks'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `each'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-0.12.10/lib/eventmachine.rb:996:in `run_deferred_callbacks'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'
    from /home/daniel/.rvm/gems/ruby-1.9.2-p0@zen/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'
    from tailer.rb:41:in `'

You can see the code I'm testing with here: https://gist.github.com/746500

It seems it's trying to recreate a FileTail with the old path.

I've come across an issue where file updates are no longer detected after the log file being tailed was rotated. Turning on $DEBUG, I can see that the file move event was being handled as expected and the file was being reopened correctly. However, it looks like the EM::FileWatch also needs to be re-started in order to catch updates on the newly-minted log file. I've patched some stuff together in my dev environment to test this theory out and it seems to fix the issue.

We ran into issues on 1.9 with a file that is supposed to be UTF-8 having invalid characters in it.

A fix was suggested for remote_syslog that should clearly go directly into eventmachine-tail, but I haven't been able to figure out exactly how I would want to fix it.

Here is the discussion that we've had so far:
papertrail/remote_syslog#13

Any thoughts would be welcome for the best way to solve this.

On Linux at least, FileTail handles rotated files correctly because it gets a moved notification. However, because inotify only notifies for file deletion when all file descriptors on the file are closed, FileTail is never notified when the file is flat out deleted (or, say, another file is renamed over the one being monitored). This means that the tailer hangs, even if the file is created again later.

Here's a fairly crude test case demonstrating the issue - note that the file tail fails to pick up the line written after the file is deleted and recreated ("bar"):

#!/usr/bin/env ruby

require 'fileutils'
require 'eventmachine-tail'

EM.run do
  FileUtils.touch('test')
  EM.file_tail('test') do |filetail, line|
    puts "Got line: #{line}"
  end

  EM.next_tick { File.open('test', 'a') {|f| f.puts("foo")} }

  EM.add_timer(1) do
    FileUtils.rm('test')
    EM.add_timer(1) do
      FileUtils.touch('test')
      File.open('test', 'a') {|f| f.puts("bar")}
      EM.add_timer(1) { EM.stop }
    end
  end
end