tagomoris / fluent-agent-lite Goto Github PK

Usually, fluent-agent-lite dies if tail command dies by some reasons (for example, someone killed tail process by mistake).

However, during reconnect loop of fluent-agent-lite (AgentLite.pm#L113-L119), fluent-agent-lite does not detect the tail's death, and results in zombie tail process (detection is at AgentLite.pm#L203)

This behavior can easily reproduced by folllowing procedure:

1. Shutdown receiver fluentd process.
2. kill tail process

fluent-agent-lite.conf に存在しないログファイルを書いている場合、

$ sudo /etc/init.d/fluent-agent-lite start
failed. (timeout after 10sec)

とエラーが出ることがある。エラーが出ずに ok. となることもある。

対処法は２つ考えられる

1. 設定ファイルに存在しないファイルを書いた場合、必ず起動スクリプトが失敗するように修正する
2. tail -F を使用しているため、起動時にファイルが存在していなくても問題ない。bin/fluent-agent-lite でのファイル存在チェックのロジックをはずす

手元では 2. の修正を加えている、方針を伺ってから pull req したい。

REPRODUCE PROCEDURE:

ssh -t hostname sudo /etc/init.d/fluent-agent-lite restart

It seems using disown to daemonize is bad.

Some deploy tools such as mina uses ssh -t as default. I'm happy if this is fixed.

I also heard from my colleague as fluent-agent-lite could not be booted via knife-solo (I did not examine well, though)

http://tagomoris.github.io/tarballs/fluent-agent-lite.v0.9.tar.gz contain old spec file.
so rpm build failed on change directory.

rpmbuild -v -ba --clean SPECS/fluent-agent-lite.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.tvtefP
+ umask 022
+ cd /home/akira/repos/make-fluent-agent-lite-rpm/rpmbuild//BUILD
+ cd /home/akira/repos/make-fluent-agent-lite-rpm/rpmbuild/BUILD
+ rm -rf fluent-agent-lite
+ /bin/tar -xf -
+ /usr/bin/gzip -dc /home/akira/repos/make-fluent-agent-lite-rpm/rpmbuild/SOURCES/fluent-agent-lite.v0.9.tar.gz
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd fluent-agent-lite
/var/tmp/rpm-tmp.tvtefP: line 34: cd: fluent-agent-lite: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.tvtefP (%prep)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.tvtefP (%prep)

6c702cf is lastest.

@tagomoris I've noticed that fluent-agent-lite reads 10 lines of log file when we restarts it because tail command prints the last 10 lines of a file as default.

Should fluent-agent-lite read 0 lines at restarting using -n 0 option of the tail command? Otherwirse, it would send again some same lines of log at restarting.

I can make patch for this. Any opinions?

LOGS=$(zcat <<"EOF"
test /root/1.gz
EOF
)

/etc/init.d/fluent-agent-lite start

gzip: stdin: not in gzip format
Starting fluent-agent-lite: ok.