package main
import (
"fmt"
"log"
"os"
"strings"
"github.com/nxadm/tail"
)
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
func run() error {
fmt.Printf("PID: %v\n", os.Getpid())
logfile := os.Args[1]
t, err := tail.TailFile(logfile, tail.Config{Follow: true, ReOpen: true})
if err != nil {
return err
}
for line := range t.Lines {
fmt.Printf("line: %v; newline ending: %v\n", line.Text, strings.HasSuffix(line.Text, "\n"))
}
return nil
}
And run this against a random text file, go run repro.go tmp.log
and then write to it:
echo -n "foo" >> tmp.log
echo "foo" >> tmp.log
echo "foofoo" >> tmp.log
This creates two lines, both containing foofoo
, but we can't tell from the reader's point of view:
$ go run repro.go tmp.log
PID: 57959
2021/03/29 11:17:07 Waiting for tmp.log to appear...
line: foo; newline ending: false
line: foo; newline ending: false
line: foofoo; newline ending: false
System information
- tail version v1.4.8
- OS: macOS
- Arch: amd64
Additional context
This was reported back in 2014 in the original repo #35 and "fixed" without tests in #40, which deadlocked the process if the tailed file didn't end with a newline. It was then reverted in this repo by merging in #126.
My proposal doesn't affect the library's behaviour, just adds a flag to inform the user of partial rows. I wasn't super sure how to handle splitting or large lines, so please review my code.
If you sent an issue/PR to hpcloud, you can have a look at
this meta issue tracking issues
and PRs of the dormant upstream. However, as the code bases diverge the
issue may have been solved already.
Describe the bug
The StopAtEOF
function can sometimes stop tailing the file before we actually encounter an EOF. In particular, when using the polling function for checking for file changes, we might select this tail.Dying channel here first before the poller gets another chance to notify us of new changes, which I believe makes us then return here, even though we should probably be waiting for another poll cycle.
Expected behaviour
The library checks again for new changes in the file after StopAtEOF
is reached.
To Reproduce
This is a bit awkward to reproduce, since it's based on racyness. However tailing a file that's being written to in short intervals, and calling StopAtEOF
might not return all data before exiting.
Additional context
I'd be happy to contribute a PR for this, but probably won't have the time to do so before next week.
Describe the bug
Simple code:
func main() {
if len(os.Args) != 2 {
panic("not enough arguments")
}
filename := os.Args[1]
t, err := tail.TailFile(filename, tail.Config{Follow: true, ReOpen: true})
if err != nil {
panic(err)
}
for line := range t.Lines {
fmt.Println(line.Text)
}
}
When used in conjunction with continuous truncation (>
) as opposed to appending (>>
) only reacts to the first event, ignore all following ones. Simple test case:
$ while true; do echo "$(date)" > watched.log; sleep 1; done
It will truncate file every second before writing a single line into it.
Expected behaviour
The code correctly detects the truncation events and re-opens the file accordingly. So the output would look something like this:
Mon Dec 18 04:23:44 PM UTC 2023
2023/12/18 16:23:45 Re-opening truncated file watched.log ...
2023/12/18 16:23:45 Successfully reopened truncated watched.log
Mon Dec 18 04:23:45 PM UTC 2023
2023/12/18 16:23:46 Re-opening truncated file watched.log ...
2023/12/18 16:23:46 Successfully reopened truncated watched.log
Mon Dec 18 04:23:46 PM UTC 2023
...
Actual behaviour
It detects first event, and reads a single line after that ignoring everything else:
2023/12/18 16:31:45 Waiting for watched.log to appear...
Mon Dec 18 04:31:48 PM UTC 2023
To Reproduce
See steps above. I've tried with Poll:true
to no avail.
System information
Additional context
If you sent an issue/PR to hpcloud, you can have a look at
this meta issue tracking issues
and PRs of the dormant upstream. However, as the code bases diverge the
issue may have been implemented already.
Is your feature request related to a problem? Please describe.
Support for tail -n server.log
feature, do not watch file, only get last N lines only.
If you start tailing the same file twice (by calling TailFile
and supplying an identical filename), and are using the inotify backend, then only one of the Tail
objects is notified when a line is appended to the file. Which one of the two objects gets notice is chosen at random, and the one which does then catches up on lines it has slept on in previous appends.
I would expect both of the objects to be notified at each append.
Branch: https://github.com/nxadm/tail/tree/noTomb
Tail depends on Tomb v1, a version of the library not updated updated in the last 5 years. The next version of Tomb, v2, is not API compatible with v1 and hasn't been updated for 2 years. A better solution is using the core coroutine workflow around context.Context. This branch implements this workflow and passes all the tests. However, because the API of Tomb v1 is very different to the one of context.Context, the changes in this branch need to be well tested in order to make sure no threading issues occur.
This branch also removes internal packages that resulted in many exported functions and structs. Most of these, probably all, were only useful within tail and their public nature was just a consequence of being in a package. This methods were removed from the public API. Also, the OpenFile function was removed from the API as it doesn't do anything special and is not related to tailing. The documentation was updated and public functions and structs were documented. Programs that use the "useful" API of previous releases will not be impacted:
- the variables DefaultLogger and DiscardingLogger.
- the structs Config, Line, Seekinfo and Tail.
- the functions TailFile, Tell, Stop and StopAtEOF.
This branch, once ready, will be released as a major release, v2.
When the log qps is large, the message read is not a complete line but only a part
I used tail.TailFile
and t.Lines
happily. Until one day I had a UTF16-LE encoded file.
Since golang string
type supports utf-8, it is not capable to throw a UTF-16 string correctly.
It will be nice if a tail.Line
can return a []byte. What do you think?
While looking at the noTomb
branch, I noticed that several of the old tomb.Kill
calls in tail.tailFileSync
were replaced with unreceived fmt.Errorf
calls, which doesn't actually do anything.
For example:
|
fmt.Errorf("Seek error on %s: %s", tail.Filename, err) |
Describe the bug
When errors occur, it is important to get the error message logged out. Otherwise it can get very annoying to debug. I have for example hit the following error:
|
util.Fatal("failed to create Watcher") |
I don't know why fsnotify.NewWatcher()
would fail, but since the error message isn't included in the error log, it is almost impossible to debug.
Expected behaviour
I would expect the error message is a part of the logs using something like the following:
util.Fatal(fmt.Sprintf("failed to create Watcher: %v", err))
Additional context
I actually came onto this error by using https://github.com/hpcloud/tail which is used in https://github.com/kubeflow/katib.
Describe the bug
for example , if a log file is "abcd"
sometimes it can only collect "abc"
this is my code
tails, err := tail.TailFile(fileName, tail.Config{ ReOpen: true, Follow: true, Location: &tail.SeekInfo{Offset: 0, Whence: flag.Whence}, MustExist: false, Poll: false, })
Expected behaviour
To Reproduce
sry ,I have used this to collect over a million log entries, and only a few hundred of them are difficult to reproduce.
System information
- tail version [v1.4.8]
- OS: centos
- Arch: [e.g. arm64]
If you StartTail(file_x) following from end, and then append "hello\n" to file_x, there's a race that makes it uncertain whether your appended line will come out of the tail operation or not; the StartTail operation seeks the end of the file in its own goroutine, and provides no signal that it has done so, so there's no way to reliably force the ordering of the operations.
#48 adds a config option "SyncOpen" that allows StartTail to perform any requested seek synchronously before it returns, thereby guaranteeing an order of operations.
Recommend Projects
-
-
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
An Open Source Machine Learning Framework for Everyone
-
The Web framework for perfectionists with deadlines.
-
A PHP framework for web artisans
-
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
Some thing interesting about web. New door for the world.
-
A server is a program made to process requests and deliver data to clients.
-
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Some thing interesting about visualization, use data art
-
Some thing interesting about game, make everyone happy.
-
Recommend Org
-
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Open source projects and samples from Microsoft.
-
Google โค๏ธ Open Source for everyone.
-
Alibaba Open Source for everyone
-
Data-Driven Documents codes.
-
China tencent open source team.
-