bensadeh / tailspin Goto Github PK
View Code? Open in Web Editor NEW๐ A log file highlighter
License: MIT License
๐ A log file highlighter
License: MIT License
Allow me to pass in a directory. Tail every file in it that is named *.log
.
I want to only see ERROR, or WARN, or INFO logs.
Press 1 = only show ERROR
Press 2 = show > WARN
Press 3 = show > INFO
Press 4 = show all
If the input is in these two formats, parse them to canonical format.
I donโt want to waste terminal space on todays date. It is today. I know that. Give me back 10 charactes (i.e. 15%) of my terminal area.
Hey! I love this tool! One thing that would be really useful for low level developers is highlighting pointers/memory addresses, potentially also splitting them up into 32bit chunks so they're easier to read :)
I want to add new custom keywords but I want them to match the colors used by default for INFO, WARN, ERROR etc
I have created my own ~/.config/tailspin/config.toml
file but don't want to guess what those color values are.
So what does tailspin use to decide what those colors should be by default? Is it embedded in the code or is there some config file I can look at?
Hi,
Nice tool. I am now looking into packaging it for Debian.
One little nit, though: The executable is called "spin" which clashes with another tool.
Please consider renaming this tool. I suggest same name as the project: "tailspin".
In fact, for now I am preparing the Debian package with name changed to "tailspin" because it is not permitted for multiple packages in Debian to provide same name (unless they provide same ABI which clearly this name clash is not an example of).
spin -f logfile
compared to less +F logfile
just stops following on one and the same file.
Hi,
I was asked to package tailspin
for pkgsrc/NetBSD yesterday.
I'll be working on getting this done in an hour or so, but I've noticed that renaming the binary and sed
ing the new name into the manpage will be required, as there's already a spin
binary in the package tree.
Considering the existence of spin, it would probably be better to call the binary tailspin
and avoid conflicts.
I'm open to other suggestions if you have other preferences.
Regards
I think it would be better if subcommand generate-config
would not create default config file, but simply spew config data to stdout. That way, the user is free to either create default or custom config file, or to use the output to make a diff against an existing config file.
Additionally, I think it would be better if a) instead of a hardcoded default path XDG evironment variables was consulted (which leads to same default path as now, so no surprises for the millions of existing users of the tool), and b) not only a user-specific path is used but (when that is missing) system-shared paths are consulted too.
All of this seems possible with the crate microxdg
using Xdg::search_config_file()
.
(the more popular crate Directories
unfortunately does not support system-wide lookups).
Recently introduced std::io::IsTerminal was stabilized only since rustc v0.70. You might want to add rust-version = "1.70.0"
to your Cargo.toml, or switch to using atty.
Here are some logging definitions from loguru
in Python.
Level name | Severity value | Logger method |
---|---|---|
TRACE | 5 | logger.trace() |
DEBUG | 10 | logger.debug() |
INFO | 20 | logger.info() |
SUCCESS | 25 | logger.success() |
WARNING | 30 | logger.warning() |
ERROR | 40 | logger.error() |
CRITICAL | 50 | logger.critical() |
Hi! May I ask what you think of trying out minus instead of less
?
Hi, gave tailspin a spin :) and while it works on smaller logfiles and tailing others (or even folders which is great) it seems to have problems with large (>100mb) files. So I think there may be something off with the less pager.
Welcome to the rude world of character encoding:
~ โฏ echo "รฉ" | iconv -f utf8 -t latin1 | /tmp/tailspin/spin
Error reading from stdin: stream did not contain valid UTF-8
thread 'main' panicked at 'Could not receive EOF signal from oneshot channel: RecvError(())', /home/compil/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tailspin-1.2.1/src/main.rs:107:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Here, the error message is clear, but I have logs with Latin1 encoded value, and the application crashes. It shouldn'tย :)
I suspect other encoding not compatible with UTF-8 generate the same crash.
With a process name like my-program[12345]
, only the part behind the dash is highlighted.
I use tspin -f 2023-12-30.log
command, and prompt "'less' command not found. Please ensure it is installed and on your PATH."
what should I do?
Manifest declares an unused build-dependency on clap_mangen (should have been drop when reverting to generating with asciidoc).
It would be lovely to have a homebrew-core package to just run: brew install tailspin
.
Alternatively a tailpsin tap would be a nice solution, then a bensadeh/homebrew-tap
repo is needed and to install would need:
brew tap bensadeh/tap
brew install tailspin
The defaults are pretty good but would like to disable highlighting of "true" and "false" - how to best achieve that? The examples don't seem to mention booleans. Thanks!
With both the -f
and -l
flags on the v2.0.0 release, The output is just printed plain to the screen, no formatting.
Hi!
Running the following command panics at the end of the file:
> cargo run example-logs/example2 -p
// ... the rest of the file ...
2022-09-10 02:37:29,743 [Worker-63: Building] INFO // ...
The application panicked (crashed).
Message: called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "task was cancelled" }
Location: src/main.rs:60
And on the offending line, we find ourselves with a gift! (an unwra- like u-unwrapping a gift, opening a present, etc. etc.)
Lines 53 to 61 in c6ed6ab
Consider adding statically-linked binaries for Mac OS X, Linux, and OpenBSD. This would make it super convenient for users to download and use the executable via HTTPS, quickly and securely.
Some users are reporting errors when installing using apt
: https://news.ycombinator.com/item?id=38111874.
Looking for help troubleshooting as I don't use a Debian based system.
The application panicked (crashed).
Message: HOME directory not set: NotPresent
Location:
Line 14 in c148d6a
The HOME
environment variable name is USERPROFILE
on Windows.
tailspin claims to be a drop-in replacement for tail -f, yet it does not support data from stdin.
This works: "echo foobar | tail -f"
This fails: "echo foobar | tailspin -f"
The more interesting use case is this: "journalctl -f | tailspin -f"
Ubuntu 22.04 is on rustc 1.66.1 not able to upgrade to 1.67 due to dependencies.
rustc 1.66.1+dfsg0ubuntu1-0ubuntu0.22.04.1 [Ubuntu/jammy main]
If I install tailspin via Cargo I end up in missing dependency toml v0.8.6 because it require rustc 1.67 which is unavailable.
error: failed to compile `tailspin v2.1.0`, intermediate artifacts can be found at `/tmp/cargo-installyKblhI`
Caused by:
package `toml v0.8.6` cannot be built because it requires rustc 1.67 or newer, while the currently active rustc version is 1.66.1
For Others come by the same issue, maybe can be statet somewhere, you need to manually upgrade rsutc on your ubuntu.
First uninstall rustc apt remove rustc
then install the latest rust from rustup
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
For now, only IPv4 dot-decimal notation is supported.
It could be good to support IPv6 hexadecimal notation.
The following uses case.
$ cat large_log_file.log | tspin | less
Do not scroll till EOF in less, but quit at some point before. This results in crash due to broken pipe.
The application panicked (crashed).
Message: failed printing to stdout: Broken pipe (os error 32)
Location: library/std/src/io/stdio.rs:1019
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
The application panicked (crashed).
Message: Could not receive EOF signal from oneshot channel: RecvError(())
Location: /home/XXX/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tailspin-2.0.0/src/main.rs:48
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
This is probably an unconventional use case, but I noticed that feeding large (>10MB) file directly to tspin
gets slow, while in the above case it is pretty much instant.
tspin version 2.0.0
I have a feature request that would benefit my workflow.
I have a directory with a bunch of compressed log files in them, so I end up doing something like this:
gunzip some_info.log.gz
tspin some_info.log
It would be nice if there was an option to automatically decompress the file and store it in the same location without the .gz
extension, or in the case of a small file, just decompress it in memory so no extra file is created.
Not sure if this is something you would want to implement, but it would make viewing log files easier for me.
I tested tailspin on a 868M log file (one I routinely check on my system) and it took 2 minutes and 45 seconds to open in tailspin, pegging a core the entire time. This same file opens instantly with less.
I'd love to use tailspin when this is resolved!
Thanks for this project, it's great!
One unexpected behavior I've noticed (in 2.0.0) is:
$ rm -f empty.log
$ touch empty.log
$ tspin -p empty.log # hangs indefinitely
If I leave that hanging, and in a separate Zsh session run:
$ print -rl -- one two three >empty.log
then the first session with the hanging tspin
command prints
one
two
three
and exits.
I wouldn't expect tspin -p
to wait for any/more input -- is that intentional? I would expect instead for it to print nothing and exit immediately, in the case of an empty file.
Strings such as the following are not appropriately highlighted.
"Hello \"World\""
Steps to reproduce
echo "\"Hello \\\"Mars\\\"\"" | spin
would appreciate easy differentiation between INFO and DEBUG and etc.
Crashes on a simple file, due to alpine using the busybox version of less.
Installation was via cargo:
nightly-x86_64-unknown-linux-musl (default)
rustc 1.76.0-nightly (1e9dda77b 2023-11-22)
localhost:/var/log/nginx# tspin -f error.log
less: unrecognized option: ignore-case
BusyBox v1.36.1 (2023-11-06 11:32:24 UTC) multi-call binary.
Usage: less [-EFIMmNSRh~] [FILE]...
View FILE (or stdin) one screenful at a time
-E Quit once the end of a file is reached
-F Quit if entire file fits on first screen
-I Ignore case in all searches
-M,-m Display status line with line numbers
and percentage through the file
-N Prefix line number to each line
-S Truncate long lines
-R Remove color escape codes in input
-~ Suppress ~s displayed past EOF
The application panicked (crashed).
Message: called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "background task failed" }
Location: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tailspin-2.2.0/src/main.rs:61
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
With full backtrace:
localhost:/var/log/nginx# RUST_BACKTRACE=1 tspin -f error.log
less: unrecognized option: ignore-case
BusyBox v1.36.1 (2023-11-06 11:32:24 UTC) multi-call binary.
Usage: less [-EFIMmNSRh~] [FILE]...
View FILE (or stdin) one screenful at a time
-E Quit once the end of a file is reached
-F Quit if entire file fits on first screen
-I Ignore case in all searches
-M,-m Display status line with line numbers
and percentage through the file
-N Prefix line number to each line
-S Truncate long lines
-R Remove color escape codes in input
-~ Suppress ~s displayed past EOF
The application panicked (crashed).
Message: called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "background task failed" }
Location: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tailspin-2.2.0/src/main.rs:61
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ BACKTRACE โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โฎ 9 frames hidden โฎ
10: tspin::process_lines::{{closure}}::h989036fa9c27e244
at <unknown source file>:<unknown line>
11: tokio::runtime::task::core::Core<T,S>::poll::h0a8230ff4d14ff3a
at <unknown source file>:<unknown line>
12: tokio::runtime::task::raw::poll::hc3dd1119f6f12f69
at <unknown source file>:<unknown line>
13: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h5d0e9fc330ceb7c3
at <unknown source file>:<unknown line>
14: tokio::runtime::scheduler::multi_thread::worker::Context::run::h33a108d097c8f023
at <unknown source file>:<unknown line>
15: tokio::runtime::context::scoped::Scoped<T>::set::h44e407490c6ddb0e
at <unknown source file>:<unknown line>
16: tokio::runtime::context::runtime::enter_runtime::h453fa88dbaa438f8
at <unknown source file>:<unknown line>
17: tokio::runtime::scheduler::multi_thread::worker::run::h1b17132e983f4d16
at <unknown source file>:<unknown line>
18: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll::hf82f57bab33c7896
at <unknown source file>:<unknown line>
19: tokio::runtime::task::core::Core<T,S>::poll::haac949cf21fb3699
at <unknown source file>:<unknown line>
20: tokio::runtime::task::harness::Harness<T,S>::poll::h108e9b04e86acd01
at <unknown source file>:<unknown line>
21: tokio::runtime::blocking::pool::Inner::run::h8360b594cf7221b8
at <unknown source file>:<unknown line>
22: std::sys_common::backtrace::__rust_begin_short_backtrace::h3f48898d763cdb34
at <unknown source file>:<unknown line>
23: core::ops::function::FnOnce::call_once{{vtable.shim}}::hc50d5f34188abc7a
at <unknown source file>:<unknown line>
24: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc347b84c4e72a07c
at /rustc/1e9dda77b5b8e690c7e21871bbd2dcf182e1a841/library/alloc/src/boxed.rs:2007
25: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::he567d8d07a5ab38b
at /rustc/1e9dda77b5b8e690c7e21871bbd2dcf182e1a841/library/alloc/src/boxed.rs:2007
26: std::sys::unix::thread::Thread::new::thread_start::h6824dd6ac9aad219
at /rustc/1e9dda77b5b8e690c7e21871bbd2dcf182e1a841/library/std/src/sys/unix/thread.rs:108
Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Run with RUST_BACKTRACE=full to include source snippets.
Accessed file:
localhost:/var/log/nginx# cat access.log
10.0.1.26 - - [23/Nov/2023:10:24:31 +0100] "GET / HTTP/1.1" 404 146 "-" "insomnia/2023.5.8" "-"
10.0.1.26 - - [23/Nov/2023:10:24:59 +0100] "GET / HTTP/1.1" 404 146 "-" "insomnia/2023.5.8" "-"
10.0.1.26 - - [23/Nov/2023:10:25:10 +0100] "GET / HTTP/1.1" 404 146 "-" "insomnia/2023.5.8" "-"
10.0.1.26 - - [23/Nov/2023:10:25:55 +0100] "GET / HTTP/1.1" 404 146 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:29:16 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:29:43 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:30:00 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:30:46 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:32:04 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:33:15 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:33:48 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:33:55 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:34:23 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:34:53 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:35:42 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:36:13 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:36:53 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:38:23 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:38:40 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:39:07 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:39:36 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:40:01 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:40:52 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:42:27 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:43:10 +0100] "GET / HTTP/1.1" 502 150 "-" "curl/8.0.1" "-"
10.0.1.26 - - [23/Nov/2023:10:46:06 +0100] "GET / HTTP/1.1" 200 2716 "-" "curl/8.0.1" "-"
On my Debian system, running tailspin on a logfile warns that output "may be a binary file", and if I accept to see it anyway it shows the logfile with lots of escape codes where coloring was expected.
If I instead run tailspin with environment variable LESS="R" then it displays colors as expected.
I suspect your test environment is configured to have less parse escape sequences, which is not the default behaviour of less, as far as I can see in the less decoumentation.
I guess the proper thing to do is to add option -R to lines 7 and 9 of src/less/mod.rs, to ensure that ANSI escape code parsing is enabled regardless of environment setup.
On MacOS 13.1, using iTerm.
tailspin version 1.6.1
I use a black background in my terminal window and the blue in the time format was hard to read. I changed it to yellow
in my ~/.config/tailspin/config.toml
file as such:
...
[time]
time = { fg = "yellow" }
zone = { fg = "red" }
# To shorten the time, uncomment the line below
# shorten = { to = "โฃ", style = { fg = "blue" } }
Anyway, when I tried to view this log file, part of the date appears to be interpretted as a unix file path.
Also, I think the logic expects a .
after the time but instead is seeing a :
and interpretting it as something else.
$cat "com.flexibits.fantastical2.mac 2023-10-02--15-42-31-759.log"
2023/10/02 11:42:31:738 Location manager failed: Error Domain=kCLErrorDomain Code=0 "(null)"
2023/10/02 11:43:27:698 Location manager error: Error Domain=kCLErrorDomain Code=0 "(null)"
2023/10/02 11:47:39:850 Location manager failed: Error Domain=kCLErrorDomain Code=0 "(null)"
This is an incredible valuable project, thanks!
But to be honest, sometimes, I much prefer the behavior of a simple tail -f
, where I can "put" separator by hitting Enter
key.
Is it possible to add a mode where tailspin
or tailspin -f
completely skip the less
part?
I have debugged my setup for 2 hours because tailspin did not show my logs in piped mode. Only once I looked at the source code I realized, I have to set the bucket size to zero in order to show realtime log output.
I expected this to be the default when using tspin
in piped mode because with a bucket_size of 10,000, the output is delayed af.
Set the bucket size to zero by default when used in piped mode. This would prevent any future users from having the same problem. The tspin
process runs in parallel anyway when used in pipes, so the performance loss should be neglectable. If more performance at the cost of "up-to-dateness" is required, then one could just manually increase the bucket size.
It is safer to tell less that the path must not be accidentally parsed as an option, by using "--" as delimiter.
I.e. instead of "less +F somefile" it is safer to call it as "less +F -- somefile".
Recently I came across spacer, and it is the first time I've considered changing my approach to console based tailing..
https://github.com/samwho/spacer
This would be a really cool feature to have with tailspin! I may admit though, I think tailspin has won me over already...
keep up good work!
I recently ran tspin
on a system without less
installed, and I got this error:
lavender@pi:~$ tspin -f /opt/my/path/log_info.log
Failed to run less: No such file or directory (os error 2)
The application panicked (crashed).
Message: called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "background task failed" }
Location: /home/lavender/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tailspin-2.0.0/src/main.rs:59
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets
I think having a nicer error message would really help people understand what they need to do (install less) in order to get it working.
You might want to add a note to documentation that this is how to make tailspin your pager for systemd journalctl:
SYSTEMD_PAGERSECURE=true PAGER=/usr/bin/tailspin journalctl
...but you might wanna first implement #27 since currently your pager isn't secure.
Also, the above does not actually work - adding option -f
or -e
works but calling bare journalctl just hangs with nothing output to screen. :-(
Originally posted by seqizz December 29, 2023
Hi ๐
First of all, cool project. Tried on few places and liking it.
A small question is: Do you plan to support custom patterns? Something like:
[our_special_fqdn]
segment = { fg = "cyan", italic = true }
separator = { fg = "red" }
regexp = ' [a-z0-9-]+.[a-z0-9]+\.business\.local '
Here is an example I see on latest release:
As far as I understand, this is the "process" but can't catch udev-worker
due to parentheses.
Considering weird logs we see every day, I thought this might be a nice addition for ones who'd like to lose some sanity via regex and gain amazing customization powers.
less can do dangerous things, which are commonly unneeded for paging logfiles.
Please have tailspin set environment variable LESSSECURE=1 before calling less, to tighten security.
For now spin sometimes works nicely with grep:
$ echo 12345 | grep --color=always 34 | tspin
<magenta>12</magenta><red>34</red>5
$ echo '123"4"56' | grep --color=always '3"4' | tspin
<magenta>12</magenta><red>3</red><yellow>"4</yellow>"56
Not sure whether it's expected or not but I believe it might be helpful to have the following options:
Hi!
I'm struggling to disable the highlight group for Date
, i.e., hide the Date
column in log output
From README:
To disable a highlight group, either comment it out or delete it.
I've attempted:
# 1) comment out
#[groups.date]
# 2) set disabled = true
[groups.date]
disabled = true
It looks like I can set disabled on Date
and it should work? I may be missing something obvious here ๐
Lines 62 to 68 in 504e47f
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.