textualize / toolong Goto Github PK
View Code? Open in Web Editor NEWA terminal application to view, tail, merge, and search log files (plus JSONL).
Home Page: https://www.textualize.io/
License: MIT License
toolong's People
Contributors
Stargazers
Watchers
Forkers
koaning jg1vxg genzo1977 hill zmeir richardsonjf mbrukman alejandrosuarez mfaridi1394 jrcribb jenningsje dhirschfeld cryptoxunm jrycw singmoonshell rookie-man-fack lawweiliang wokuang hold7door mu-l amix134 cdwen0526 13851809588 tkyztshouhou saeedsamadyar gdbrianlu ewels kanonwy lstormsen 619561504 todayliao icecreammatt zhbjerry z872845991 lufeisimida ligho rictus moonxx123 o289697 mark2185 arun-nanduri tiandaji photon-platform supercoder-dev ccodingman onlypiglet zhnathaniellee xn-9527 maxlvcktoolong's Issues
Pressing `/` when the find panel is visible closes it
How to reproduce
Open the find panel with / (or CTRL+F). Press TAB to change the focus from the find input box to the "Case Sensitive" checkbox. Press / again.
Expected behavior
I kinda expect the input box to be focused again.
Actual behavior
The find panel is closed.
Screen corruption viewing Help
Great project, looks pretty neat, however when you visit the Help page, the screen gets corrupted. I'm viewing a Rails log file which uses terminal escape sequences itself to format the view, so I wonder if that is messing with the output from toolong?
Incompatible types for older Python versions
This is what I see when I install toolong in a virtualenv and immediately try to run it. I'm using Python 3.9
> tl
Traceback (most recent call last):
File "/Users/vincent/Development/arxiv-frontpage/venv/bin/tl", line 5, in <module>
from toolong.cli import run
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/cli.py", line 4, in <module>
from toolong.ui import UI
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/ui.py", line 13, in <module>
from toolong.log_view import LogView
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/log_view.py", line 22, in <module>
from toolong.messages import (
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/messages.py", line 6, in <module>
from toolong.log_file import LogFile
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/log_file.py", line 14, in <module>
from toolong.format_parser import FormatParser, ParseResult
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/format_parser.py", line 10, in <module>
from toolong import timestamps
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/timestamps.py", line 6, in <module>
class TimestampFormat(NamedTuple):
File "/Users/vincent/Development/arxiv-frontpage/venv/lib/python3.9/site-packages/toolong/timestamps.py", line 8, in TimestampFormat
parser: Callable[[str], datetime | None]
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'
Adding a from __future__ import annotations at the top of the aforementioned file seems to move the issue to another file, so I'm guess this is the fix? I think the | annotations actually only got introduced in 3.10 so to support 3.8 and 3.9 every file may need that import.
SIGBUS when viewing a file that is truncated while loading
Description
When opening a file that is then truncated, the program crashes with a SIGBUS.
Platform
MacOS 14.3
tl v1.0.0
How to reproduce
The easiest way is to create a large file with a bunch of lines, run tl bigfile.txt. Then, while tl is scanning for new lines, run truncate -s 0 bigfile.txt in another terminal.
Details
Using mmap() on a file the app doesn't own/control is problematic since changes to the file can mess up your process. It would probably be best to avoid mmap() and use regular file I/O instead.
While this may seem unlikely, it can happen in practice because of things like logrotate's copytruncate option which will copy the file contents and then truncate it.
Windows Support?
First off - thank you for this project and your others (we use Textual a lot!)
Having some issues using this in Windows, not sure if Windows is supposed to be supported or not.
Installation:
- pipx 1.4.3
- Windows 11
- Python 3.11.2
Issues so far:
- mmap.mmap call is a little different on Windows or atleast on my machine?
Had to changeprot=mmap.PROT_READtoaccess=mmap.ACCESS_READ- found this error here: mvt-project/mvt#61
Now I am getting - AttributeError: module 'os' has no attribute 'pread'
It is unclear to me how I can edit this to work on Windows because I am not confident I understand what you are doing here with os.preread
- Cheers
Feature Request: add ability to export or save merged log files
Context
toolong allows for merging multiple log files and sorting log entries in chronological order by auto detecting timestamps.
What is the problem
Merged log files sorted in chronological order can't be exported, saved, or copy/pasted out of toolong
Why is this a problem?
Very large, merged log files with many thousands or even millions of lines can be difficult to effectively parse or manage in the toolong TUI. In situations like this, it would be helpful to export or save the merged + sorted log file output so that one can use grep, jq, awk, sed and other tools for parsing the merged log file content.
Proposal
Add the ability to export or save merged log file data from toolong to a local text file.
Cannot follow with tail
My tl, installed via AUR on Arch, cannot seem to be able to tail files.
My test setup:
- on one terminal,
while true; do date >> test.log; sleep 1s; done - on another terminal,
tl test.log. It is stationary, even ifCTRL+Tis selected and theTAILindicator is present on the bottom right.
Any idea why?
text headers in log files should assume the timestamp of the first line with a timestamp when merging
Problem
When merging a file that starts with lines that do not have a timestamp, the lines show up at the bottom of the merged view (or not at all) instead of before the first line with a timestamp.
(For whatever reason, some log files do have textual headers.)
How to reproduce
Create two files with the following contents:
This is a header
2023-03-12T23:16:52.071 Hello, World1!
2023-03-12T23:16:53.071 Hello, World2!
This is another header
2023-03-12T23:16:58.071 Hello, World3!
2023-03-12T23:16:59.071 Hello, World4!
Load them into tl with merging turned on.
Expected
The header should be preserved and visible:
This is a header
2023-03-12T23:16:52.071 Hello, World1!
2023-03-12T23:16:53.071 Hello, World2!
This is another header
2023-03-12T23:16:58.071 Hello, World3!
2023-03-12T23:16:59.071 Hello, World4!
Actual
This is what I'm seeing now:
The header text isn't being displayed. Although, experimenting with other files, the header seemed to appear at the bottom of the view. Not sure what the difference is...
Feature Request: jump to next/previous match
Thank you for this awesome tool! We are now using with the whole team to read our logs, but there is one issue: when we are using the find tool it does not display the number of matches nor does it allow us to jump to next/previous matches - a feature that we use frequently and is available in other tools such as Lens.
It's usually necessary when reviewing long log files and we need to find an error message, but the only help toolong offers is highlighting the line which required the line to be in the window view, otherwise we have to scroll.
100% cpu usage, polling the file tl is looking at
Whenever I run tl, it uses 100% of a cpu. Running strace -fp shows three lines over and over, perhaps 100,000 times a second.
[pid 219465] lseek(7, 0, SEEK_CUR) = 423
[pid 219465] read(7, "", 65536) = 0
[pid 219465] poll([{fd=7, events=POLLIN}], 1, 100) = 1 ([{fd=7, revents=POLLIN}])
[pid 219465] lseek(7, 0, SEEK_CUR) = 423
[pid 219465] read(7, "", 65536) = 0
[pid 219465] poll([{fd=7, events=POLLIN}], 1, 100) = 1 ([{fd=7, revents=POLLIN}])
File descriptor 7 is the file I'm running tl on. 423 is the length of the file in bytes.
This is on Ubuntu 22.04.3. tl was installed via pipx, version 1.0.3
SInce efficiently watching a file for changes is a feature, something seems broken.
It doesn't work on windows
This app doesn't work on Windows, this is a big issue.
a number in a multiline log message is mistaken for a timestamp
Problem
A number in a log message can be mistaken for a timestamp, which will mess up log merging behavior.
How to reproduce
Load the following log snippet into tl:
Feb 08 09:50:05 Foo-MacBook-Air loginwindow[588]: IASGetCurrentInstallPhaseList: phases = (
{
ConclusionDelay = 0;
InstallPhase = "IOKit Boot";
InstallPhaseActualPercentageKey = "21.77734375";
InstallPhasePercentageKey = 22;
},
{
ConclusionDelay = 0;
InstallPhase = "Boot-Time Installation";
InstallPhaseActualPercentageKey = "69.01998901367188";
InstallPhasePercentageKey = 69;
}
)
Select the line with "69.01998901367188" and note that the time in the bottom right is something like "04/07/8304 22:58:38". This causes problems when merging log files because these lines show up at the bottom of the merged logs.
Details
Since the TimestampScanner is looking for timestamps anywhere in a log line, a format like the following one is going to mistakenly pick up logged data and treat it as a timestamp:
https://github.com/Textualize/toolong/blob/9fe54a6bcd6bcc2b1066c762057b0de812dcd155/src/toolong/timestamps.py#L92-L93
Add Support for EVTX Files
Add Support for EVTX Files
Feature Request: Implement EVTX file support in toolong for importing, and reading Windows Event Viewer logs.
Why This Matters:
- Relevance: EVTX is a crucial format for Windows event logs, widely used in IT security and troubleshooting.
- Benefit: Facilitates direct log analysis within
toolong, enhancing its utility for Windows system users.
Thank you for considering this enhancement.
Request for ability to wrap lines or faster sidways scroll
Thanks for making this. I have been searching for something like this for ages.
The only issue is that you need to scroll with the arrow keys to see long lines in the right-hand window. It would be nice if there was an option to wrap long lines.
If that is too difficult, could there be a faster sideways scroll; something akin to pageup/pagedown but for sideways scrolling?
Incorrect syslog timestamp
Description
When viewing a syslog file, the timestamp reports as the year 1900.
How to reproduce
Open a syslog file that has lines like:
Feb 11 20:44:26 Foo-MacBook-Air AMPDeviceDiscoveryAgent[1574]: Entered:__thr_AMMuxedDeviceDisconnected, mux-device:682
Feb 11 20:45:07 Foo-MacBook-Air AMPDeviceDiscoveryAgent[1574]: Entered:_AMMuxedDeviceDisconnected, mux-device:683
Feb 11 20:45:07 Foo-MacBook-Air AMPDeviceDiscoveryAgent[1574]: Entered:__thr_AMMuxedDeviceDisconnected, mux-device:683
Check the time reported in the bottom right, it will have "02/11/1900".
Details
For timestamps that are missing components, it's probably best to use the file's timestamp as a basis. Note that you will also need to keep track of differences from the previous timestamp in order to detect a rollover. For example, if the previous timestamp was "Dec 31" and the next was "Jan 01", you'll need to go back through the previously scanned dates to subtract a year to get the correct timestamp.
feature request: fold / unfold sections of json files
Would be great to be able to fold / unfold sections of json files in similar way as vscode does it. Here's an example of folded section at line 2:
Slow search?
Hi, I really appreciate efforts to create better tools for investigating logs!
But, the search is very slow on large log files? grep processes a 2M lines long 400MB log file in ~250ms while toolong spends too long time to measure to completion :/
Very high CPU usage when piping
Hi! First time user, I just found this (pretty cool) tool and started using it right away.
I noticed a high CPU usage, which I think might be related to #17.
I'm using dpkg.log here as an example, but happen with other files too. When I do tl /var/log/dpkg.log, I have very low CPU usage and I can see the app normally, like this
However, if I do cat /var/log/dpkg.log | tl, I get a very VERY high cpu usage
Sorry the command is not fully displayed, but it definetly running tl here.
Also, I have to double ctrl+c to exit, which throws this error
Feature request: display format when viewing JSON logs
In my opinion, JSON logs are not meant to be read as such by human eye, as one line can carry a lot of information. To really be useful in my workflow, I would need to be able to display a formatted version of each log instead of the raw JSON, while still being able to look at the whole object when highlighted.
The display format would be something like a string with placeholders, being filled by the data in the logs.
For example, let's say all my logs have at least a field called "mesage", which contain a human readable line, and another called "level". I would press a shortcut like "ctrl-D" (display), a one-line window would appear somewhere where I would type "$level $message", press enter, and all my logs would be seen in this format. Highlighting the desired log would allow me to see the whole object.
This feature would make the tool my default for analysing logs, as this is already my workflow, that I use with home-made CLI tools.
tab title does not change when file is renamed
Description
When a file is renamed, like when a file is rolled, the tab title does not update.
How to reproduce
Open multiple files so there are tabs and then rename one of the files. Note that the tab title does not change.
Details
If the app is not going to follow files by name and continue to view ones that have been renamed/deleted, it should probably update the tab title accordingly. (Note that keeping open a file that has been deleted will prevent the disk space from being reclaimed.)
ImportError on Ubuntu 20.04
Hello,
I want to report the following behavior when installing toolong.
OS Info:
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"Problem:
I installed toolong using the command pip3 install toolong and got the output:
ERROR: frogmouth 0.9.2 has requirement textual<0.44.0,>=0.43.0, but you'll have textual 0.50.0 which is incompatible.
Installing collected packages: python-dateutil, textual, toolong
Attempting uninstall: textual
Found existing installation: textual 0.43.2
Uninstalling textual-0.43.2:
Successfully uninstalled textual-0.43.2
Successfully installed python-dateutil-2.8.2 textual-0.50.0 toolong-1.0.0When I use the command tl I get the following error:
Traceback (most recent call last):
File "/home/wjtd8139/.local/bin/tl", line 5, in <module>
from toolong.cli import run
File "/home/wjtd8139/.local/lib/python3.8/site-packages/toolong/cli.py", line 4, in <module>
from toolong.ui import UI
File "/home/wjtd8139/.local/lib/python3.8/site-packages/toolong/ui.py", line 13, in <module>
from toolong.log_view import LogView
File "/home/wjtd8139/.local/lib/python3.8/site-packages/toolong/log_view.py", line 22, in <module>
from toolong.messages import (
File "/home/wjtd8139/.local/lib/python3.8/site-packages/toolong/messages.py", line 6, in <module>
from toolong.log_file import LogFile
File "/home/wjtd8139/.local/lib/python3.8/site-packages/toolong/log_file.py", line 14, in <module>
from toolong.format_parser import FormatParser, ParseResult
File "/home/wjtd8139/.local/lib/python3.8/site-packages/toolong/format_parser.py", line 4, in <module>
from typing import TypeAlias
ImportError: cannot import name 'TypeAlias' from 'typing' (/usr/lib/python3.8/typing.py)Copying out of tl
I could see tl being super helpful but there's one thing missing that means I cannot use it for real.
Often the end result of searching for something in the logs is to copy the information out (share with coworker, create jira, etc)
It seems like there's no current way to do that?
Double clicking on a log line opens the detail.
Trying to click-drag select in detail does noting.
Ctrl-C does nothing.
If Ctrl-C could copy the current line that would be good enough but I would love to be able to copy all lines found by selection. And being able to click-drag in the detail to select part of the json would be nice.
Fix blank canvas when reading file with a very long line
I have a unformatted JSON file that is one very long line. toolong either gives me a blank canvas or the entire screen is blank.
I understand that toolong is not intended to be used with files like this—you should probably use less. I'm asking for a more informative screen that tells me toolong can't handle it, so I can try something else (e.g., cat long.json | jq | toolong)
[Enh request] Filter
Please add a filter that lets you enter a string and only lines with (or without) that string are shown.
AttributeError: 'DOMQuery' object has no attribute 'set'
This is pretty much everything I have to report:
❯ tl /media/sda3/Books/log_162.txt
╭───────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────────────────────────────────────────────╮
│ /usr/lib/python3.11/site-packages/toolong/ui.py:68 in on_mount │
│ │
│ 65 │ ╭────── locals ──────╮ │
│ 66 │ def on_mount(self) -> None: │ self = LogScreen() │ │
│ 67 │ │ assert isinstance(self.app, UI) ╰────────────────────╯ │
│ ❱ 68 │ │ self.query("TabbedContent Tabs").set(display=len(self.query(TabPane)) > 1) │
│ 69 │ │ active_pane = self.query_one(TabbedContent).active_pane │
│ 70 │ │ if active_pane is not None: │
│ 71 │ │ │ active_pane.query("LogView > LogLines").focus() │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'DOMQuery' object has no attribute 'set'
My OS: Linux kodi-nnue 6.6.10-1-MANJARO #1 SMP PREEMPT_DYNAMIC Fri Jan 5 17:38:36 UTC 2024 x86_64 GNU/Linux
Python: 3.11.6
ha ha ha... YES!!!
Thank you for making this. This is going to substantially improve my quality of life. Textual looks really cool too.
Request ability to interpret new line characters in the right panel
Thank you for developing toolong; it has proven to be incredibly useful.
I've noticed that my logs sometimes contain numerous newline characters (\n), particularly evident when outputting results from commands like nvidia-smi. When I click on a logline, the right panel opens, which is a convenient feature for viewing the entirety of the log with a vertical scrollbar. However, since newline characters are not interpreted, the content can become difficult to read depending on its structure.
Could we consider adding the interpretation of newline characters ? Maybe optionally? This would enhance the readability of the logs for users dealing with similar formatting issues.
I am keen to contribute if necessary
/usr/lib/python3.11/site-packages/toolong/ui.py:68 in on_mount
Thanks guys,
this nice project.
I use manjaro and install your application by pamac and when I run for example
tl /var/log/boot.log
I see something like this and I think it does not work
Add Less-like jumping shortcuts
See less --help
I've no idea what are the most popular ones people use. For me the one I cannot do without is g and G for jumping to first and last line of file (respectively)
TypeError: '<' not supported between instances of 'LogFile' and 'LogFile'
While merging a directory with ros log files toolong crashes with the following error:
TypeError: '<' not supported between instances of 'LogFile' and 'LogFile'
Traceback
❯ tl *.log --merge
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ /home/smartrobotics/.local/pipx/venvs/toolong/lib/python3.8/site-packages/textual/worker.py:365 in _run │
│ │
│ 362 │ │ self.state = WorkerState.RUNNING ╭──────────────────────────────────────── locals ─────────────────────────────────────────╮ │
│ 363 │ │ app.log.worker(self) │ app = UI(title='UI', classes={'-dark-mode'}) │ │
│ 364 │ │ try: │ error = TypeError("'<' not supported between instances of 'LogFile' and 'LogFile'") │ │
│ ❱ 365 │ │ │ self._result = await self.run() │ self = <Worker ERROR name='run_scan' description='run_scan()'> │ │
│ 366 │ │ except asyncio.CancelledError as error: │ Traceback = <class 'rich.traceback.Traceback'> │ │
│ 367 │ │ │ self.state = WorkerState.CANCELLED ╰─────────────────────────────────────────────────────────────────────────────────────────╯ │
│ 368 │ │ │ self._error = error │
│ │
│ /home/smartrobotics/.local/pipx/venvs/toolong/lib/python3.8/site-packages/textual/worker.py:349 in run │
│ │
│ 346 │ │ Returns: ╭──────────────────────────── locals ────────────────────────────╮ │
│ 347 │ │ │ Return value of the work. │ self = <Worker ERROR name='run_scan' description='run_scan()'> │ │
│ 348 │ │ """ ╰────────────────────────────────────────────────────────────────╯ │
│ ❱ 349 │ │ return await ( │
│ 350 │ │ │ self._run_threaded() if self._thread_worker else self._run_async() │
│ 351 │ │ ) │
│ 352 │
│ │
│ /home/smartrobotics/.local/pipx/venvs/toolong/lib/python3.8/site-packages/textual/worker.py:319 in _run_threaded │
│ │
│ 316 │ │ else: ╭───────────────────────────────────────── locals ─────────────────────────────────────────╮ │
│ 317 │ │ │ raise WorkerError("Unsupported attempt to run a thread worker") │ run_awaitable = <function Worker._run_threaded.<locals>.run_awaitable at 0x7f706e6d58b0> │ │
│ 318 │ │ │ run_callable = <function Worker._run_threaded.<locals>.run_callable at 0x7f706e6d59d0> │ │
│ ❱ 319 │ │ return await asyncio.get_running_loop().run_in_executor( │ run_coroutine = <function Worker._run_threaded.<locals>.run_coroutine at 0x7f706e6d5940> │ │
│ 320 │ │ │ None, runner, self._work │ runner = <function Worker._run_threaded.<locals>.run_callable at 0x7f706e6d59d0> │ │
│ 321 │ │ ) │ self = <Worker ERROR name='run_scan' description='run_scan()'> │ │
│ 322 ╰──────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/lib/python3.8/concurrent/futures/thread.py:57 in run │
│ │
│ 54 │ │ │ return ╭── locals ───╮ │
│ 55 │ │ │ self = None │ │
│ 56 │ │ try: ╰─────────────╯ │
│ ❱ 57 │ │ │ result = self.fn(*self.args, **self.kwargs) │
│ 58 │ │ except BaseException as exc: │
│ 59 │ │ │ self.future.set_exception(exc) │
│ 60 │ │ │ # Break a reference cycle with the exception 'exc' │
│ │
│ /home/smartrobotics/.local/pipx/venvs/toolong/lib/python3.8/site-packages/textual/worker.py:304 in run_callable │
│ │
│ 301 │ │ def run_callable(work: Callable[[], ResultType]) -> ResultType: ╭─────────────────────────────────────── locals ───────────────────────────────────────╮ │
│ 302 │ │ │ """Set the active worker, and call the callable.""" │ self = <Worker ERROR name='run_scan' description='run_scan()'> │ │
│ 303 │ │ │ active_worker.set(self) │ work = functools.partial(<function LogLines.run_scan at 0x7f706fbf3160>, LogLines()) │ │
│ ❱ 304 │ │ │ return work() ╰──────────────────────────────────────────────────────────────────────────────────────╯ │
│ 305 │ │ │
│ 306 │ │ if ( │
│ 307 │ │ │ inspect.iscoroutinefunction(self._work) │
│ │
│ /home/smartrobotics/.local/pipx/venvs/toolong/lib/python3.8/site-packages/toolong/log_lines.py:292 in run_scan │
│ │
│ 289 │ │ worker = get_current_worker() ╭───────────────────────────── locals ─────────────────────────────╮ │
│ 290 │ │ │ self = LogLines() │ │
│ 291 │ │ if len(self.log_files) > 1: │ worker = <Worker ERROR name='run_scan' description='run_scan()'> │ │
│ ❱ 292 │ │ │ self.merge_log_files() ╰──────────────────────────────────────────────────────────────────╯ │
│ 293 │ │ │ return │
│ 294 │ │ │
│ 295 │ │ try: │
│ │
│ /home/smartrobotics/.local/pipx/venvs/toolong/lib/python3.8/site-packages/toolong/log_lines.py:374 in merge_log_files │
│ │
│ 371 │ │ │ ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── locals ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ 372 │ │ │ position += log_file.size │ append = <built-in method append of list object at 0x7f706c822340> │ │
│ 373 │ │ │ append_meta = <built-in method append of list object at 0x7f706e6fb7c0> │ │
│ ❱ 374 │ │ merge_lines.sort() │ break_position = 152454 │ │
│ 375 │ │ │ line_no = 1430 │ │
│ 376 │ │ self.post_message(ScanComplete(total_size, total_size)) │ log_file = <LogFile 'sr_signal_lamp-12.log' size=152523> │ │
│ 377 │ merge_lines = [ │ │
│ │ │ (1707986182.531916, 11, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.611236, 12, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.611554, 13, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.613183, 14, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.614288, 15, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.615322, 16, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.61651, 17, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.617511, 18, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.618456, 19, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ (1707986182.61943, 20, <LogFile 'bag_rotator-24-stdout.log' size=520650>), │ │
│ │ │ ... +541332 │ │
│ │ ] │ │
│ │ position = 122834144 │ │
│ │ self = LogLines() │ │
│ │ timestamp = 1707994955.0 │ │
│ │ timestamps = [(0, 0, 1707986181.0), (1, 92, 1707986181.0), (2, 168, 1707986181.0), (3, 257, 1707986181.0), (4, 342, 1707986181.0), (5, 430, 1707986181.0), (6, 534, 1707986181.0), (7, 612, 1707986181.0), (8, 679, 1707986181.0), (9, 757, 1707986181.0), ... +1421] │ │
│ │ total_size = 122834144 │ │
│ │ worker = <Worker ERROR name='run_scan' description='run_scan()'> │ │
│ ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: '<' not supported between instances of 'LogFile' and 'LogFile'
File names that start with a number cause a crash
Problem
File names that start with a number seem to cause a problem with the file name sorting done for the UI.
How to reproduce
Create a file that starts with a number, like 1.txt. Then, run tl with that file and another one:
tl 1.txt /var/log/system.log
I see a traceback that ends with this:
File "/opt/homebrew/lib/python3.11/site-packages/toolong/ui.py", line 91, in __init__
self.file_paths = self.sort_paths(file_paths)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/toolong/ui.py", line 88, in sort_paths
return sorted(paths, key=key)
^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'
Details
It looks like the following code is trying to do a natural sort, but doesn't handle all the cases well:
Lines 82 to 86 in 9fe54a6
PermissionError on WSL2
Howdy,
I am opening this issue in case other people run into this, and will investigate more later.
I just installed toolong on Windows + WSL2 (Fedora) via pipx on Python3.11 (default Fedora Python).
Running tl on any file gives me a PermissionError:
╭─────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────╮
│ /home/aprengere/.local/pipx/venvs/toolong/lib64/python3.11/site-packages/toolong/log_lines.py:905 in on_scan_complete │
│ │
│ 902 │ │ self.update_line_count() │
│ 903 │ │ self.refresh() │
│ 904 │ │ if len(self.log_files) == 1 and self.can_tail: │
│ ❱ 905 │ │ │ self.start_tail() │
│ 906 │ │
│ 907 │ @on(ScanProgress) │
│ 908 │ def on_scan_progress(self, event: ScanProgress): │
│ │
│ ╭──────── locals ────────╮ │
│ │ event = ScanComplete() │ │
│ │ self = LogLines() │ │
│ ╰────────────────────────╯ │
│ │
│ /home/aprengere/.local/pipx/venvs/toolong/lib64/python3.11/site-packages/toolong/log_lines.py:281 in start_tail │
│ │
│ 278 │ │ │ """Callback when there is an error watching the file.""" │
│ 279 │ │ │ self.post_message(FileError(error)) │
│ 280 │ │ │
│ ❱ 281 │ │ self.watcher.add( │
│ 282 │ │ │ self.log_file, │
│ 283 │ │ │ size_changed, │
│ 284 │ │ │ watch_error, │
│ │
│ ╭─────────────────────────────────────── locals ────────────────────────────────────────╮ │
│ │ self = LogLines() │ │
│ │ size_changed = <function LogLines.start_tail.<locals>.size_changed at 0x7f58cb657a60> │ │
│ │ watch_error = <function LogLines.start_tail.<locals>.watch_error at 0x7f58cb657880> │ │
│ ╰───────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/aprengere/.local/pipx/venvs/toolong/lib64/python3.11/site-packages/toolong/watcher.py:70 in add │
│ │
│ 67 │ │ size = log_file.size │
│ 68 │ │ self._file_descriptors[fileno] = WatchedFile(log_file, callback, error_callback) │
│ 69 │ │ os.lseek(fileno, size, os.SEEK_SET) │
│ ❱ 70 │ │ self._selector.register(fileno, EVENT_READ) │
│ 71 │ │
│ 72 │ def run(self) -> None: │
│ 73 │ │ """Thread runner.""" │
│ │
│ ╭──────────────────────────────────────── locals ─────────────────────────────────────────╮ │
│ │ callback = <function LogLines.start_tail.<locals>.size_changed at 0x7f58cb657a60> │ │
│ │ error_callback = <function LogLines.start_tail.<locals>.watch_error at 0x7f58cb657880> │ │
│ │ fileno = 8 │ │
│ │ log_file = <LogFile 'toto' size=0> │ │
│ │ self = <Watcher(Thread-1, started 140019327899328)> │ │
│ │ size = 0 │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/lib64/python3.11/selectors.py:359 in register │
│ │
│ 356 │ │ if events & EVENT_WRITE: │
│ 357 │ │ │ poller_events |= self._EVENT_WRITE │
│ 358 │ │ try: │
│ ❱ 359 │ │ │ self._selector.register(key.fd, poller_events) │
│ 360 │ │ except: │
│ 361 │ │ │ super().unregister(fileobj) │
│ 362 │ │ │ raise │
│ │
│ ╭────────────────────────────── locals ──────────────────────────────╮ │
│ │ data = None │ │
│ │ events = 1 │ │
│ │ fileobj = 8 │ │
│ │ key = SelectorKey(fileobj=8, fd=8, events=1, data=None) │ │
│ │ poller_events = 1 │ │
│ │ self = <selectors.EpollSelector object at 0x7f58cb5ee6d0> │ │
│ ╰────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
PermissionError: [Errno 1] Operation not permitted
Add help screen without needing to open a file
Help screen in TUI and tl --help are very different. A lot of information in the TUI help screen would be more useful also in tl --help, like examples.
Using an invalid regex in the find panel causes a crash
Problem
Entering an invalid regular expression in the find panel will cause a crash.
How to reproduce
- Start
tlwith a file - Press
/to open the find dialog - Enter
* - Turn on the
Regexcheckbox - Press the the down arrow key
Expected
An error message should be shown and maybe the invalid part of the regex should be highlighted
Actual
tl crashes with a traceback and error: nothing to repeat at position 0
FileNotFoundError: [Errno 2] No such file or directory: '/dev/tty'
I am unable to redirect output of commands to tl via a pipe in Windows 11 (powershell or cmd)
tree | tl
File "\site-packages\toolong\cli.py", line 55, in run
with open("/dev/tty", "rb", buffering=0) as tty_stdin:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/dev/tty'
...
I know windows support has been recently added. I didn't see any open/closed issues regarding this error.
FWIW - I am able to pipe commands to toolong using wsl in a linux distro just fine.
Thanks!
Support for the parquet file format
I am not sure if this is within the scope of the project but I thought that the UI of toolong would be really nice to quickly have a look at the contents of parquet files.
Arguably the parquet file format is quite often used to archive structured event data (e.g. in "data lakes") but I am not sure that would qualify as a proper log file in the usual sense (in particular, it is not very well suited for small append-only updates).
Pipe Support
Hello, toolong looks very nice in my quick testing, thanks for making it!
The mentions of tail and less in the readme got me excited to try to use it as a pager for journalctl but that doesn't seem to work. In fact piping any text to it doesn't seem to work.
if I set:
export SYSTEMD_PAGER=/home/me/.local/bin/tl
and then run journalctl, I get the tl help output
also if I just pipe text from journalctl or cat I get the tl help output.
Thanks for reading!
using a non-existent file name with `-m` crashes with a traceback
Problem
Running tl in merge mode with a non-existent file path crashes with a traceback due to a FileNotFoundError
How to reproduce
Run tl like so:
tl -m badname1 badname2
Expected
When not in merge-mode, error notifications are popped up, so I would expect the same behavior.
feature request: jump to line number
when in 'show line numbers mode' would be great to be able to jump to a line number similar like vim.
Error installing Toolong with pipx
OS: Linux Debian 11.8 armv7l
Python: 3.9.2
Pipx: 0.12.3.1
Command: pipx install toolong
Error:
Traceback (most recent call last):
File "/usr/bin/pipx", line 11, in <module>
load_entry_point('pipx==0.12.3.1', 'console_scripts', 'pipx')()
File "/usr/lib/python3/dist-packages/pipx/main.py", line 496, in cli
exit(run_pipx_command(parsed_pipx_args, binary_args))
File "/usr/lib/python3/dist-packages/pipx/main.py", line 147, in run_pipx_command
commands.install(
File "/usr/lib/python3/dist-packages/pipx/commands.py", line 312, in install
if venv.get_venv_metadata_for_package(package).package_version is None:
File "/usr/lib/python3/dist-packages/pipx/Venv.py", line 65, in get_venv_metadata_for_package
data = json.loads(
File "/usr/lib/python3.9/json/__init__.py", line 359, in loads
return cls(**kw).decode(s)
TypeError: __init__() got an unexpected keyword argument 'encoding'Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.