Comments (12)
Hey @pizzafox, sorry for this experience and thanks for reporting this. Mind helping me out with some questions to debug this?
Does this happen for every folder you scan? Does the behaviour change if you change the terminal window size? (as in, does it crash earlier, later or not at all with a smaller terminal window?)
In the folder you scan, do you happen to have folder/file names with non-latin characters?
from diskonaut.
Did you notice the reported sizes? One of the last figures seen is 11.6 EB - not far off overflowing a u64.
from diskonaut.
This is about 1 TB, definitely not that big. I assume Diskonaut is getting screwed up with a weird Windows FS thing.
from diskonaut.
Thanks for pointing it out, @Freaky. I indeed hadn't noticed.
@pizzafox - I suspect you're right and that since we're trying to read hard-disk blocks, something in the WSL is throwing us off (and causing us to report huge file sizes which are clearly incorrect... this might be what's causing the crash in the end as @Freaky pointed out). But this is of course just a guess.
I think when we implement #56, it might solve this issue. I hope to get to it soon (unless someone does it before me), and will give you a ping once it's done and released so you can give it a shot if you want to.
from diskonaut.
I can reproduce the crash by adding 2^62 to file sizes:
thread 'main' panicked at 'Trying to access position outside the buffer: x=144, y=20, area=Rect { x: 0, y: 0, width: 106, height: 44 }', /home/freaky/.cargo/registry/src/github.com-1ecc6299db9ec823/tui-0.9.5/src/buffer.rs:245:9
Sadly backtraces are currently broken on FreeBSD, so here's one from gdb:
#27 0x000000000148d1dd in std::panicking::begin_panic_fmt () at src/libstd/panicking.rs:385
#28 0x000000000143c756 in tui::buffer::Buffer::index_of (self=0x7fffffffdad0, x=144, y=20) at /home/freaky/.cargo/registry/src/github.com-1ecc6299db9ec823/tui-0.9.5/src/buffer.rs:245
#29 0x000000000119d64b in tui::buffer::Buffer::set_stringn (self=0x7fffffffdad0, x=144, y=20, string=..., width=18446744073709551615, style=...)
at /home/freaky/.cargo/registry/src/github.com-1ecc6299db9ec823/tui-0.9.5/src/buffer.rs:319
#30 0x000000000119d576 in tui::buffer::Buffer::set_string (self=0x7fffffffdad0, x=144, y=20, string=..., style=...) at /home/freaky/.cargo/registry/src/github.com-1ecc6299db9ec823/tui-0.9.5/src/buffer.rs:303
#31 0x0000000001216ccb in diskonaut::ui::grid::draw_rect::draw_tile_text_on_grid (buf=0x7fffffffdad0, tile=0x806e5f480, selected=false) at src/ui/grid/draw_rect.rs:197
#32 0x000000000124c5a1 in <diskonaut::ui::grid::rectangle_grid::RectangleGrid as tui::widgets::Widget>::render (self=..., area=..., buf=0x7fffffffdad0) at src/ui/grid/rectangle_grid.rs:74
#33 0x000000000123c152 in tui::terminal::Frame<B>::render_widget (self=0x7fffffffbd68, widget=..., area=...) at /home/freaky/.cargo/registry/src/github.com-1ecc6299db9ec823/tui-0.9.5/src/terminal.rs:65
#34 0x0000000001259acc in diskonaut::ui::display::Display<B>::render::{{closure}} (f=...) at src/ui/display.rs:128
#35 0x000000000123cbcf in tui::terminal::Terminal<B>::draw (self=0x7fffffffda98, f=...) at /home/freaky/.cargo/registry/src/github.com-1ecc6299db9ec823/tui-0.9.5/src/terminal.rs:187
#36 0x0000000001259214 in diskonaut::ui::display::Display<B>::render (self=0x7fffffffda98, file_tree=0x7fffffffd9f8, board=0x7fffffffd990, ui_mode=0x7fffffffd938, ui_effects=0x7fffffffdb30)
at src/ui/display.rs:48
#37 0x0000000001210211 in diskonaut::app::App<B>::render (self=0x7fffffffd938) at src/app.rs:81
#38 0x00000000012100ed in diskonaut::app::App<B>::render_and_update_board (self=0x7fffffffd938) at src/app.rs:71
#39 0x00000000012103a0 in diskonaut::app::App<B>::start_ui (self=0x7fffffffd938) at src/app.rs:103
#40 0x0000000001261627 in diskonaut::messages::instruction::handle_instructions (app=0x7fffffffd938, receiver=...) at src/messages/instruction.rs:53
#41 0x000000000121007d in diskonaut::app::App<B>::start (self=0x7fffffffd938, receiver=...) at src/app.rs:65
#42 0x000000000123f7d7 in diskonaut::start (terminal_backend=..., keyboard_events=..., path=...) at src/main.rs:226
#43 0x0000000001221aff in diskonaut::try_main () at src/main.rs:74
#44 0x0000000001221490 in diskonaut::main () at src/main.rs:55
Really, it's not our fault if Windows is reporting weird sizes, but it is that we're overflowing. If anything, --apparent-size
could make this more likely, because it's a lot easier to make a file that's apparently gobsmackingly massive than it is to make one physically massive.
Sizes should probably be summed using checked maths (checked_add
, saturating_add
, etc), and/or perhaps the sums should be u128.
from diskonaut.
I see. Using checked_add
/saturating_add
makes sense. Probably combined with handling an overflow with some sort of descriptive error message. Moving stuff to u128 is something I'll have to look into. Mostly the implications of the memory footprint, since we're going to have to change each tree node. I'm assigning this to myself and will take a closer look soon.
from diskonaut.
Actually... does it even make sense to move stuff to u128? The max value of a u64 is 18446744073709551615. That's ~18 million terabytes. Is this even relevant to storage one would want to run diskonaut on? (please do correct me if my math is wrong :) )
from diskonaut.
It's perfectly valid to have huge files. I just made my diskonaut folder ~37EB:
.rw-r--r-- 9.2E freaky freaky 29 Jun 21:48 huge1
.rw-r--r-- 9.2E freaky freaky 29 Jun 21:49 huge2
.rw-r--r-- 9.2E freaky freaky 29 Jun 21:49 huge3
.rw-r--r-- 9.2E freaky freaky 29 Jun 21:49 huge4
It'd be nice to be able to Just Work, which is more than I can say for... any other du-alike I've tried. GNU du gives up and just says Infinity
, BSD du
and other tools just overflow and give meaningless results.
from diskonaut.
Alright. So let's try with u128 and see if it doesn't take up too big of a memory footprint. Would you like to work on this, @Freaky ?
from diskonaut.
Hey @pizzafox, in 0.9.0
there's an --apparent-size
flag that shows the actual file sizes rather than counting how many blocks they take on disk. I wonder if running with this flag still causes this issue?
from diskonaut.
The --apparent-size
flag fixed the issue.
Using v0.9.0
without the flag no longer crashes (at least, not after 45 seconds). The total size got up to 100 zettabytes before I quit the program. Perhaps Diskonaut should suggest using the --apparent-size
flag when the size gets extraordinarily large?
from diskonaut.
Thanks @pizzafox! The not crashing is thanks to @Freaky's work, I'd say.
I think somehow suggesting using the flag is a good idea. I'll open an issue in the next few days with some thoughts.
from diskonaut.
Related Issues (20)
- Bug: When deleting files or entering folders with multibyte characters, the app might crash
- Feature: implement `--apparent-size` HOT 1
- Bug: Small files hidden from view HOT 7
- Bug: Runaway memory usage when run on large directories HOT 9
- Option to delete without confiration? HOT 2
- failure="0.1" not available, the current version is 0.1.8 HOT 4
- An option not to cross filesystems HOT 1
- most dirs are missing HOT 4
- Feature Request: Give an option to exclude directory HOT 1
- offer prebuild deb files in a ppa
- Update jwalk dependency to 0.6
- aarch64 and armhf doest pass tests ?
- The folder takes up too much space (7091231513.3G) HOT 2
- Feature Request: Ability to not cross filesystem boundaries. HOT 2
- Graphics break down when running over SSH to Windows HOT 1
- Feature: 'explode' a directory
- Numbers aren't adding up
- Feature: option `--ignore` to skip certain directory/directories HOT 2
- OneDrive on-demand - files showing full size
- Cannot delete anything on Debian Linux
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diskonaut.