johnnydouble / filemango Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 1.0 5.52 MB

License: Mozilla Public License 2.0

Go 88.66% TeX 11.34%

filemango's People

Contributors

Stargazers

Watchers

Forkers

prologic

filemango's Issues

Empty file queue causes index out of range error.

If the file queue is empty, calling managePool() will produce the following error.

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
FileMango/src/scheduler.managePool()
        /home/melkor/go/src/FileMango/src/scheduler/worker_pool.go:52 +0x2a5

implement batching on file discovery

Send multiple files to the usr/bin/file command at once to reduce startup time.

QueueFile function should ensure that it disallows directories from being Queued

Queuing directories may cause problems later in the processing pipeline or at the least will create more work for providers later on.

Handle system calls related to stopping execution.

System calls for exiting gracefully should be handled so that the program can be stopped as safely as possible.

On second run db.Fold function causes a segmentation violation

works fine on first run, after a second run without deleting the 000000000.data file gives this error:

loading config... done
loading database... done
watching...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x51c102]

goroutine 1 [running]:
github.com/prologic/bitcask.(*Bitcask).Fold(0x0, 0x617938, 0xc000000180, 0x5a85f5)
        /home/melkor/go/pkg/mod/github.com/prologic/[email protected]/bitcask.go:304 +0x62
FileMango/src/db.FoldQueue(...)
        /home/melkor/go/src/FileMango/src/db/database.go:19
FileMango/src/scheduler.addInitialJobs()
        /home/melkor/go/src/FileMango/src/scheduler/worker_pool.go:147 +0x39
FileMango/src/scheduler.RunAnalysis()
        /home/melkor/go/src/FileMango/src/scheduler/worker_pool.go:28 +0x3a
main.run()
        /home/melkor/go/src/FileMango/main.go:37 +0x356
main.main()
        /home/melkor/go/src/FileMango/main.go:17 +0x25

Process finished with exit code 2

if the 000000000.data file is deleted works fine again.

Ensure only a single instance runs at a time

if args are given run the requested thing,
else if no other instance is already running run the program normally
else end this instance

Worker pool

Write a worker pattern using goroutines to dynamically pool providers.

Corrupted file queue leading to null pointer dereference

Something is breaking the file queue by not properly writing new lines to it in some cases.
This leads to invalid file paths in the file queue.
It eventually leads to to a null pointer defeference in worker_pool.go and the following error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x595d6a]

goroutine 1 [running]:
os.(*File).Name(...)
        /usr/lib/go/src/os/file.go:54
FileMango/src/scheduler.(*fileQueue).addNewJobs(0x741ec0, 0xc00011a180)
        /home/melkor/go/src/FileMango/src/scheduler/worker_pool.go:124

johnnydouble / filemango Goto Github PK

filemango's People

Contributors

Stargazers

Watchers

Forkers

filemango's Issues

Empty file queue causes index out of range error.

implement batching on file discovery

QueueFile function should ensure that it disallows directories from being Queued

Handle system calls related to stopping execution.

On second run db.Fold function causes a segmentation violation

Ensure only a single instance runs at a time

Worker pool

Corrupted file queue leading to null pointer dereference

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent