Comments (5)
And forgot to ask, could you breifly explain what is the num_candidates
parameter for in the prediction?
Thank you!!!!
from logdeep.
It depend on your parsing tool, my benchmark result is depend on "the ground truth" number of the template(28) in dataset"
num_candidates means the label in top num_candidates is labeled as normal log.
(you need to read the deeplog paper to get a better understanding of num_candidates...)
- try to finetune num_candidates to get a better F1 score.
- try to modify your parsing code to get a result close to the Ground truth(28 templates)
from logdeep.
Thank you so much for the suggestions! That's really helpful!
One follow up question I have is that, this may sounds a naive question, but do we always know the ground truth number of the log? And when we are using the parsing tool, we want to have the result/template as close as possible to the ground truth number we know by modifying the parsing code?
from logdeep.
In industrial applications, the constantly updated log has no definite ground truth templates, you need to continuously optimize the model based on performance indicators :)
from logdeep.
Got it! Thank you! I don't have further question for now! :))
from logdeep.
Related Issues (20)
- hdfs parsing
- hdfs_train sequence file doesn't correspond to the sequence file generated for 100k structured file provided in the repository HOT 21
- hdfs文件夹下的event2semantic_vec.json这个文件是怎么用原始日志得到的 HOT 2
- 请问作者,data_read('template.txt')中template.txt文件是怎么得到的?第二个脚本里deepLog_hdfs_train.txt文件在data文件夹下也没看到 HOT 4
- 请问下,deeplog输出的这些指标是基于啥计算的,无监督的话咋知道哪些是对的,哪些是错的?最后有输出啥结果文件,找出有问题的日志窗口吗 HOT 1
- 关于 TP,FP,TN,FN的问题!
- 怎么生成训练数据hdfs_train呢? HOT 1
- '../result/deeplog/deeplog_last.pth 这个文件怎么产生 HOT 1
- prepare_log 这个的内容是什么 HOT 4
- Question about hdfs_train, hdfs_test_normal, and hdfs_test_abnormal HOT 1
- In HDFS templates count is 28? HOT 4
- An error occurs when the terminal command line runs
- Question about deeplog in logs Apache
- A data processing problem HOT 1
- One-hot encoding?
- F1 not achieved
- DeepLog hdfs original unpased data
- Possible implementation errors for session_windows
- In RobustLog's code, I didn't see the operation of weighting the semantic vector with TF-IDF
- Anomaly log file type detection and predict future log error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logdeep.