Comments (5)
@hyprvisr Well, you could extend easily logsniffer by implementing your own Reader for specific Log to parse.
Otherwise as there are 2 main classes implemented, you have out-of-box log4j where you need to follow ConverionPattern (look at doc), or you can go with regular expressions.
Here is example for parsing PostgreSQL log file for messages that show tables being locked:
.*user=(?P<lock_user_name>\w+),db=(?P<lock_database>\w+)
LOG:\s+process (?P<lock_process_id>\d+)
acquired (?P<lock_type>\w+) on (?<lock_on>\w+)
(?P<lock_tuple>[\(\)0-9,]+) of (?P<lock_object_type>\w+)
(?P<lock_object_oid>\d+) of database (?P<lock_db_oid>\d+)
after (?P<lock_wait_time>[0-9.]+).*
You can also define scanner under Events in GUI, where you can select log LEVEL you are interested in. Here you have some builtin mechanism for log4j, but you are again free to write custom regular expression.
As real world scenario, you can do your data source only for ERROR level (by regular expression) and the scanner to send emails/http requests to destination per specific content:
- DB down -> send notification to infrastructure team
- APP down -> send notification to app team
I plan to extend notifications to work with AMQP protocol additionally as I use message broker services mostly (very limited of HTTP - only for external usage).
from logsniffer.
Hi, I can't get the log4j parsing to work. I just pasted in the pattern from my log4j.properties file:
%d{dd MMM HH:mm:ss} %m%n
and set %d to Date and %m to Message in the conversion mapping, but it gives me a parsing error on the date part. Also tried %dd %MMM %HH:mm:ss
with corresponding mappings, still no luck.
from logsniffer.
@kuzma725: Strange behavior. I've tested %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] %m%n
and it works fine. Could you provide few log lines to see the syntax?
from logsniffer.
@mbok Sure, copied this from Log entry details popup. You can see that it recognized the timestamp, but assumes year 1970 because the year is not specified (that's fine). But then it still considers the date as part of the message(?) which causes multiple entries to be concatted into one:
If_timestamp: Sep 6, 1970 11:17:34 PM
lf_unformatted: true
Message: 06 Sep 23:17:34 File watch poll still alive... 06 Sep 23:17:48 Close client socket with status 1002: 1277288408 06 Sep 23:18:24 File watch poll still alive... 06 Sep 23:19:14 File watch poll still alive...
from logsniffer.
I should have mentioned this earlier: some of my log entries have multiple lines of text and extra newlines; for example, this is all 1 entry (modifed content to remove sensitve info):
06 Sep 16:28:25
Line 1 of Entry 1
Line 2 of Entry 1
Line 3 of Entry 1
I was able to get rid of the parsing errors by using [ ] around the message, but it still doesn't parse the log correctly because, as I showed in my previous post, it lumps multiple entries into one, including the timestamps as part of the message. I guess my question is more about the conversion pattern spec: for example, it doesn't explicitly say anywhere that [ ] means the pattern is optional, but I happen to know this from using regular expressions. If someone could give me a working pattern for the sample entry above, I would appreciate it.
from logsniffer.
Related Issues (20)
- JSON deserialization failure HOT 2
- Migrate to Elasticsearch 2.x HOT 3
- CSV Reader HOT 1
- Support for compressed log files
- Unable to create new log source
- Rename the title of the project HOT 1
- Add support for an AWS SNS Publisher HOT 1
- Error: Unable to access jarfile .http HOT 2
- Is TLS Supported for the SMTP Notification? HOT 3
- network syslogs HOT 2
- init script HOT 1
- baseURL setting doesn't seem to work HOT 8
- [question] How to stream remote logs? HOT 2
- Migrating to Elastic Search 5.*
- Allow sniffers against multiple log sources
- log4j Conversion Pattern Issue HOT 1
- First bug after 30 seconds
- web ui uses too much cpu power - are you mining coins?
- Sniffers: Send notification(f.i mail) with multiple lines sends multiple mails
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logsniffer.