New Fields to Openvpn about pfelk HOT 14 CLOSED

Interesting, i see it puts the username or domain\username inf front of the pattern. so we can try changing the openvpn section to something like this to capture that line....

# OPENVPN
OPENVPN (%{OPENVPNIP}|%{OPENVPNUSER}|%{OPENVPNLOG})
OPENVPNIP %{IP:[vpn_source][ip]}\:%{INT:[vpn_source][port]}%{SPACE}\[%{DATA:vpn_client}\]%{SPACE}Peer%{SPACE}Connection%{SPACE}Initiated%{SPACE}with%{GREEDYDATA}
OPENVPNUSER (?<openvpn_user>\b[\w\\\-.]+\b)\/%{IP:[vpn_source][ip]}\:%{INT:[vpn_source][port]}%{SPACE}%{GREEDYDATA:openvpn_message}
OPENVPNLOG %{GREEDYDATA:openvpn_message}

The real solution will be to go through a longer section of logs (or finding a sample log) to review all the different syntaxes in the log file for openvpn server. @a3ilson suggested this to me a month ago, but i opted to solve the problem more simply - which was my error :)

If you feel comfortable sanitizing a bigger raw log sample (something that looks like it gets every type of log entry, i can try to do that)

from pfelk.

Sorry my delay.

ill collect logs to share. Give me few days.

from pfelk.

In this case it sounds like you are using an openvpn server from pfsense/opnsense correct?

Do you have any examples of the raw log entries?

from pfelk.

Thanks for the quickly reply.
We are using openvpn from psfsense, yes. here is some raw's from the event.original field. For security reason i repĺace IPs for XXX.XX.XX.X.

<29>Jan 28 10:20:15 openvpn[82939]: itei\gabriela.silveira/XXX.XX.XXX.XXX:49214 peer info: IV_PLAT=win
<29>Jan 28 10:16:34 openvpn[82939]: ana.carminatti/XXX.X.XXX.XX:27480 peer info: IV_PLAT=win
<29>Jan 28 10:12:56 openvpn[82939]: gustavo.pires/XXX.XXX..XXX.XXX:57872 peer info: IV_PLAT=win
<29>Jan 28 08:55:24 openvpn[39084]: felipe.silva/XXX.XXX.XXX.X:15376 peer info: IV_PLAT=win

--

from pfelk.

Interesting, i see it puts the username or domain\username inf front of the pattern. so we can try changing the openvpn section to something like this to capture that line....

# OPENVPN
OPENVPN (%{OPENVPNIP}|%{OPENVPNUSER}|%{OPENVPNLOG})
OPENVPNIP %{IP:[vpn_source][ip]}\:%{INT:[vpn_source][port]}%{SPACE}\[%{DATA:vpn_client}\]%{SPACE}Peer%{SPACE}Connection%{SPACE}Initiated%{SPACE}with%{GREEDYDATA}
OPENVPNUSER (?<openvpn_user>\b[\w\\\-.]+\b)\/%{IP:[vpn_source][ip]}\:%{INT:[vpn_source][port]}%{SPACE}%{GREEDYDATA:openvpn_message}
OPENVPNLOG %{GREEDYDATA:openvpn_message}

The real solution will be to go through a longer section of logs (or finding a sample log) to review all the different syntaxes in the log file for openvpn server. @a3ilson suggested this to me a month ago, but i opted to solve the problem more simply - which was my error :)

If you feel comfortable sanitizing a bigger raw log sample (something that looks like it gets every type of log entry, i can try to do that)

@camarigor - Feel free to send logs. Appears to be a simple fix/update to the grok pattern

from pfelk.

Hey guys, sorry my delay.
This is the complete info stored on the field event.original, my lucene query is pf_program:openvpn AND pf_message:"peer info" AND pf_message:IV_PLAT
Whit this info i can get name/ip or ip from the user connected in our vpn. Bellow i paste the lines whit every type of log entry.

 <29>Jan 31 10:00:06 openvpn[82939]: XXX.XXX.46.234:16638 peer info: IV_PLAT=win
 <29>Jan 31 09:58:14 openvpn[39084]: XXX.XXX.208.5:1194 peer info: IV_PLAT=win
 <29>Jan 31 09:55:43 openvpn[82939]: XXX.21.XXX.74:5131 peer info: IV_PLAT=win
 <29>Jan 31 09:47:08 openvpn[39084]: pedro.adroaldo/XXX.43.XX.48:1074 peer info: IV_PLAT=win
 <29>Jan 31 09:45:06 openvpn[82939]: manoela.andrade/XXX.6.XXX.80:1646 peer info: IV_PLAT=win
 <29>Jan 31 09:44:39 openvpn[82939]: gabriela.paloma/189.73.138.141:56407 peer info: IV_PLAT=win
 <29>Jan 31 04:07:35 openvpn[82939]: manoel.junior/XXX.XXX.59.111:61788 peer info: IV_PLAT=win
 <29>Jan 30 23:44:02 openvpn[82939]: luis.daemon/XXX.32.XX.104:54712 peer info: IV_PLAT=win
 <29>Jan 30 23:29:52 openvpn[39084]: fernando/XX.XXX.82.25:1194 peer info: IV_PLAT=win
 <29>Jan 30 17:39:47 openvpn[82939]: adrianmanoel/XXX.XXX.224.57:58488 peer info: IV_PLAT=win
 <29>Jan 30 12:06:04 openvpn[82939]: Perfidia/xxx.37.XXX.17:10300 peer info: IV_PLAT=mac
 <29>Jan 28 21:28:21 openvpn[82939]: itwow\luiza/XXX.XX.38.33:13852 peer info: IV_PLAT=win
 <29>Jan 28 20:56:49 openvpn[82939]: itwow\maria.souza/XXX.XX.51.80:54337 peer info: IV_PLAT=win
 <29>Jan 28 19:24:48 openvpn[82939]: itwow\airton.magal/XXX.XXX.124.215:28865 peer info: IV_PLAT=win
 <29>Jan 28 18:21:15 openvpn[82939]: itwow\brenno.farias/XXX.6.XXX.26:21524 peer info: IV_PLAT=win
 <29>Jan 24 12:10:39 openvpn[82939]: itwow\Rick.dudalin/XXX.58.XXX.92:12005 peer info: IV_PLAT=win
 <29>Jan 30 20:06:01 openvpn[82939]: marco.ruanes/XXX.XXX.209.165:44750 peer info: IV_PLAT=linux
 <29>Jan 30 18:00:40 openvpn[82939]: XXX.60.XX.67:31099 peer info: IV_PLAT=linux ```

Thanks for the help. Let me know if i can help whit more info. 

from pfelk.

Ok, in this case since you aren't looking for every kind of log entry for openvpn, just this one with the user name, IP and platform....

Try manually adding this grok pattern in place of the existing openvpn section of the pf-grok pattern file:

# OPENVPN
OPENVPN (%{OPENVPNIP}|%{OPENVPNUSER}|%{OPENVPNLOG})
OPENVPNIP %{IP:[vpn_source][ip]}\:%{INT:[vpn_source][port]}%{SPACE}\[%{DATA:vpn_client}\]%{SPACE}Peer%{SPACE}Connection%{SPACE}Initiated%{SPACE}with%{GREEDYDATA}
OPENVPNUSER (?<openvpn_user>\b[\w\\\-.]+\b)\/%{IP:[vpn_source][ip]}\:%{INT:[vpn_source][port]}%{SPACE}%{GREEDYDATA:openvpn_message}
OPENVPNLOG %{GREEDYDATA:openvpn_message}

It should be a step in the right direction to add the fields you want

from pfelk.

I did. i add that grok at pattern file. logstash/patterns/pf-12.2019.grok

And then i collect that info from elastic

from pfelk.

Ok, very good, if you refresh the index patter, and look at the discover view - do you see those fields being populated .. or any _gokparsefailures being generated?

from pfelk.

Sir, looks great work. Now we have more fields for openvpn.
If may, i would suggest openvpn_plat field to store connection platform like linux, mac, windows or wherever. It could be awesome to use in dashboards.
After finish this dashboard ill open a pull request to add the openvpn.json dashboard and suricata, where i did some changes.
Thanks for awesome changes.

from pfelk.

Try setting this as the OPENVPNUSER line in the pattern file:

OPENVPNUSER (?<openvpn_user>\b[+\w\\.-]+\b)?/?%{IP:[vpn_source][ip]}:%{INT:[vpn_source][port]} peer info: IV_PLAT=%{WORD:openvpn_plat}

That way it does not error on strings with no user, but no matter what i try for the regex it puts an extra backslash in the ones with a domain\name... like this "domain\\user.name"

if you don't care about the domain use this:

OPENVPNUSER (?<openvpn_user>\b[+\w\.-]+\b)?/?%{IP:[vpn_source][ip]}:%{INT:[vpn_source][port]} peer info: IV_PLAT=%{WORD:openvpn_plat}

which omits the domian and keeps just the user id; but either captures "win" and "linux" etc in a field "openvpn_plat"

from pfelk.

Ok, sorry for multiple posts, i banged on this a bit more and this seems to cover all the sample lines (no username, just username, and domain\usrname) as well as setting the openvpn_plat field

Test this for a bit:

(%{WORD:openvpn_domain}?\\)?(?<openvpn_user>\b[+\w\.-]+\b)?/?%{IP:[vpn_source][ip]}:%{INT:[vpn_source][port]} peer info: IV_PLAT=%{WORD:openvpn_plat}

from pfelk.

(%{WORD:openvpn_domain}?\\)?(?<openvpn_user>\b[+\w\.-]+\b)?/?%{IP:[vpn_source][ip]}:%{INT:[vpn_source][port]} peer info: IV_PLAT=%{WORD:openvpn_plat}

works like a charm. openvpn new fields are awesome to build new dashboards. ill close this issue.
if i may ask, you gonna commit this changes at .grok file? and should i open a pull request for openvpn.json dashboards?

from pfelk.

Excellent, I will add this to the .grok file today.

As far as the dashboard/visulizations - please feel free to open a PR if you like

from pfelk.