shadonet / pfsense-pkg-zeek Goto Github PK
View Code? Open in Web Editor NEWZeek (formerly Bro) Network Security Monitor package for pfSense router/firewall
License: Apache License 2.0
Zeek (formerly Bro) Network Security Monitor package for pfSense router/firewall
License: Apache License 2.0
Hello, I can't find any logs associated with this package and the service will not start. Any suggestions?
Hello
I have installed pfSense-pkg-zeek on PfSense: 2.5.2, it installs but it does not start and it fails with the error can't find local.zeek
Thanks very much for all the work, really and truly appreciated!
Below the details
Describe the bug
Zeek fails to start: fatal error: can't find local.zeek
Version information :
Bug details
[ZeekControl] > start
starting zeek ...
Error: zeek terminated immediately after starting; check output with "diag"
[ZeekControl] > diag
[zeek]
No core file found and gdb is not installed. It is recommended to
install gdb so that ZeekControl can output a backtrace if Zeek crashes.
Zeek 4.0.2
FreeBSD 12.2-STABLE
Zeek plugins:
Zeek::Netmap - Packet acquisition via Netmap (dynamic, version 1.0.0)
==== No reporter.log
==== stderr.log
fatal error: can't find local.zeek
==== stdout.log
max memory size (kbytes, -m) unlimited
data seg size (kbytes, -d) 33554432
virtual memory (kbytes, -v) unlimited
core file size (blocks, -c) unlimited
==== .cmdline
-i vmx0 -U .status -p zeekctl -p zeekctl-live -p standalone -p local -p zeek local.zeek zeekctl zeekctl/standalone zeekctl/auto
==== .env_vars
PATH=/usr/local/bin:/usr/local/share/zeekctl/scripts:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
ZEEKPATH=/usr/local/spool/installed-scripts-do-not-touch/site::/usr/local/spool/installed-scripts-do-not-touch/auto:/usr/local/share/zeek:/usr/local/share/zeek/policy:/usr/local/share/zeek/site
CLUSTER_NODE=
==== .status
TERMINATED [fatal_error]
==== No prof.log
==== No packet_filter.log
==== No loaded_scripts.log
[ZeekControl] >
Zeek supports multiple plugins that add additional functionality to the network monitor.
If this package could add support for searching and installing plugins via the Zeek package manager it would greatly enhance the utility for traffic monitoring.
I have manually installed Zeek via the FreeBSD ports and was able to manually add this functionality to a pfSense network appliance but management had to be done via the console vs the webgui.
Hello,
I installed pfSense-pkg-zeek following your instructions and all went extremely well. The only issue I have now is that the service won't start. I am on FreeBSD 12 using 2.4.5-RELEASE (amd64) install of PfSense.
Is there a command line way to start the service?
Thanks,
Jim
Describe the bug
Zeek crash report
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Zeek starts and operates.
Version information (please complete the following information):
PfSense: 2.6.0-DEVELOPMENT (amd64)
built on Sat Oct 16 05:24:35 UTC 2021
FreeBSD 12.2-STABLE
Zeek: zeek-4.0.4
Zeekctl: 2.3.0
pfSense-pkg-zeek: 3.0.6_3
Additional context
User-Agent: ZeekControl 2.3.0
This crash report does not include a backtrace. In order for crash reports
to be useful when Zeek crashes, a backtrace is needed.
No core file found and gdb is not installed. It is recommended to
install gdb so that ZeekControl can output a backtrace if Zeek crashes.
Zeek 4.0.4
FreeBSD 12.2-STABLE
Zeek plugins:
Zeek::Netmap - Packet acquisition via Netmap (dynamic, version 1.0.0)
==== No reporter.log
==== stderr.log
fatal error: problem with interface eth0 (pcap_error: BIOCSETIF failed: Device not configured (pcap_activate))
==== stdout.log
max memory size (kbytes, -m) unlimited
data seg size (kbytes, -d) 33554432
virtual memory (kbytes, -v) unlimited
core file size (blocks, -c) unlimited
==== .cmdline
-i eth0 -U .status -p zeekctl -p zeekctl-live -p standalone -p local -p zeek local.zeek zeekctl zeekctl/standalone zeekctl/auto
==== .env_vars
PATH=/usr/local/bin:/usr/local/share/zeekctl/scripts:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
ZEEKPATH=/usr/local/spool/installed-scripts-do-not-touch/site::/usr/local/spool/installed-scripts-do-not-touch/auto:/usr/local/share/zeek:/usr/local/share/zeek/policy:/usr/local/share/zeek/site
CLUSTER_NODE=
==== .status
TERMINATED [fatal_error]
==== No prof.log
==== No packet_filter.log
==== No loaded_scripts.log
--
[Automatically generated.]
Check to enable the Zeek NSM doesn't do anything. It doesn't matter if it is checked or unchecked, this does not start or stop any services or anything
Hi - I was wondering if there is a guide for installing this package on pfsense. I did not see it in the list of available packages. Thanks.
Allow configuration as a cluster vs a single interface.
If multiple Zeek nodes are in use being able to setup a cluster makes it much easier to log data from multiple sources. Even in the event that multiple interfaces need to be monitored this solution works much more efficiently since multiple zeek processes can be used; one per interface.
Additionally this setup would make centralized logging much easier since pfSense could be setup as a worker node and send the data back to a centralized Zeek logger or manager.
Hi, I want to transfer zeek log to elasticsearch server. As I research, I need filebeat to ship log and normalize (but filebeat is not supporting FreeBSD fully). Or I just need to config Zeek log send directly to elastic server?
Thanks
I'm attempting to write my logs to an external mount that is being monitored by my SIEM, however, I noticed that the Log Directory
setting only updates the archived/compressed logs of the hour, and not the current
logs, which are written as part of the spool and configured in zeekctl.cfg
as:
# Location of the spool directory where files and data that are currently being
# written are stored.
SpoolDir = /usr/local/spool
Maybe it would be good to provide a setting to configure each of these; or simplify it so that both Log Directory and Spool Directory will be the same?
Thanks for this package, I've been looking for something like this for a while.
I've just followed the steps in your instructions but Zeek does not start after the installation. Even if I use Services > Zeek (Start button) it will not load.
I SSH'ed into the host and tried to manually start it with service zeek.sh start
which lead me to get some messaging:
Warning: ZeekControl plugin uses legacy BroControl API. Use
'import ZeekControl.plugin' instead of 'import BroControl.plugin'
starting zeek ...
creating crash report for previously crashed nodes: zeek
Error: error occurred while trying to send mail: send-mail: /usr/sbin/sendmail not found
To get even more debugging, you can use:
zeekctl
...
[ZeekControl] > diag
...
==== stderr.log
fatal error in /usr/local/share/zeek/zeekctl/standalone.zeek, line 3: can't find standalone-layout
Which then leads me to believe that Zeek wasn't "initialised" or whatever it needs to happen, so you can do so within ZeekControl
with:
[ZeekControl] > install
After that, Zeek will work as expected.
By default Zeek stores logs in a plain text that if exported to a SIEM or log aggregator require additional parsing and sorting.
Zeek does have the ability to format logs as structured JSON making importing much easier.
Zeek does not start on 2.4.5-RELEASE-p1 as sendmail was not found.
Please advise how to proceed.
Jun 4 20:20:36 php-fpm 50102 /pkg_edit.php: The command '/usr/local/etc/rc.d/zeek.sh start' returned exit code '1', the output was 'Error: error occurred while trying to send mail: send-mail: /usr/sbin/sendmail not found Error: zeek terminated immediately after starting; check output with "diag" Hint: Run the zeekctl "deploy" command to get started. starting zeek ... creating crash report for previously crashed nodes: zeek'
[2.4.5-RELEASE][[email protected]]/: find / -name "sendmail" -print
/usr/libexec/sendmail
/usr/lib/debug/usr/libexec/sendmail
/usr/share/sendmail
[2.4.5-RELEASE][[email protected]]/: pkg search sendmail
geany-plugin-sendmail-1.37 Geany plugin: send a document as attachment
geany-plugin-sendmail-gtk2-1.37 Geany plugin: send a document as attachment
mini_sendmail-1.3.9 Tiny MTA with only smart relay capability
p5-Mail-Sendmail-0.80 Perl module implementing a simple, platform-independent mailer
p5-Sendmail-AccessDB-1.04 Perl5 interface to the Sendmail access.db list
p5-Sendmail-Milter-0.18_2 Module to write mail filters in Perl using sendmail's mail filter API
popa3d-before-sendmail-1.0.3 Secure, performance, tiny POP3 daemon
sendmail+tls+sasl2-8.16.1_4 Reliable, highly configurable mail transfer agent with utilities
sendmail-devel-8.16.0.50_2 Reliable, highly configurable mail transfer agent with utilities
usendmail-0.1.6_2 Replacement for qmail's sendmail drop-in
I am getting
checking configurations ...
zeek scripts failed.
[libhilti] Internal error: could not determine path of current executable: /proc needs to be mounted
--- Aborting in libhilti
/usr/local/share/zeekctl/scripts/check-config: line 50: 35589 Abort trap (core dumped) ZEEKCTL_DISABLE_LISTEN=1 ZEEKCTL_CHECK_CONFIG=1 "${zeek}"
after attempting to run zeekctl deploy
pfsense is 2.6.0-RELEASE
Is this compatibility issue?
Describe the bug
After trying to start zeek complains about /usr/local/logs not existing in an email sent. Have tried uninstalling and reinstalling and get error on install. Not sure if it is related to /usr/local/logs issue, as emails no longer arrive with this error. It now tries to start and fails. see below for more detail.
To Reproduce
Steps to reproduce the behavior:
php-fpm | 42909 | /rc.start_packages: The command '/usr/local/etc/rc.d/zeek.sh start' returned exit code '1', the output was 'checking configurations ... zeek scripts failed. fatal error: can't find local.zeek '
Expected behavior
Zeek to start without error
Zeek to install with out error?
Screenshots
If applicable, add screenshots to help explain your problem.
Version information (please complete the following information):
Additional context
>>> Installing pfSense-pkg-zeek...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 6 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
bash: 5.1.8 [pfSense]
ipsumdump: 1.86 [pfSense]
lbl-cf: 1.2.5 [pfSense]
lbl-hf: 1.9 [pfSense]
pfSense-pkg-zeek: 3.0.6_3 [pfSense]
zeek: 4.0.2 [pfSense]
Number of packages to be installed: 6
The process will require 68 MiB more space.
13 MiB to be downloaded.
[1/6] Fetching pfSense-pkg-zeek-3.0.6_3.txz: ... done
[2/6] Fetching zeek-4.0.2.txz: .......... done
[3/6] Fetching lbl-hf-1.9.txz: ... done
[4/6] Fetching lbl-cf-1.2.5.txz: . done
[5/6] Fetching bash-5.1.8.txz: .......... done
[6/6] Fetching ipsumdump-1.86.txz: .......... done
Checking integrity... done (0 conflicting)
[1/6] Installing lbl-hf-1.9...
[1/6] Extracting lbl-hf-1.9: ..... done
[2/6] Installing lbl-cf-1.2.5...
[2/6] Extracting lbl-cf-1.2.5: ..... done
[3/6] Installing bash-5.1.8...
[3/6] Extracting bash-5.1.8: .......... done
[4/6] Installing ipsumdump-1.86...
[4/6] Extracting ipsumdump-1.86: ......... done
[5/6] Installing zeek-4.0.2...
===> Creating groups.
Using existing group 'zeek'.
===> Creating users
Using existing user 'zeek'.
[5/6] Extracting zeek-4.0.2: .......... done
Traceback (most recent call last):
File "/usr/local/bin/zeekctl", line 814, in <module>
sys.exit(main())
File "/usr/local/bin/zeekctl", line 781, in main
loop = ZeekCtlCmdLoop(ZeekCtl, interactive, cmd)
File "/usr/local/bin/zeekctl", line 29, in __init__
self.zeekctl = zeekctl_class(ui=self)
File "/usr/local/lib/zeek/python/zeekctl/ZeekControl/zeekctl.py", line 73, in __init__
self.config = config.Configuration(
File "/usr/local/lib/zeek/python/zeekctl/ZeekControl/config.py", line 73, in __init__
self.config = self._read_config(cfgfile)
File "/usr/local/lib/zeek/python/zeekctl/ZeekControl/config.py", line 596, in _read_config
with open(fname, "r") as f:
PermissionError: [Errno 13] Permission denied: '/usr/local/etc/zeekctl.cfg'
[6/6] Installing pfSense-pkg-zeek-3.0.6_3...
[6/6] Extracting pfSense-pkg-zeek-3.0.6_3: ......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...done.
Executing custom_php_resync_config_command()...done.
Menu items... done.
Services... done.
Writing configuration... done.
=====
Message from zeek-4.0.2:
--
The rc.d script now honors the zeek_user rc.d variable. To run as
a user other than root (the default) you need to make a few changes.
For example to run as the user zeek, add this to /etc/rc.conf:
zeek_enable="YES"
zeek_user="zeek"
Add this to /etc/devfs.conf:
own bpf root:bpf
perm bpf 0660
And add zeek to the bpf group:
bpf:*:81:zeek
and restart the devfs service:
service devfs restart
or reboot.
If the interface defined in node.cfg is configured for NIC checksum
offloading (the default when this feature is supported by the
hardware) you will want to set ignore_checksums in site/local.zeek:
redef ignore_checksums = T;
=====
Message from pfSense-pkg-zeek-3.0.6_3:
--
Please visit Services > Zeek NSM menu to configure the package.
>>> Cleaning up cache... done.
Success
Hi
I have tried several times to install zeek on my pfsense. The installation seems to work, but after enabling zeek, it does not startup.
My System:
BIOS Vendor: coreboot
Version: v4.13.0.1
Release Date: Wed Nov 25 2020
Version 2.5.0-RC (amd64)
built on Sat Feb 13 03:07:19 EST 2021
FreeBSD 12.2-STABLE
CPU Type AMD GX-412TC SOC
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (inactive)
Kernel PTI Disabled
MDS Mitigation Inactive
installed modules:
acme security 0.6.9_3
arping net 1.2.2_2
arpwatch net-mgmt 0.2.0_4
bandwidthd net-mgmt 0.7.4_5
darkstat net-mgmt 3.1.3_5
frr net 1.1.0_4
iperf benchmarks 3.0.2_5
Lightsquid www 3.0.6_8
lldpd net-mgmt 0.9.11
mailreport mail 3.6.3_2
mtr-nox11 net 0.85.6_2
map security 1.4.4_2
ntopng net 0.8.13_9
openvpn-client-export security 1.5_5
pfBlockerNG-devel net 3.0.0_10
RRD_Summary sysutils 2.0_1
Shellcmd sysutils 1.0.5_2
squid www 0.4.45_3
Status_Traffic_Totals net 2.3.2_2
syslog-ng sysutils 1.15_7
zeek security 3.0.6_1
I am not really familar with zeek/(bro).
If there is a need for more information like logs etc, please contact me
many thanks for help
thiamata
the dialog box to select logs on the real-time logs page does not point to the spool directory where the logs are actually being written
Installing pfSense-pkg-zeek...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 6 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
bash: 5.1.4 [pfSense]
ipsumdump: 1.86 [pfSense]
lbl-cf: 1.2.5 [pfSense]
lbl-hf: 1.9 [pfSense]
pfSense-pkg-zeek: 3.0.6_1 [pfSense]
zeek: 3.0.12_2 [pfSense]
Number of packages to be installed: 6
Message from zeek-3.0.12_2:
--
The rc.d script now honors the zeek_user rc.d variable. To run as
a user other than root (the default) you need to make a few changes.
For example to run as the user zeek, add this to /etc/rc.conf:
zeek_enable="YES"
zeek_user="zeek"
Add this to /etc/devfs.conf:
own bpf root:bpf
perm bpf 0660
And add zeek to the bpf group:
bpf:*:81:zeek
and restart the devfs service:
service devfs restart
or reboot.
If the interface defined in node.cfg is configured for NIC checksum
offloading (the default when this feature is supported by the
hardware) you will want to set ignore_checksums in site/local.zeek:
redef ignore_checksums = T;
=====
Message from pfSense-pkg-zeek-3.0.6_1:
--
Please visit Services > Zeek NSM menu to configure the package.
Cleaning up cache... done.
Success
Changing the log directory in the Zeek settings page does not change where logs are stored
The system has em0 instead of eth0:
==== stderr.log
fatal error: problem with interface eth0 (pcap_error: BIOCSETIF failed: Device not configured (pcap_activate))
==== stdout.log
max memory size (kbytes, -m) unlimited
data seg size (kbytes, -d) 33554432
virtual memory (kbytes, -v) unlimited
core file size (blocks, -c) unlimited
==== .cmdline
-i eth0 -U .status -p zeekctl -p zeekctl-live -p standalone -p local -p zeek local.zeek zeekctl zeekctl/standalone zeekctl/auto
Hello,
Do i need to keep the option to keep the upstream FreeBSD repo enabled? I changed it from enabled back to false and the BRO service doesn't seem to start and there are no logs to look at
I am unable to install on a 2.4.x (FreeBSD 11 based) pfSense box. Any chance of an update? Output below, thanks!
pkg add pfSense-pkg-bro-2.4.1.txz
Installing pfSense-pkg-bro-2.4.1...
pkg: wrong architecture: FreeBSD:10:amd64 instead of FreeBSD:11:amd64
Failed to install the following 1 package(s): pfSense-pkg-bro-2.4.1.txz
Any chance you got this to work?
Hello,
I am running 2.4.5-RELEASE-p1 and I had to edit another file to enable the upstream pkg repositories.
I did edit the files as instructed. But the file "/usr/local/etc/pkg/repos/pfSense.conf" links to "/usr/local/share/pfSense/pkg/repos/pfSense-repo-245.conf" which contains "FreeBSD: { enabled: no }" too.
After changing this from "no" to "yes" I was able to search and install packages from the FreeBSD repository.
One other thing: I was able to download "pfSense-pkg-zeek-0.1.1.txz" directly by issuing the following command: curl -O "https://raw.githubusercontent.com/shadonet/pfSense-pkg-zeek/master/data/pfSense-pkg-zeek-0.1.1.txz"
Kind regards,
Bouke
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.