This repository holds the documentation for the crowdsec project.
Online version of this documentation is available here: https://doc.crowdsec.net/
CrowdSec Documentation: Comprehensive guides, tutorials, and references for installing, configuring, and using CrowdSec, an open-source cybersecurity platform. Contributions welcome!
Home Page: https://doc.crowdsec.net/
This repository holds the documentation for the crowdsec project.
Online version of this documentation is available here: https://doc.crowdsec.net/
Just noted when helping a user that we have not documented how to use exclude_regexps
from version >= 1.4.2
Placing this here to remind me or if anyone has the time to do it ๐๐ป
There isn't any documentation for the wordpress bouncer on how to configure captcha
Hey, I will create a PR for these changes!
When a user supplies a list of Grok nodes in a parser and they want all nodes to be evaluated they must omit the apply_on
key from all nodes. If they do not the node that evaluates first and / or the only node with apply_on
will be the value for all parsed information this was confusing and was not explained in docs!
I will do this shortly, just finding the right words!
Here a link to the caddy parser which I was following for my own to explain what I mean
We should include in haproxy bouncer some information on how to set the src IP before it hits the lua code to prevent users from getting the proxy IP.
// Will update with more information
The readme says I should build the docs with yarn, however:
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
Should this be replaced by yarn.lock?
otherwise, it might confuse people that already have services using port 8080
As the dashboard is not compatible with ARM architectures, it should be great to inform the user and blocking him directly from the cscli command, else, patient users will wait for long long time... ๐ด
admin@ip-XXXXXXXXXX:~$ sudo cscli dashboard setup --listen 0.0.0.0
INFO[01-07-2021 10:24:19 AM] /var/lib/crowdsec/data/metabase.db exists, skip.
INFO[01-07-2021 10:24:19 AM] Pulling docker image metabase/metabase:v0.37.0.2
............................................................................................................
INFO[01-07-2021 10:24:29 AM] creating container '/crowdsec-metabase'
INFO[01-07-2021 10:24:33 AM] waiting for metabase to be up (can take up to a minute)
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
CrowdSec since 1.5 has alot more options to build RE2 / WASM / STATIC we should document these options better within docs as currently if you are not internal you dont know how to compile with these options unless you love reading makefiles
now that we switch to packagecloud, we need to edit the install where it's written
echo "deb https://packagecloud.io/crowdsec/crowdsec/debian/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/crowdsec.list > /dev/null
and add the same for ubuntu
echo "deb https://packagecloud.io/crowdsec/crowdsec/ubuntu/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/crowdsec.list > /dev/null
For crowdsec documentation (https://docs.crowdsec.net/docs/getting_started/install_crowdsec), it is mentionned that with must install with yum
ย on centos8 and dnf
on centos7, which is the invert.
For bouncers, we don't provide different installation method for centos7/centos8. (eg. https://docs.crowdsec.net/docs/bouncers/firewall)
Not sure where to post this, just a quick note that in https://doc.crowdsec.net/docs/parsers/create there is a invalid command cscli hubtest inspect
(mentioned twice) I believe it should be cscli hubtest explain
Add blurb about use_wal support and what improvements it has over not using it?
Linked to
crowdsecurity/crowdsec#1860
Hi,
it's related to https://doc.crowdsec.net/Crowdsec/v1/getting_started/concepts/ essentially but it can be a global approach about your whole approach.
The problem is that you directly use concepts like API, framework etc...
Where actually average people or lambda people are familiarized with software. Not abstract concept. They are used to their usage and their command line usage if they have installed a few ubuntu servers for example for their personal use.
Its a reproach that I would like have done to many other tech companies, so don't worry it's a common thing among devs etc. But if you ever teached among students even older students that would maybe be used at some level to manage the servers at work etc and who would learn computer science in a continuing training (formation continue).... and as your tech corp is a french initiative, I know you will be sensitive to what I'm going to say... they are used to what they are used to write in their command line ssh. Not to an abstract approach who would require that you actually are quite knowladgeable in the field already.
So what would be nice is that there is like an overview-bis or something alike where average people can rely on.
What does know an average people ? or What is he familiar with?
iptables. That they know. And if they are a little bit advanced they would even know framework like ufw, firewalld or others that actually all rely on iptables. And if they are even more advanced they will know fail2ban.
We need to know at which level are we or how far are we from iptables.
So is it related to iptables? or is your "bouncer package" replacing iptables?
I would have thought for example that your package would download a blacklsit and then, as fail2ban iptables would integrate the whole list in its list.
But apparently not.
So do you see where the problem is here?
cscli decisions import
is missing documentation.
The documentation should state the minimal needed resources for crowdsec itself and the other components :
I think it would be good to have in the crowdsec tour documentation ๐
The debian package supports debconf
, but we do not document how to take advantage of it anywhere.
Either we should publish some packages, or make it explicit in the doc that we don't support it natively
After installing the FreeBSD this message is shown. Could you please elaborate how in the docs?
Message from crowdsec-1.1.1:
--
crowdsec is installed.
You need to edit the agent config file /usr/local/etc/crowdsec/crowdsec.yaml and
enable rc via sysrc.
# sysrc crowdsec_enable="YES"
https://docs.crowdsec.net/docs/local_api/database#mysql-and-mariadb
On the databases page it's advises to mysql> CREATE USER 'crowdsec'@'%' IDENTIFIED BY '<password>';
MySQLTuner advises this is insecure
Restrict Host for 'crowdsec'@'%' to 'crowdsec'@LimitedIPRangeOrLocalhost
RENAME USER 'crowdsec'@'%' TO 'crowdsec'@LimitedIPRangeOrLocalhost;
Current doc only refers to debian-based environements
This is not indicated in the documentation but it should : notification plugins are at LAPI level
While the doc gives some examples, it can be improved:
sprig
and they have access to more functions in the templateRelated to https://docs.crowdsec.net/blog/metabase_without_docker/
The repo for liberodark's helper script has been stale for quite some time.
I've made some significant improvements to the script, increasing automation for users, and making it easier to use and install.
Originally had hoped that my PR would have been looked at and merged by the owner, but doesn't seem to be any interest in doing so.
I'd be happy to take over maintenance of the script and apply any changes required if needs be.
Please let me know and if you're happy to, I'll sort out my repo to take over, and create a PR here to update the docs.
Add the sudo apt-get update
before sudo apt-get install crowdsec
when installing crowdsec from repository
On fresh buster , gnupg is not installed (please add the package to install in the documentation)
for centos8 and fc3{3,4}, documentation should refer to dnf install
instead of yum as it's the now preferred way to install stuff.
The example configuration for Gmail shows:
smtp_port: 587
encryption_type: ssltls # Required
Which yields the following error on attempted email send:
Mail Error on dialing with encryption type SSL/TLS: tls: first record does not look like a TLS handshake error
The Gmail SMTP docs say port 465 is for "SSL" and 587 for "TLS/STARTTLS".
Either of these work for me:
smtp_port: 465
encryption_type: ssltls
Or:
smtp_port: 587
encryption_type: starttls
Also, even though starttls
works it isn't listed as a valid value in my email.yaml template:
# One of "ssltls", "none"
encryption_type:
Email via Gmail works per docs example.
I think just set up email with gmail per the docs example.
Suggest:
# One of "ssltls", "starttls", "none"
2023/02/21 20:04:48 version: v1.4.6-debian-pragmatic-5f71037b40c498045e1b59923504469e2b8d0140
2023/02/21 20:04:48 Codename: alphaga
2023/02/21 20:04:48 BuildDate: 2023-02-09_14:41:04
2023/02/21 20:04:48 GoVersion: 1.19.2
2023/02/21 20:04:48 Platform: linux
2023/02/21 20:04:48 Constraint_parser: >= 1.0, <= 2.0
2023/02/21 20:04:48 Constraint_scenario: >= 1.0, < 3.0
2023/02/21 20:04:48 Constraint_api: v1
2023/02/21 20:04:48 Constraint_acquis: >= 1.0, < 2.0
# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
$ uname -a
Linux gitea 5.15.84+ crowdsecurity/crowdsec#1613 Thu Jan 5 11:58:09 GMT 2023 armv6l GNU/Linux
$ cscli hub list -o raw
LePresidente/gitea,enabled,0.2,Gitea Support : parser and brute-force detection,collections
crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections
crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections
LePresidente/gitea-logs,enabled,0.4,Parse gitea logs,parsers
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/sshd-logs,enabled,2.0,Parse openSSH logs,parsers
crowdsecurity/syslog-logs,enabled,0.8,,parsers
LePresidente/gitea-bf,enabled,0.2,Detect gitea bruteforce,scenarios
crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios
# On Linux:
$ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/*
#Generated acquisition file - wizard.sh (service: sshd) / files : /var/log/auth.log
filenames:
- /var/log/auth.log
labels:
type: syslog
---
#Generated acquisition file - wizard.sh (service: linux) / files : /var/log/syslog /var/log/kern.log /var/log/messages
filenames:
- /var/log/syslog
- /var/log/kern.log
- /var/log/messages
labels:
type: syslog
---
# Hand written file
filenames:
- /mnt/foo/gitea/log/gitea.log
labels:
type: gitea
cat: '/etc/crowdsec/acquis.d/*': No such file or directory
$ cscli config show
Global:
- Configuration Folder : /etc/crowdsec
- Data Folder : /var/lib/crowdsec/data
- Hub Folder : /etc/crowdsec/hub
- Simulation File : /etc/crowdsec/simulation.yaml
- Log Folder : /var/log/
- Log level : info
- Log Media : file
Crowdsec:
- Acquisition File : /etc/crowdsec/acquis.yaml
- Parsers routines : 1
- Acquisition Folder : /etc/crowdsec/acquis.d
cscli:
- Output : human
- Hub Branch :
- Hub Folder : /etc/crowdsec/hub
Local API Server:
- Listen URL : 127.0.0.1:8080
- Profile File : /etc/crowdsec/profiles.yaml
- Trusted IPs:
- 127.0.0.1
- ::1
- Database:
- Type : sqlite
- Path : /var/lib/crowdsec/data/crowdsec.db
- Flush age : 7d
- Flush size : 5000
$ cscli metrics
Acquisition Metrics:
โญโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ Source โ Lines read โ Lines parsed โ Lines unparsed โ Lines poured to bucket โ
โโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโผโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโค
โ file:/var/log/auth.log โ 6 โ - โ 6 โ - โ
โ file:/var/log/syslog โ 25 โ - โ 25 โ - โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโฏ
Parser Metrics:
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโฌโโโโโโโโโฌโโโโโโโโโโโฎ
โ Parsers โ Hits โ Parsed โ Unparsed โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโผโโโโโโโโโผโโโโโโโโโโโค
โ child-crowdsecurity/sshd-logs โ 20 โ - โ 20 โ
โ child-crowdsecurity/syslog-logs โ 31 โ 31 โ - โ
โ crowdsecurity/sshd-logs โ 2 โ - โ 2 โ
โ crowdsecurity/syslog-logs โ 31 โ 31 โ - โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโดโโโโโโโโโดโโโโโโโโโโโฏ
Local Api Metrics:
โญโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโฌโโโโโโโฎ
โ Route โ Method โ Hits โ
โโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโผโโโโโโโค
โ /v1/decisions/stream โ GET โ 11 โ
โ /v1/heartbeat โ GET โ 10 โ
โ /v1/watchers/login โ POST โ 2 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโดโโโโโโโฏ
Local Api Machines Metrics:
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโฌโโโโโโโโโฌโโโโโโโฎ
โ Machine โ Route โ Method โ Hits โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโผโโโโโโโโโผโโโโโโโค
โ 6fc549d69f0b4cfb9f14fef65c2d23d2PHFTzoaJ3bNUgHwU โ /v1/heartbeat โ GET โ 10 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโดโโโโโโโโโดโโโโโโโฏ
Local Api Bouncers Metrics:
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโฌโโโโโโโฎ
โ Bouncer โ Route โ Method โ Hits โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโผโโโโโโโค
โ FirewallBouncer-1676929718 โ /v1/decisions/stream โ GET โ 11 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโดโโโโโโโฏ
Local Api Decisions:
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโฌโโโโโโโโโฌโโโโโโโโฎ
โ Reason โ Origin โ Action โ Count โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโผโโโโโโโโโผโโโโโโโโค
โ crowdsecurity/ssh-bf โ CAPI โ ban โ 13109 โ
โ crowdsecurity/ssh-slow-bf โ CAPI โ ban โ 4236 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโดโโโโโโโโโดโโโโโโโโฏ
Local Api Alerts:
โญโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโฎ
โ Reason โ Count โ
โโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโค
โ LePresidente/gitea-bf โ 36 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโฏ
We added some option to config.yaml
these should be documented in https://docs.crowdsec.net/docs/next/configuration/crowdsec_configuration maybe #384 already does?? but I think it just moving next -> 1.5.0
Maybe @blotus can confirm
https://doc.crowdsec.net/docs/user_guides/multiserver_setup
it'd really help if you clarified the docker environment variables you needed for a multi server setup on the agent end:
DISABLE_LOCAL_API=true
LOCAL_API_URL=http://host.ip.address:8080
AGENT_USERNAME=username
AGENT_PASSWORD="password"
The blog post you reference certainly helps, but there are a bunch of steps you'd do on bare metal that don't make sense in the context of containerisation, and would require editing the dockerfile to do.
The only other thing is that it's not immediately clear (at least it wasn't to me) that the LAPI and agent are part of the same container - when i was looking at the multi server setup initially, i was looking for an agent container I didn't need.
is it a really hard thing to simply write about how to configure grafana's connection to prometheus and then configure the board ?????????????
https://doc.crowdsec.net/docs/observability/prometheus#exploitation-with-prometheus-server--grafana
is very rediculous that "hei , i show how crowdsec can be observed,use grafana !" and nothing else then , what XXXX of the writer's thought ?????
I say , God , teache me how , OK??
Suggest defining "tainted" (in Crowdsec context) somewhere in the docs. I didn't know what it meant without asking in Discord. As long as it's findable via search (example) then users like me would be okay.
(but only "host" seems to work)
Hi
In terms of providing documentation for CrowdSec on 3. party platforms it's probably a good idea to link to well-maintained documentation rather than trying to keep our own maintained. Therefore I want to suggest that in the case of OpenWRT we link to https://openwrt.org/docs/guide-user/services/crowdsec from our documentation.
update doc for raspberry pi os which is outdated as of now raspberry pi os is 64bits, and is basically debian.
hub management
parser => reference (bad link)
enricher => parsers link (bad link)
decisions management => link vers cscli decisions pour le command usage
manual installation => Build docker image titre pas formatรฉ
crowdsec_configuration => max_age pas formattรฉ
cloudwatch => no source directives
datasources/monitoring => lien vers prometheus plutot que cscli ?
scenarios / introduction => lien vers leaky bucket cassรฉ
simulation: preciser si niveau agent ou niveau API
profiles: preciser si cest niveau lapi ou agent
observability/intro => lien de cscli vers cscli metrics
observability/dashboard => mettre un lien de cscli_dashboard command
use triple` bash for bash command
and don't put the $ in front
localapi/intro => ### Server pas formattรฉ , ### configuration pas formattรฉ
local api / intro => !!!tips pas formattรฉ
central APi / intro => point a la fin de phrase (scenario list)
get statistics and insights on your alerts compared
bouncer/intro => changer lien du hub par lien des bouncers dans le hub
in v1.1 => Bouncers => rename page Bouncers par Contributing
In new docs in https://docs.crowdsec.net/docs/user_guides/hub_mgmt/ (and everywhere else in that section) it isn't taken into consideration that cscli
is renamed to crowdsec-cli
on FreeBSD
It is unclear what happens when both configuration_path and configuration_dir are specified: if both are used, which one takes precedence, if any merging is done (don't think so)
acquisition_path
Path to the yaml file containing logs that needs to be read.
acquisition_dir
(>1.0.7) Path to a directory where each yaml is considered as a acquisition configuration file containing logs that needs to be read.
CentOS 8 is EOL. It would make sense to replace CentOS 8 with AlmaLinux in the file install.mdx
.
In https://packagecloud.io/crowdsec/crowdsec/install#bash-deb
For the /etc/apt/sources.list.d/crowdsec_crowdsec.list
section
It should be great to just have to copy-paste a command, @see Docker install doc for example:
echo \
"deb [arch=arm64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
for people dealing with go template, having a sample event at hand reach and pointers to online validation tools (such as https://camlittle.com/go-template-validation / https://github.com/apexskier/go-template-validation) might be useful !
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.