This will ease people to try this out. For example @anuraaga did this in coraza-wasm-filter.

cc @jptosso

I have multiple virtuals host on haproxy.
I wanted to know which is the right way to configure coraza-spoa with different configurations based on the backend or domain. To activate or deactivate the waf or activate only some rules.
I created multiple apps in the configuration file /etc/coraza-spoa/config.yaml, and tested the app name in /etc/haproxy/coraza.cfg args app=str(appname).
What is the right procedure?
Do you have a configuration example that can help me?

Currently the SPOA needs to restart to reload the rule set and update log file handles (log rotation currently doesn't work). Implement a reload mechanism, which doesn't disturb request processing & reloads the rule set.

• Originally mentioned in corazawaf/coraza#151
• Tableflip might be an option https://github.com/cloudflare/tableflip

Related to #92, is there any other fetch method that can be used? I tried to set a new header and try to get it into spoe app parameter without success.

Haproxy frontend
http-request add-header X-App open
SPOE
spoe-message coraza-req args app=req.hdr(X-App) id=unique-id src-ip=src...

And coraza logs:

{"level":"debug","ts":1699303677.986173,"msg":"application not found, using default","application":"","default":"open"}

(of course, I have two apps that I want to set up different)
UPDATE: The header is not being added in haproxy. Is this an expected behaviour when using spoe filters?

Hello,

According to existing docs, coraza-spoa should return "1" for a verdict that will be denied. Verdicts that are blocked by the crs ruleset yield a "-" however. It does return "0" on a clean request as shown below.

https://github.com/corazawaf/coraza-spoa
"The OWASP Coraza action is returned in a variable named "txn.coraza.fail". It contains the verdict of the request. If the variable is set to 1, the request will be denied."

however, using current/latest main branch, it seems it always returns "-" instead. Not sure if this is new intended beahviour, a configuration error on my side, or a bug.

haproxy log shows:

127.0.0.1:7583 [14/Jul/2023:19:28:34.635] test test/<NOSRV> 0/0/2/2/-1/-1/-1/2 403 513 - - PR-- 1/1/0/0/0 0/0 "GET /?x=/etc/passwd HTTP/1.1" 4ebdaca0-4c1d-49a1-9141-3abbe12493aa spoa-error: - waf-hit: - waf-data: -
127.0.0.1:7589 [14/Jul/2023:19:28:38.179] test test_backend/<NOSRV> 0/0/1/1/-1/-1/-1/1 200 80 - - LR-- 1/1/0/0/0 0/0 "GET / HTTP/1.1" 38b0b0e8-b04d-4588-a143-8aeb9fc0f6b1 spoa-error: - waf-hit: 0 waf-data: -

adjusting the haproxy config from commented to uncommented as shown bewlow will return to desired beahiour (blocking request/passing verdict)

    # Deny for Coraza WAF hits
    #http-request  deny if { var(txn.coraza.fail) -m int eq 1 }
    #http-response deny if { var(txn.coraza.fail) -m int eq 1 }
    http-request  deny if !{ var(txn.coraza.fail) -m int eq 0 }
    http-response deny if !{ var(txn.coraza.fail) -m int eq 0 }

coraza-spoa log shows:

{"level":"warn","ts":1689355984.7389846,"msg":"[client \"\\x7f\\x00\\x00\\x01\"] Coraza: Access denied (phase 1). Host header is a numeric IP address [file \"/data/modsecurity-crs4/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf\"] [line \"2398\"] [id \"920350\"] [rev \"\"] [msg \"Host header is a numeric IP address\"] [data \"127.0.0.100\"] [severity \"warning\"] [ver \"OWASP_CRS/4.0.0-rc1\"] [maturity \"0\"] [accuracy \"0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-protocol\"] [tag \"paranoia-level/1\"] [tag \"OWASP_CRS\"] [tag \"capec/1000/210/272\"] [tag \"PCI/6.5.10\"] [hostname \"\\x7f\\x00\\x00d\"] [uri \"/?x=/etc/passwd\"] [unique_id \"682016cb-3bcf-4e6c-a117-fee53bcb8647\"]\n"}
{"level":"error","ts":1689355984.739399,"msg":"[client \"\\x7f\\x00\\x00\\x01\"] Coraza: Access denied (phase 2). OS File Access Attempt [file \"/data/modsecurity-crs4/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf\"] [line \"4227\"] [id \"930120\"] [rev \"\"] [msg \"OS File Access Attempt\"] [data \"Matched Data: etc/passwd found within ARGS:x: /etc/passwd\"] [severity \"critical\"] [ver \"OWASP_CRS/4.0.0-rc1\"] [maturity \"0\"] [accuracy \"0\"] [tag \"application-multi\"] [tag \"language-multi\"] [tag \"platform-multi\"] [tag \"attack-lfi\"] [tag \"paranoia-level/1\"] [tag \"OWASP_CRS\"] [tag \"capec/1000/255/153/126\"] [tag \"PCI/6.5.4\"] [hostname \"\\x7f\\x00\\x00d\"] [uri \"/?x=/etc/passwd\"] [unique_id \"682016cb-3bcf-4e6c-a117-fee53bcb8647\"]\n"}
{"level":"error","ts":1689355984.7395911,"msg":"[client \"\\x7f\\x00\\x00\\x01\"] Coraza: Access denied (phase 2). Remote Command Execution: Unix Shell Code Found [file \"/data/modsecurity-crs4/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf\"] [line \"5056\"] [id \"932160\"] [rev \"\"] [msg \"Remote Command Execution: Unix Shell Code Found\"] [data \"Matched Data: etc/passwd found within ARGS:x: /etc/passwd\"] [severity \"critical\"] [ver \"OWASP_CRS/4.0.0-rc1\"] [maturity \"0\"] [accuracy \"0\"] [tag \"application-multi\"] [tag \"language-shell\"] [tag \"platform-unix\"] [tag \"attack-rce\"] [tag \"paranoia-level/1\"] [tag \"OWASP_CRS\"] [tag \"capec/1000/152/248/88\"] [tag \"PCI/6.5.2\"] [hostname \"\\x7f\\x00\\x00d\"] [uri \"/?x=/etc/passwd\"] [unique_id \"682016cb-3bcf-4e6c-a117-fee53bcb8647\"]\n"}
{"level":"error","ts":1689355984.7401335,"msg":"[client \"\\x7f\\x00\\x00\\x01\"] Coraza: Access denied (phase 2). Inbound Anomaly Score Exceeded (Total Score: 13) [file \"/data/modsecurity-crs4/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"10980\"] [id \"949110\"] [rev \"\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 13)\"] [data \"\"] [severity \"emergency\"] [ver \"OWASP_CRS/4.0.0-rc1\"] [maturity \"0\"] [accuracy \"0\"] [tag \"anomaly-evaluation\"] [hostname \"\\x7f\\x00\\x00d\"] [uri \"/?x=/etc/passwd\"] [unique_id \"682016cb-3bcf-4e6c-a117-fee53bcb8647\"]\n"}

not sure why host is encoded and not shown as 127.0.0100 as per request: curl -v http://127.0.0.100/?x\=/etc/passwd

did the verdict move to a different variable?

Currently the audit log producer fields are not populated, a binary version should be added so the fields provide accurate information which coraza-spoa version is in use:

tx = s.waf.NewTransaction()
tx.Waf.ProducerConnector = "coraza-spoa"
tx.Waf.ProducerVersion = "0.0.1 (Build f073fc2)

Hello,
the doc indicates the server "s1" on ip:port as below:
# /etc/haproxy/haproxy.cfg
....
backend coraza-spoa
mode tcp
server s1 127.0.0.1:9000

do you think it could makes sense to also use a .sock (/var/run/coaza-spoa.sock), it could be nice to have that instead of having dozens of open ports on the computer and it might be faster in transaction..

thanks,
bye Fred

It should be trivial to run an example with a single command. It helps to triage issues and reproduce users issues.

For example https://owasp.slack.com/archives/C03H4LQSHNZ/p1686216435947609

Hi,

we would like to use coraza-spoa in production but unfortunately as soon as we put production traffic on it, after a few hours the coraza-spoa daemon "goes nuts", starts using up all available CPU resources and shows a rapidly increasing memory consumption up to consuming tens of GBs of RAM. The system then starts swapping and becomes entirely unresponsive until the OOM killer finally kills the coraza-spoa daemon. And this happens quite frequently for us, like several times a day.

We run multiple haproxy load-balancing servers and what I observed is that often when coraza-spoa on one "goes nuts" the issue propagates also to further load-balancers, so the coraza daemon also on other servers starts excessively using memory and cpu almost at the same time. My conclusion therefore would be that the problem is externally triggered, so caused by certain (malicious?) requests that come in. Only this would explain the time-correlation of the OOM event among multiple load-balancer servers.

However, I went through our access logs at the respective times and couldn't find any obviously unusual or suspicious requests. There also were not unusually many requests at those times, so it's not an issue with general load or request rate. But still, there must be something apparently, so certain problematic requests that trigger the issue and cause the excessive resource use. I also checked the coraza logs but there is absolutely nothing logged when it happens. So whatever it is, coraza apparently didn't attempt to block those requests (or wasn't even able to get that far at all in processing the request).

So I don't really have a handle on how to get down to the root of the problem. I don't even know whether the issue is in coraza itself or the problem is unique to the haproxy SPOE integration. So if you have any ideas what might be the problem or how we could debug it more effectively, please let me know.

Thanks!
Michael

Coraza-spoa got error after some day running, this is log message:
{"level":"error","ts":1681351374.8902335,"msg":"failed to get transaction from cache","transaction_id":"cc401a49-0815-4a3e-a355-69a6b4b08006","error":"Key not found.","app":"haproxy"}
What is the cause? How can I fix it?

Right now this connector spins up a new zap logger on bootstrap. Spawning a new zap logger for a connector is overkill, more so when the transactions carry a logger that are controlled by the directives log level (so it is consistent) and it is efficient when it comes to allocation. Removing a dependency is usually a good practice.

Lines https://github.com/corazawaf/coraza-spoa/blob/main/internal/spoa.go#L175-L193 should be removed. In case spoe has a logger we could plug that into WAF setup (e.g. https://github.com/corazawaf/coraza-caddy/blob/main/logger.go), otherwise we should follow and use the default WAF logger.

From @jptosso

Now in the configuration file, we need to configure traffic forwarding fields like this:

spoe-message coraza-req
    args unique-id src method path query req.ver req.hdrs req.body
    event on-frontend-http-request

spoe-message coraza-res
    args unique-id status res.ver res.hdrs res.body
    event on-http-response

These fields are fixed. If the users configure the fields in the wrong order or configure fewer fields, coraza-spoa will fail to parse and report an error.

We should use the below configuration format, instead of being forced to send everything, the users will be able to send just what they need:

spoe-message coraza-req
    args id=unique-id src=src method=method path=path query=query version=req.ver headers=req.hdrs body=req.body
    event on-frontend-http-request

spoe-message coraza-res
    args id=unique-id status=status version=res.ver headers=res.hdrs body=res.body
    event on-http-response

And, HAProxy supports the above configuration format.

Coraza's logs look like:

{"level":"error","ts":1689775831.1240058,"msg":"[client \"\\xac\\x14\\x00\\x01\"] Coraza: Access denied (phase 2). Inbound Anomaly Score Exceeded (Total Score: 10) [file \"/etc/coraza-spoa/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"9504\"] [id \"949110\"] [rev \"\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 10)\"] [data \"\"] [severity \"emergency\"] [ver \"OWASP_CRS/4.0.0-rc1\"] [maturity \"0\"] [accuracy \"0\"] [tag \"anomaly-evaluation\"] [hostname \"\\xac\\x14\\x00\\x04\"] [uri \"/?x=/etc/passwd\"] [unique_id \"8a04f0d9-f52a-4c71-a6dd-ce567a10c3fa\"]\n"}

There are corrupted client and hostname fields.

Reproduces in dd5eb86.

cc @sts, @amsnek.

Hi, is there any official image we can reference on our deployments? Any plan to add or the recommendation is, and will continue to ourselves build and push container images on our own repositories?

coraza, coraza-caddy and coraza-proxy-wasm use magefiles instead of makefiles for the tasks (e.g. build, test, package, etc). We should do the same in this project for the sake of consistency and cohesion of the knowledge.

The audit log is currently not written for blocked responses.

The issue just appears for a whole request-response cycle. What currently works, is the audit log is already written in case the request gets filtered. In turn, if the request is clean, but the response should be filtered by a Response filter (sample), the block is not written to the audit log.

It looks like tx.ProcessLogging() is not invoked after tx.ProcessResponseBody().

https://github.com/corazawaf/coraza-spoa/blob/main/internal/response.go#L110

The issue will need similar treatment as the issue created for blocked requests: #11

I submitted a couple of enhancements to the sample configuration which I would find useful, when using coraza-spoa for the first time!

• #8
• #9

Hi,

you might want to chime into this discussion or prepare for it....

https://www.mail-archive.com/[email protected]/msg44680.html

Bernd

Audit log doesn't log a full connection tuple, client ip is already logged, but source port, destination ip & port are missing.

tx.ProcessConnection() in coraza-spoa is not populated with the required information: https://github.com/corazawaf/coraza-spoa/blob/main/internal/request.go#L68

We should add support for running FTW same as we do in coraza, coraza-caddy and coraza-proxy-wasm

A good reference can be found in https://github.com/corazawaf/coraza-caddy/tree/main/ftw

level=error msg="spoe error during first notify handle: handle notify: failed to get transaction from cache, transaction_id: , app: sample_app, error: Key not found."

I picked up the container as written here
Build the coraza-spoa image docker-compose build
Run haproxy, coraza-spoa and a mock server docker-compose up

when will I make a request for 4000
I receive an error message ( level=error msg="spoe error during first notify handle: handle notify: failed to get transaction from cache, transaction_id: 5dae59a1-687a-4ed1-b3d1-240e5a74d4b2, app: sample_app, error: Key not found.")
If you comment out the #spoe-message coraza-res block then everything works fine

ubuntu 22.04
Docker version 24.0.6

When testing the service locally, manual refresh often has detection failures

When running coraza-spoa in an environment like Kubernetes, handling SIGTERM becomes important for zero-downtime upgrades. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination

There are two main ways to deploy coraza-spoa: As a standalone pod, or as a sidecar in an haproxy pod.

Standalone coraza-spoa

When coraza-spoa is running standalone, it should respond to SIGTERM by processing existing messages but refusing new ones. In Kubernetes, since there can be slight delays between when a container enters the TERMINATING state and when traffic starts being sent to other replicas, it's also good to allow for a configurable delay before new messages start being denied (see --shutdown-grace-period here).

Sidecar coraza-spoa

When running coraza-spoa as a sidecar inside an existing haproxy pod, if you try to roll out a new version of that haproxy+coraza-spoa Pod, the old haproxy+coraza-spoa containers will be sent a SIGTERM signal simultaneously. Haproxy will typically begin draining existing connections, but those connections still need to be serviced by Coraza. In this situation, coraza-spoa should probably be configured to ignore SIGTERM entirely, or set a very long --shutdown-grace-period. That way Coraza still processes all of those remaining requests until the bitter end (SIGKILL). But you also don't want to delay shutdown of the pod for longer than necessary, so maybe there's something in SPOP where we can detect when there are no more connected haproxy instances? At that point it would be safe to shut down.

Somewhat related to #19 which might require some signal handling as well.

Integrations require e2e test to make sure basic functionality and that this won't break from the beginning. This checks must run on every PR to ensure no regressions.

One good example is https://github.com/corazawaf/coraza-proxy-wasm/tree/main/e2e which spins up an envoy with the proxy wasm plugin and runs basic checks about blocking.

Hello,

On a fresh install under Debian Bullseye I get this error :

/lib/systemd/system/coraza-spoa.service:48: Unknown key name 'inaccessiblepaths' in section 'Service', ignoring

Error is "inaccessiblepaths" instead of "InaccessiblePaths" in /lib/systemd/system/coraza-spoa.service line 48 :

inaccessiblepaths=-/bin/find

Thanks for your work !

The library that we use to speak SPOP appears to be unmaintained: criteo/haproxy-spoe-go#37

It also has a small public API and lack of interfaces that makes testing our code extremely difficult. For example, there's no way to construct our own spoe.Message in Go; you need to give it a real byte array. I hit this issue when trying to add some unit tests to coraza-spoa.

There is another more-active library Stefan found that I think we should use: https://github.com/negasus/haproxy-spoe-go

I'm about 50% done with a proof of concept PR to complete this. Just thought I'd file a tracking ticket.

Hey there, thank you for aiming to replace Trustwave ModSecurity. I have been following the instruction to setup coraza-spoa with haproxy and finally got it running. I am not seeing it block however.

The agent reports the below on every single request:

INFO[0000] spoe: listening on 127.0.0.1:9000 Argument 'version' not found Argument 'headers' not found

Haproxy log shows the following:

Jul 11 13:15:41 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-frontend-http-request> sid=17 st=0 0/0/0/0/1 1/1 0/0 0/27 Jul 11 13:15:41 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-frontend-http-request> sid=17 st=0 0/0/0/0/1 1/1 0/0 0/27 Jul 11 13:15:41 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-http-response> sid=17 st=0 0/0/0/0/0 1/1 0/0 0/28 Jul 11 13:15:41 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-http-response> sid=17 st=0 0/0/0/0/0 1/1 0/0 0/28 Jul 11 13:15:41 test haproxy[222]: 192.168.1.10:64257 [11/Jul/2023:13:15:41.954] test test_backend/s1 0/0/1/1/0/1/3/6 200 3383 - - ---- 1/1/0/0/0 0/0 "GET / HTTP/1.1" 1ec2bbea-a85f-44da-8e21-d7f382b7bc7b spoa-error: - waf-hit: 0 Jul 11 13:15:42 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-frontend-http-request> sid=19 st=0 0/0/0/0/0 1/1 0/0 0/29 Jul 11 13:15:42 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-frontend-http-request> sid=19 st=0 0/0/0/0/0 1/1 0/0 0/29 Jul 11 13:15:42 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-http-response> sid=19 st=0 0/0/0/0/0 1/1 0/0 0/30 Jul 11 13:15:42 test haproxy[222]: SPOE: [coraza-agent] <EVENT:on-http-response> sid=19 st=0 0/0/0/0/0 1/1 0/0 0/30 Jul 11 13:15:42 test haproxy[222]: 192.168.1.10:64257 [11/Jul/2023:13:15:41.961] test test_backend/s1 0/63/0/63/0/0/1/64 404 433 - - ---- 1/1/0/0/0 0/0 "GET /favicon.ico HTTP/1.1" 5bd81c35-f252-4c44-9ac1-22f17983b228 spoa-error: - waf-hit: 0

The haproxy config is defined as in the sample from the git project.

filter spoe engine coraza config /etc/coraza-spoa/coraza.cfg

and

http-request deny if { var(txn.coraza.fail) -m int eq 1 }
http-response deny if { var(txn.coraza.fail) -m int eq 1 }
http-request deny deny_status 504 if { var(txn.coraza.error) -m int gt 0 }
http-response deny deny_status 504 if { var(txn.coraza.error) -m int gt 0 }
use_backend test_backend

backend is:

backend test_backend
mode http
server s1 127.0.0.1:8080

backend coraza-spoa
mode tcp
server s1 127.0.0.1:9000

This:

curl http://serverip/\?x\=/etc/passwd

Results in the request being passed to apache which is listening on port 8080 same server.
Browsing to the url results in the same.

Corerules are loaded and configured as per example in docs.

Ref: https://github.com/corazawaf/coraza/tree/main/http/e2e

SPOA crashes if app is not set.

Test coraza.cfg:

spoe-message coraza-req
    args app=req.hdr(test-header) ...

2023-06-09 13:20:12 time="2023-06-09T10:20:12Z" level=info msg="spoe: listening on [::]:9000"
2023-06-09 13:20:20 panic: interface conversion: interface {} is nil, not string
2023-06-09 13:20:20 
2023-06-09 13:20:20 goroutine 13 [running]:
2023-06-09 13:20:20 coraza-spoa/internal.(*SPOA).processRequest(0xc001115380, {{0xc000596008?, 0xa?}, 0xc0007740a0?})
2023-06-09 13:20:20 /build/internal/request.go:60 +0x26b4
2023-06-09 13:20:20 coraza-spoa/internal.(*SPOA).Start.func1(0xc0013c0040)
2023-06-09 13:20:20 /build/internal/spoa.go:56 +0x95
2023-06-09 13:20:20 github.com/criteo/haproxy-spoe-go.(*conn).handleNotify(0xc000774000, {0x3, 0x1, 0x15, 0x1, {0xc000596007, 0xef, 0x3ff5}, {0xc000596000, 0x3ffc, ...}}, ...)
2023-06-09 13:20:20 /go/pkg/mod/github.com/criteo/[email protected]/notify.go:109 +0xea
2023-06-09 13:20:20 github.com/criteo/haproxy-spoe-go.(*conn).runWorker(0xc000774000, {0x3, 0x1, 0x15, 0x1, {0xc000596007, 0xef, 0x3ff5}, {0xc000596000, 0x3ffc, ...}}, ...)
2023-06-09 13:20:20 /go/pkg/mod/github.com/criteo/[email protected]/conn.go:153 +0x7b
2023-06-09 13:20:20 created by github.com/criteo/haproxy-spoe-go.(*conn).run
2023-06-09 13:20:20 /go/pkg/mod/github.com/criteo/[email protected]/conn.go:136 +0xbb0
2023-06-09 13:20:12 Loading 1 applications
2023-06-09 13:20:21 Loading 1 applications
2023-06-09 13:20:21 time="2023-06-09T10:20:21Z" level=info msg="spoe: listening on [::]:9000"
2023-06-09 13:20:22 time="2023-06-09T10:20:22Z" level=info msg="spoe session ending with: write frame: write tcp 172.25.0.3:9000->172.25.0.4:59074: write: connection reset by peer"
2023-06-09 13:20:22 time="2023-06-09T10:20:22Z" level=warning msg="spoe: error handling connection: write frame: write tcp 172.25.0.3:9000->172.25.0.4:59074: write: connection reset by peer"

To make it easier to configure all settings when running coraza-spoa in a docker container, all configuration settings should be exposed as CLI parameter.

Required Paramerters & Default configuration:

--bind 127.0.0.0.1:9000
--transaction-ttl 60000
--transaction_active_limit 100000
--error-log-file /dev/stdout
--server-log-file /var/log/coraza-spoa/server.log
--server-log-level debug
--debug-log-file /var/log/coraza-spoa/debug.log
--audit-log-file /var/log/coraza-spoa/audit.log

User required parameters should be the included seclang config:

--include /etc/coraza-spoa/coraza.conf
--include /etc/coraza-spoa/crs-setup.conf
--include /etc/coraza-spoa/rules/*.conf

Possible libraries:

• https://github.com/spf13/viper (cli & config file)
• https://github.com/spf13/cobra (cli only)

Hi,

I am trying to log to syslog but there seems to be no way to do that.
RSyslog is running directly on my server so it only needs to reference the unix socket that is open to the syslog server.

Any hints on how this can be done?

Best and thanks
Sven

I've the following rule in REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf:

SecRule REQUEST_HEADERS:Host "@pm test.domain.com"
"id:1005,
phase:1,
pass,
nolog,
ctl:ruleEngine=Off"

I want Coraza to not work for the site test.domain.com.

If I run the curl command: 'curl http://test.domain.com/\?x\=/etc/passwd' it's ok. It doesn't block anything. But if I run 'curl -A "" -H "User-Agent;" https://test.domain.com/' to send a request without the user-agent header, I get the log:

May 23 12:31:40 SERVER coraza-spoa[1167]: {"level":"info","ts":1684855900.383147,"msg":"[client "IP_ADDRESS"] Coraza: Warning. Empty User Agent Header [file "/etc/coraza-spoa/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "0"] [id "920330"] [rev ""] [msg "Empty User Agent Header"] [data ""] [severity "notice"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [hostname ""] [uri "/?"]\n"}

It seems that rules with ID 920xxx are executed despite my rule 1005. What can be happening? I did the same rule in Modsecurity and it works fine.

internal package has no tests :(

There has been some breaking changes in the coraza API and many improvements. We should catch up rather sooner than later with latest version of coraza and then revisit the issues to see if they are still occuring.

When I apply this exception:

SecRule REQUEST_HEADERS:User-Agent "@beginsWith myappi-sdk-go" \
   "id:1300,\
   phase:2,\
   pass,\
   nolog,\
   ctl:ruleRemoveTargetById=932237;REQUEST_HEADERS:User-Agent,\
   ctl:ruleRemoveByTag=attack-disclosure"

there is a runtime error:

time="2023-11-11T10:19:39+01:00" level=info msg="spoe: listening on [::]:9000"
panic: runtime error: slice bounds out of range [:-1]

goroutine 175 [running]:
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Transaction).GetField(0xc003517b48?, {0x0, 0x36, 0x0, {0xc00057c980, 0xa}, {0xc00364dbf0, 0x2, 0x2}})
  /go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/transaction.go:574 +0x405
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Rule).doEvaluate(0xc001aae530, 0xc003610c90?, 0xc001929800, 0xc000768ab0?, 0x0, 0x40cd45?)
  /go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/rule.go:234 +0xd3b
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Rule).Evaluate(...)
  /go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/rule.go:171
github.com/corazawaf/coraza/v3/internal/corazawaf.(*RuleGroup).Eval(0xc00021a010, 0x2, 0xc001929800)
  /go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/rulegroup.go:219 +0x358
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Transaction).ProcessRequestBody(0xc001929800)
  /go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/transaction.go:1003 +0x3a5
github.com/corazawaf/coraza-spoa/internal.(*SPOA).processRequest(0xc001670b58?, 0xc003517248?)
  /build/internal/spoa.go:316 +0xe1a
github.com/corazawaf/coraza-spoa/internal.(*SPOA).Start.func1(0xc004c2f740)
  /build/internal/spoa.go:49 +0xc5
github.com/criteo/haproxy-spoe-go.(*conn).handleNotify(0xc004f3d180, {0x3, 0x1, 0xee9, 0x1, {0xc0037e8009, 0x3af, 0x3ff3}, {0xc0037e8000, 0x3ffc, ...}}, ...)
  /go/pkg/mod/github.com/criteo/[email protected]/notify.go:109 +0xce
github.com/criteo/haproxy-spoe-go.(*conn).runWorker(0xc004f3d180, {0x3, 0x1, 0xee9, 0x1, {0xc0037e8009, 0x3af, 0x3ff3}, {0xc0037e8000, 0x3ffc, ...}}, ...)
  /go/pkg/mod/github.com/criteo/[email protected]/conn.go:153 +0x6c
created by github.com/criteo/haproxy-spoe-go.(*conn).run in goroutine 237
  /go/pkg/mod/github.com/criteo/[email protected]/conn.go:136 +0xaec

UPDATE: Even this rule throws error:

SecRule REQUEST_HEADERS:User-Agent "@pm myappi-sdk-go v1.5.1" \
   "id:1300,\
   phase:2,\
   pass,\
   nolog,\
   ctl:ruleRemoveTargetById=932237;REQUEST_HEADERS:User-Agent,\
   ctl:ruleRemoveByTag=attack-disclosure"

panic: runtime error: slice bounds out of range [:-1]

goroutine 13 [running]:
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Transaction).GetField(0xc001d3adf8?, {0x0, 0x36, 0x0, {0xc0001724c0, 0xa}, {0xc00174e930, 0x2, 0x2}})
	/go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/transaction.go:574 +0x405
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Rule).doEvaluate(0xc0024943d8, 0xc0004fb140?, 0xc000681c00, 0xc0021bef90?, 0x0, 0x40cd45?)
	/go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/rule.go:234 +0xd3b
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Rule).Evaluate(...)
	/go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/rule.go:171
github.com/corazawaf/coraza/v3/internal/corazawaf.(*RuleGroup).Eval(0xc000206010, 0x2, 0xc000681c00)
	/go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/rulegroup.go:219 +0x358
github.com/corazawaf/coraza/v3/internal/corazawaf.(*Transaction).ProcessRequestBody(0xc000681c00)
	/go/pkg/mod/github.com/corazawaf/coraza/[email protected]/internal/corazawaf/transaction.go:1003 +0x3a5
github.com/corazawaf/coraza-spoa/internal.(*SPOA).processRequest(0xc0022ca228?, 0xc001d3a948?)
	/build/internal/spoa.go:316 +0xe1a
github.com/corazawaf/coraza-spoa/internal.(*SPOA).Start.func1(0xc0013fe0c0)
	/build/internal/spoa.go:49 +0xc5
github.com/criteo/haproxy-spoe-go.(*conn).handleNotify(0xc0016a1ae0, {0x3, 0x1, 0x142a, 0x1, {0xc0016b2009, 0x3af, 0x3ff3}, {0xc0016b2000, 0x3ffc, ...}}, ...)
	/go/pkg/mod/github.com/criteo/[email protected]/notify.go:109 +0xce
github.com/criteo/haproxy-spoe-go.(*conn).runWorker(0xc0016a1ae0, {0x3, 0x1, 0x142a, 0x1, {0xc0016b2009, 0x3af, 0x3ff3}, {0xc0016b2000, 0x3ffc, ...}}, ...)
	/go/pkg/mod/github.com/criteo/[email protected]/conn.go:153 +0x6c
created by github.com/criteo/haproxy-spoe-go.(*conn).run in goroutine 53
	/go/pkg/mod/github.com/criteo/[email protected]/conn.go:136 +0xaec

I think until now it has been treated as an experiment but now it is being included in a quite used library and I think we should spend some time and effort making coraza-spoa stable to not to give the impression coraza is not usable.

Another option is to archive this if no maintainer is joining we archive this repo.

In any case I curated a very basic list of issues that should be tackled by the adjoint maintainer https://github.com/corazawaf/coraza-spoa/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22

Any thoughts @mac-chaffee @sts @jptosso @NullIsNot0, any volunteering?

UPDATE: curated a list of issues that should be tackled by maintainers

Currently the variable returned by coraza-spoa to inform HAProxy to perform a waf block, is called fail. The term fail is confusing and should be replaced.

• Additionally types.Interruption exposes more variables (Action, Data, RuleID & Status), which should also be exposed to HAProxy.
  • Other actions such as Redirect, Deny, Drop, could also be handled correctly in the example HAProxy configuration.

Looking at other modules:

• haproxy/spoa-modsecurity originally exposed a 'code' and blocked if > 0:
  https://github.com/haproxy/spoa-modsecurity/blob/master/README#L97

• corazawaf/coraza/examples/http-server checks whether it.Action == deny (https://github.com/corazawaf/coraza/blob/v2/master/examples/http-server/main.go#L99)

• corazawaf/caddy checks whether it.Status > 0
  https://github.com/corazawaf/coraza-caddy/blob/master/coraza.go#L211

Everything we have tested with haproxy and coraza spoa has been detected and marked/blocked accordingly including various mysql injection tests.

However, we have a website form that we know was sql injection vulnerable at one point before we sanitized it. We were hoping coreruleset would notice the sql injections in the name fields. Unfortunately, it doesn't appear to flag it. Here is the TCP flow:
coraza-req
sample_app
$00113247-d3e9-4777-b6ea-f53b32cd7e64
src-ip
src-port
dst-ip
dst-port
method
POST
path
(/membership-account/membership-checkout/
query
level=1
version
headers
content-length: 1352
cache-control: max-age=0
sec-ch-ua: "Not/A)Brand";v="99", "Microsoft Edge";v="115", "Chromium";v="115"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
upgrade-insecure-requests: 1
content-type: application/x-www-form-urlencoded
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7
sec-fetch-site: same-origin
sec-fetch-mode: navigate
sec-fetch-user: ?1
sec-fetch-dest: document
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
x-forwarded-for: 192.168.1.1
body
Elevel=1&checkjavascript=1&username=moredavetests&password=@##$@%@#$SDFw&password2=@#$@#%SDFS&bemail=moredavetest%40fakename.com&bconfirmemail=moredavetests%40fakename.com&fullname=&bfirstname=Dave+%27%3B+update+subscribers+set+username%3D%27joe%27+where+username%3D%27joe%27%3B--%2F*&blastname=Nobody&baddress1=47+Turnpike+Road&baddress2=&bcity=Pelham&bstate=nh&bzipcode=03076&bcountry=US&bphone=978-555-1212&sameasbilling=1&sfirstname=Dave+%27%3B+update+subscribers+set+username%3D%27joe%27+where+username%3D%27joe%27%3B--%2F*&slastname=Nobody&saddress1=47+Turnpike+Road&saddress2=&scity=Pelham&sstate=nh&szipcode=03076&sphone=978-555-1212&scountry=US&CardType=Visa&AccountNumber=4111111111111111&ExpirationMonth=01&ExpirationYear=2023&CVV=444&submit-checkout=1&javascriptok=1&g-recaptcha-response=03ADUVZwB73WbDQi2LE38bxBiprqIWbiQl6k2G3c4TbOEGcPsKzlMPCo2jbVfba2vid3d0KjgAasSX3QsApgjn874C6aqGugCWErJC7rX4CxFAxoAaMfA30SqTJi5wIPNtYq4kb-RZFCjUpI2df16pbdSvfftePVgGOqJLmhqvvAyZRoybUQXt9249coDVcrj7ZBdwuoHlGmh0XEsmrxmTFutSQsuJrGJ9odMOHmyRkvixRrEvXDWZkn_CRhOhG_Dmz0h--ZdpVOvTQM5JrjEXMdNN1v9tnv89STq0nk2dVjOSGqyAcaZ2M1PzPHt5Xn308pARbtK7xYE-i6jiOsa8PftvrZuWMl8nsGEjPYkpSNQmvPBiWKTcrjP1C9-RsDqHBqo3ZryOSMqlb2wbFftWaWMxymvCBBhKLQ5EaLsjdFzE50cFlHJhBfjYjsxYuSoGPZwRaruiHoOfFuQOrjUFXOwHeEVMRBOajNSiCJW6hrEM4H3ziAxB51vqXN8X9UsjoPVntkPfthz8EUxEhxb9K88hxr0b9VqnmA

You can see in the body section that there is clearly an attempt at mysql injection in the first name field:

&fullname=&bfirstname=Dave+%27%3B+update+subscribers+set+username%3D%27joe%27+where+username%3D%27joe%27%3B--

Some have reported that this was blocked when tested against various other systems including libmodsecurity.

Summary

It would be nice if we could log these events as json so that for example Kibana can digest them in that format. Other logs are consumed in this way and it would be good to follow the same standard if possible. I discussed this briefly with @jptosso on the OWASP Slack.

Basic example

Maybe we can add a option so that we can change the log format
https://github.com/corazawaf/coraza-spoa/blob/main/config.yaml.default#L29-L32

Example:

# The log level configuration, one of: debug/info/warn/error/panic/fatal
log_level: info
# The log file path
log_file: /dev/stdout
# The log format
log_format: json

I got referred to this block

The current output that needs to be converted can look like this:

[client "10.1.1.1"] Coraza: Warning. Inbound Anomaly Score Exceeded (Total Score: 5) [file "/etc/coraza-spoa/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "9415"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "emergency"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "anomaly-evaluation"] [hostname "10.1.1.2"] [uri "/test"] [unique_id "xyz"]

We could for example add the following to rule_match.go. Did some testing earlier.

func (mr MatchedRule) JSONErrorLog() map[string]interface{} {
	var msg string
	var data string

	for _, md := range mr.MatchedDatas_ {
		if md.Data() != "" {
			data = md.Data()
		}
		// Use 1st set message of rule chain as message
		if md.Message() != "" {
			msg = md.Message()
		}
		if data != "" && msg != "" {
			break
		}
	}
	if len(msg) > maxSizeLogMessage {
		msg = msg[:maxSizeLogMessage]
	}

	m := make(map[string]interface{})
	m["client"] = mr.ClientIPAddress_
	m["message"] = msg
	m["disruptive"] = mr.Disruptive_
	m["file"] = mr.Rule_.File()
	m["line"] = mr.Rule_.Line()
	m["id"] = mr.Rule_.ID()
	m["rev"] = mr.Rule_.Revision()
	m["msg"] = msg
	m["data"] = data
	m["severity"] = mr.Rule_.Severity()
	m["ver"] = mr.Rule_.Version()
	m["maturiy"] = mr.Rule_.Maturity()
	m["accuracy"] = mr.Rule_.Accuracy()

	m["tag"] = mr.Rule_.Tags()
	m["hostname"] = mr.ServerIPAddress_
	m["uri"] = mr.URI_
	m["unique_id"] = mr.TransactionID_
	m["phase"] = mr.Rule_.Phase()

	switch mr.DisruptiveAction_ {
	case DisruptiveActionAllow:
		m["message"] = "Coraza: Access allowed."
	case DisruptiveActionDeny:
		m["message"] = "Coraza: Access denied."
	case DisruptiveActionDrop:
		m["message"] = "Coraza: Access dropped."
	case DisruptiveActionPass:
		m["message"] = "Coraza: Warning."
	case DisruptiveActionRedirect:
		m["message"] = "Coraza: Access redirected."
	default:
		m["message"] = "Coraza: Custom disruptive action triggered."
	}

	return m
}

Test run

=== RUN   TestJSONErrorLogMessages/no_disruptive_action
{"level":"info","ts":1694003837.95571,"caller":"corazarules/rule_match_test.go:235","msg":"test","data":"","maturiy":0,"accuracy":0,"client":"","file":"","id":1234,"rev":"","msg":"","message":"Coraza: Custom disruptive action triggered.","disruptive":false,"severity":"emergency","ver":"","unique_id":"","hostname":"","uri":"","phase":0,"line":0,"tag":[]}

Motivation

Having logs in json is great and it is easy to parse.

Hello,

My test shows a memleak in coraza-spoa.

After coraza-spoa start :

 total          2049336K

While running ab -n 100000 -c 150 'https://XXXX/':

 total          2049336K
 total          2049336K
 total          2049336K
 total          2049336K
 total          2049336K
 total          2049336K
 total          2049336K
 total          2328872K #ab starts
 total          2397020K
 total          2535256K
 total          2672288K
 total          2740872K
 total          2809356K
....
 total          9585944K
 total          9585944K
 total          9586008K
 total          9587352K
....
 total         17622640K
 total         17691572K
 total         17896576K
...

Then it gets killed by OOMKiller.

PS: running multiple "small" ab run with sleep between runs does exactly the same : memory is growing up continuously as soon as coraza-spoa get requests.

Anything that I can do to help finding the memleak source ?

My config :
coraza-spoa based on experimental branch, commit id 82e0444. Build with go1.19.3 and run on a Debian 11.

coraza.yml:

bind: 0.0.0.0:9000

applications:
  sample_app:
    include:
      - /root/coraza-spoa/coraza.conf
      - /usr/local/coreruleset-v4.0-dev/crs-setup.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-901-INITIALIZATION.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-905-COMMON-EXCEPTIONS.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-911-METHOD-ENFORCEMENT.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-913-SCANNER-DETECTION.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-921-PROTOCOL-ATTACK.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-931-APPLICATION-ATTACK-RFI.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-933-APPLICATION-ATTACK-PHP.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-934-APPLICATION-ATTACK-GENERIC.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-944-APPLICATION-ATTACK-JAVA.conf
      - /usr/local/coreruleset-v4.0-dev/rules/REQUEST-949-BLOCKING-EVALUATION.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-950-DATA-LEAKAGES.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-951-DATA-LEAKAGES-SQL.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-952-DATA-LEAKAGES-JAVA.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-953-DATA-LEAKAGES-PHP.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-954-DATA-LEAKAGES-IIS.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-955-WEB-SHELLS.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-959-BLOCKING-EVALUATION.conf
      - /usr/local/coreruleset-v4.0-dev/rules/RESPONSE-980-CORRELATION.conf

    transaction_ttl: 60000
    transaction_active_limit: 100000

    log_level: info
    log_file: /var/log/coraza-spoa/server.log

coraza.conf (based on https://github.com/corazawaf/coraza/blob/v3/dev/coraza.conf-recommended):

SecRuleEngine On
SecRequestBodyAccess On
SecRule REQUEST_HEADERS:Content-Type "^(?:application(?:/soap\+|/)|text/)xml" \
     "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"
SecRule REQUEST_HEADERS:Content-Type "^application/json" \
     "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072
SecRequestBodyLimitAction Reject
SecRule REQBODY_ERROR "!@eq 0" \
"id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"

SecRule MULTIPART_STRICT_ERROR "!@eq 0" \
"id:'200003',phase:2,t:none,log,deny,status:400, \
msg:'Multipart request body failed strict validation: \
PE %{REQBODY_PROCESSOR_ERROR}, \
BQ %{MULTIPART_BOUNDARY_QUOTED}, \
BW %{MULTIPART_BOUNDARY_WHITESPACE}, \
DB %{MULTIPART_DATA_BEFORE}, \
DA %{MULTIPART_DATA_AFTER}, \
HF %{MULTIPART_HEADER_FOLDING}, \
LF %{MULTIPART_LF_LINE}, \
SM %{MULTIPART_MISSING_SEMICOLON}, \
IQ %{MULTIPART_INVALID_QUOTING}, \
IP %{MULTIPART_INVALID_PART}, \
IH %{MULTIPART_INVALID_HEADER_FOLDING}, \
FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"

SecRule MULTIPART_UNMATCHED_BOUNDARY "@eq 1" \
    "id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"

SecRule TX:/^COR_/ "!@streq 0" \
        "id:'200005',phase:2,t:none,deny,msg:'Coraza internal error flagged: %{MATCHED_VAR_NAME}'"

SecResponseBodyAccess On
SecResponseBodyMimeType text/plain text/html text/xml
SecResponseBodyLimit 524288
SecResponseBodyLimitAction ProcessPartial
SecTmpDir /tmp/
SecDataDir /tmp/
SecDebugLog /var/log/coraza-spoa/debug.log
SecDebugLogLevel 3
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:(5|4)(0|1)[0-9])$"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial
SecArgumentSeparator &
SecCookieFormat 0

coraza.cfg

[coraza]
spoe-agent coraza-agent
    messages coraza-req coraza-res
    option var-prefix coraza
    timeout hello      100ms
    timeout idle       2m
    timeout processing 10ms
    use-backend coraza-spoa
    log global

spoe-message coraza-req
    args app=str(sample_app) id=unique-id body=req.body headers=req.hdrs version=req.ver query=query path=path method=method src-ip=src
    event on-frontend-http-request

spoe-message coraza-res
    args app=str(sample_app) id=unique-id version=res.ver status=status headers=res.hdrs body=res.body
    event on-http-response

haproxy.cfg

backend coraza-spoa
    mode tcp
    balance roundrobin
    timeout connect 5s
    timeout server  3m
    server coraza-spoa1 127.0.0.1:9000
...
frontend front
    filter spoe engine coraza config /etc/haproxy/coraza.cfg
    http-request deny if { var(txn.coraza.fail) -m int eq 1 }
    http-response deny if { var(txn.coraza.fail) -m int eq 1 }
    http-request deny deny_status 504 if { var(txn.coraza.error) -m int gt 0 }
    http-response deny deny_status 504 if { var(txn.coraza.error) -m int gt 0 }

The audit log is currently not written, regardless of the configuration applied.

SecAuditEngine On
SecAuditLogRelevantStatus ".*"
SecAuditLog /var/log/coraza-spoa/audit.log
SecAuditLogFormat json

lsof -n -p $(pgrep coraza-spoa)
COMMAND     PID        USER   FD      TYPE             DEVICE SIZE/OFF   NODE NAME
coraza-sp 57494 coraza-spoa  cwd       DIR                8,1     4096      2 /
coraza-sp 57494 coraza-spoa  rtd       DIR                8,1     4096      2 /
coraza-sp 57494 coraza-spoa  txt       REG                8,1  7397744  27158 /usr/bin/coraza-spoa
coraza-sp 57494 coraza-spoa  mem       REG                8,1  1839792    976 /usr/lib/x86_64-linux-gnu/libc-2.31.so
coraza-sp 57494 coraza-spoa  mem       REG                8,1   149520    989 /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
coraza-sp 57494 coraza-spoa  mem       REG                8,1   464848   1489 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3
coraza-sp 57494 coraza-spoa  mem       REG                8,1   177928    972 /usr/lib/x86_64-linux-gnu/ld-2.31.so
coraza-sp 57494 coraza-spoa    0r      CHR                1,3      0t0      4 /dev/null
coraza-sp 57494 coraza-spoa    1u     unix 0x00000000175ce0f4      0t0 827181 type=STREAM
coraza-sp 57494 coraza-spoa    2u     unix 0x00000000175ce0f4      0t0 827181 type=STREAM
coraza-sp 57494 coraza-spoa    3w      REG                8,1     8979 262858 /var/log/coraza-spoa/server.log
coraza-sp 57494 coraza-spoa    4u  a_inode               0,13        0   9056 [eventpoll]
coraza-sp 57494 coraza-spoa    5r     FIFO               0,12      0t0 826132 pipe
coraza-sp 57494 coraza-spoa    6w     FIFO               0,12      0t0 826132 pipe
coraza-sp 57494 coraza-spoa    7w      CHR                1,3      0t0     10 /dev/null
coraza-sp 57494 coraza-spoa    9w      REG                8,1    61706 262860 /var/log/coraza-spoa/audit.log
coraza-sp 57494 coraza-spoa   10u     IPv4             826133      0t0    TCP 127.0.0.1:9000 (LISTEN)

This is caused by coraza-spoa currently missing to call tx.ProcessLogging()

Hello, i've been testing coraza-spoa and found that when a rule triggers, it's written to the config.yml file.
It also writes the events to the file i configured in /var/log.
ie: my config.yml after an event

The transaction cache lifetime(ms)

transaction_ttl: 60000

The transaction cache limit

transaction_active_limit: 100000
coraza2022/08/03 09:36:32 spoa.go:122: [client "192.168.1.10"] Coraza: Warning. Invalid character in request (outside of very strict set) [file "/etc/coraza-spoa/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "0"] [id "920273"] [rev ""] [msg "Invalid character in request (outside of very strict set)"] [data "ARGS:something=../../etc/passwd"] [severity "critical"] [ver "OWASP_CRS/3.3.2"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [tag "paranoia-level/4"] [hostname ""] [uri "/?something=../../etc/passwd"] [unique_id "90c441a0-ad38-4bb3-9e01-4cac0c939ce7"]

best regards
Alejandro

When sending a request without query arguments, coraza-spoa will by default log it as an error, when using the default configuration:

# coraza.cfg
spoe-message coraza-req
    args id=unique-id src-ip=src method=method path=path query=query version=req.ver headers=req.hdrs body=req.body
    event on-frontend-http-request

# error.log
{"level":"error","ts":"2022-05-22T08:42:08.165Z","caller":"logger/logger.go:75","msg":"invalid argument for http request query, string expected, got <nil>"}

Also the message could be enhanced, giving the user a hint where the error is happening, something like: spoe received invalid argument in spoe-message for: query. string expected, got <nil> would make it more clear that the error is happening between haproxy > spoe.

Hi. We are evaluating this tool and we noticed that it's crashing from time to time. Output from the log:

2023-03-01T15:05:33.577718925Z stdout F {"level":"error","ts":1677683133.5776238,"msg":"failed to get transaction from cache","transaction_id":"41f17f8c-f0f1-4d5a-b2kl1-03de1a64e9g0","error":"Key not found.","app":"testcoraza"}
2023-03-01T15:05:33.579874622Z stderr F panic: runtime error: invalid memory address or nil pointer dereference
2023-03-01T15:05:33.579991989Z stderr F [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x673bcb]
2023-03-01T15:05:33.580015811Z stderr F
2023-03-01T15:05:33.580072224Z stderr F goroutine 602 [running]:
2023-03-01T15:05:33.58034082Z stderr F github.com/corazawaf/coraza-spoa/internal.(*SPOA).processResponse(0xc000b4f800, {{0xc00214e00a?, 0x50?}, 0xc002169bd0?})
2023-03-01T15:05:33.580441486Z stderr F         /app/internal/response.go:86 +0x1eeb
2023-03-01T15:05:33.580541527Z stderr F github.com/corazawaf/coraza-spoa/internal.(*SPOA).Start.func1(0xc0021743c0)
2023-03-01T15:05:33.580629496Z stderr F         /app/internal/spoa.go:59 +0xa9
2023-03-01T15:05:33.580989239Z stderr F github.com/criteo/haproxy-spoe-go.(*conn).handleNotify(0xc001edbae0, {0x3, 0x1, 0x15b7, 0x2, {0xc00214e009, 0x305, 0x3ff3}, {0xc00214e000, 0x3ffc, ...}}, ...)
2023-03-01T15:05:33.581078331Z stderr F         /go/pkg/mod/github.com/criteo/[email protected]/notify.go:109 +0xea
2023-03-01T15:05:33.581489496Z stderr F github.com/criteo/haproxy-spoe-go.(*conn).runWorker(0xc001edbae0, {0x3, 0x1, 0x15b7, 0x1, {0xc002064009, 0x23c, 0x3ff3}, {0xc002064000, 0x3ffc, ...}}, ...)
2023-03-01T15:05:33.581587384Z stderr F         /go/pkg/mod/github.com/criteo/[email protected]/conn.go:162 +0x231
2023-03-01T15:05:33.58166379Z stderr F created by github.com/criteo/haproxy-spoe-go.(*conn).run
2023-03-01T15:05:33.581703092Z stderr F         /go/pkg/mod/github.com/criteo/[email protected]/conn.go:136 +0xbb0

Service is setup with default values as per documentation / examples.

when compiling coraza-spoa / crs 3.3.4 -> I get the following error when running

{"level":"fatal","ts":"2022-09-29T09:32:03.773Z","caller":"logger/logger.go:83","msg":"failed to compile rule (error parsing regexp: invalid or unsupported Perl syntax: `(?<`): FILES_NAMES|FILES \"@rx (?<!&(?:[aAoOuUyY]uml)|&(?:[aAeEiIoOuU]circ)|&(?:[eEiIoOuUyY]acute)|&(?:[aAeEiIoOuU]grave)|&(?:[cC]cedil)|&(?:[aAnNoO]tilde)|&(?:amp)|&(?:apos));|['\\\"=]\" \"id:920120,phase:2,block,t:none,t:urlDecodeUni,msg:'Attempted multipart/form-data bypass',logdata:'%{MATCHED_VAR}',tag:'application-multi',tag:'language-multi',tag:'platform-multi',tag:'attack-protocol',tag:'paranoia-level/1',tag:'OWASP_CRS',tag:'capec/1000/210/272',ver:'OWASP_CRS/3.3.4',severity:'CRITICAL',setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'\""}

this seems similar to: corazawaf/coraza#138

am I missing something?