corazawaf / coraza Goto Github PK
View Code? Open in Web Editor NEWOWASP Coraza WAF is a golang modsecurity compatible web application firewall library
Home Page: https://www.coraza.io
License: Apache License 2.0
OWASP Coraza WAF is a golang modsecurity compatible web application firewall library
Home Page: https://www.coraza.io
License: Apache License 2.0
We need C wrappers in order to connect with other programs like Nginx and Apache.
We are going to copy the functions used by modsecurity but replacing msc_
with coraza_
.
A sample would be:
#include "coraza/coraza.h"
char main_rule_uri[] = "basic_rules.conf";
int main (int argc, char **argv)
{
Coraza *waf = NULL;
Parser *parser = NULL
Transaction *transaction = NULL;
waf = coraza_init();
parser = coraza_parser(waf);
coraza_parser_read_file(main_rule_uri);
transaction = coraza_new_transaction(waf);
coraza_process_connection(transaction, "127.0.0.1");
coraza_process_uri(transaction, "http://www.modsecurity.org/test?key1=value1&key2=value2&key3=value3&test=args&test=test");
coraza_process_request_headers(transaction);
coraza_process_request_body(transaction);
coraza_process_response_headers(transaction);
coraza_process_response_body(transaction);
return 0;
}
Instead of writing @ipMatch 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
we could just type @ipMatch private
SecRule REMOTE_ADDR "@ipMatch public" "id: 4, phase:1, log, deny, status:403,msg:'External access'"
Your's such great & useful software
Libinjection is not part of the main project and it must be removed, it should be cloned as a git submodule.
There are two ways it can be done, we could change
Read(reader io.Reader, mime string, storagePath string) error
// To
Read(reader io.Reader, mime string, storagePath string, fileMode fs.FileMode) error
Or we could create a new way to send parameters to the Read function, like
Read(options types.BodyBufferOptions) error
It must be solved before relasing v2
Required by many CRS tests...
if (Code >= 0 && Code <= 65535) {
hmap = utils.UnicodeMapping.At(Code)
}
It's already implemented but should we load unicode mappings like modsecurity from a file or maybe hardcode the data?
Rule variables are supposed to support regex but they are only supporting strings:
for example
SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/.*secret/ "..." "..."
See test 930121-3
ALSO ADD TESTS
Future versions of Coraza WAF must have a plugins architecture that implies a few changes to the current APIs, here is a list of the possible plugins controllers:
Additional utilities
map[int]interface{}
Possible plugins code
type TestPlugin struct{}
func (p *TestPlugin) OnLoad(plugin *engine.Plugins){
plugin.RegisterOperator("test", testOperator)
plugin.RegisterAction("test", testAction)
plugin.RegisterDirective("test", testDirective)
plugin.RegisterLoggingEngine("elastic", elasticEngine)
plugin.RegisterPersistanceEngine("memcached", memcachedEngine)
}
func (p *TestPlugin) OnExit(){
}
func (p *TestPlugin) Name() string{
return "TestPlugin"
}
We could add more hooks like OnTransactionCreate but I believe it requires more analysis.
I will start working on Coraza v2, if you are only using the high level APIs (Create WAF, tx, seclang and transaction manipulation) your migration will be smooth, because only low level APIs will be rewritten.
The idea of v2 is to extend compatibility with modsecurity v2 and remove CGO dependencies.
Other important features are:
Coraza v2 will only support highly tested features, every other feature will be included as a plugin until it can be merged to the main project. Only plugins without dependencies will be merged, some exceptions might be made.
tx.AddLogCb(cb)
)A @jwt operator should be useful to enforce the usage of JWT token, it should support pre-shared keys and certificates.
Example:
# will be triggered if expired or invalid signature
SecRule REQUEST_HEADERS:token "@jwt secret-token-key" "id: 1, deny, phase: 1, log, msg:'Invalid or expired JWT token'"
According to some reports, some rule projects use case insensitive directives, we might do that to for compatibility.
For v2
Rules should be marshaled to json so developers can create their own compilers
I have no more information by the moment but it requires testing.
GET /?var=%22in%20\u0076\u0061l\u0075e\u004F\u0066%3d HTTP/1.0
The previous payload is being read as a literal string
Help is required to improve coverage to 80%
There are two ways to generate coverage in Coraza:
A branch was created for this, improve-coverage
Log action was implemented for both audit log and rule log.
Two log variables were implemented, one for *Rule and another for *Transaction, in the case of rule, it tells a transaction whether to log the current rule if matched, meanwhile tx.Log will tell tx.ProcessLogging if the transaction should be a candidate for Audit Logging.
The variables were already added, now we have to test it is properly working.
Plugins are a features expected for later v1 releases, they should be able to register operators, actions, transformations, audit loggers and persistence engines.
We are going to use .so files compiled using go build -buildmode=plugin
.
Plugins should be importable like:
SecPluginImport /path/to/plugin.so
# or downloadable from Coraza servers:
SecPluginImport user/plugin-name@version
# report updates available on startup logs
SecPluginCheckUpdates On
Important: Coraza plugins won't support CGO, even pcre and libinjection, that's because they must be easy to share and non dependent from libc and pcre versions.
package pluginname
import (
engine"github.com/jptosso/coraza-waf"
)
type PluginName struct {}
func (PluginName) Name() string {
return "pluginName"
}
func (pn * PluginName) GetOperators() map[string]engine.Operator {
return map[string]Operator {
"TestOperator": SomeOperatorStruct
}
}
// and the same for actions, transformations, etc...
We could also use interfaces{} instead of multiple functions for each kind of plugin.
Another approach could be using a Waf instance to create objects:
package pluginname
import (
engine"github.com/jptosso/coraza-waf"
)
type CorazaPlugin struct {}
func (CorazaPlugin) Name() string {
return "pluginName"
}
func (pn * CorazaPlugin) Init(waf *engine.Waf) error {
waf.RegisterOperator("operatorName", SomeOperator)
waf.RegisterTransformation("someTransformation", SomeTransaformation)
return nil
}
I think compiling plugins may be hard for some system administrators, that's why a compiling tool should be developed.
A testing package must be provided to help developers test and benchmark their plugins.
There might be a repository like https://plugins.coraza.io/ with a plugin search engine and the options to download the .so files or find the sourcecode.
Full testing coverage is required to assure code quality.
I tried to include OWASP CRS but I am getting this error.
{"level":"error","ts":1637334926.4698696,"msg":"[Parser] [Line 0] Failed to compile rule (Invalid variable URI_PARSER_ERROR): URI_PARSER_ERROR \"!@eq 0\" \"id:'200006', phase:1,t:none,log,deny,status:400,msg:'Failed to parse request url.',logdata:'Failed to parse request url',severity:2\"","line":0} {"level":"error","ts":1637334926.4699538,"msg":"Invalid variable URI_PARSER_ERROR","path":"./rules/crs2.conf"}
Config Lines:
SecRule URI_PARSER_ERROR "!@eq 0" \
"id:'200006', phase:1,t:none,log,deny,status:400,msg:'Failed to parse request url.',logdata:'Failed to parse request url',severity:2"
waf := coraza.NewWaf()
parser, _ := seclang.NewParser(waf)
parser.FromFile("./rules/crs2.conf")
wget https://raw.githubusercontent.com/jptosso/coraza-waf/master/coraza.conf-recommended
git clone https://github.com/coreruleset/coreruleset
cat coraza.conf-recommended coreruleset/crs-setup.conf.example coreruleset/rules/*.conf > crs2.conf
JSON is now fully supported and tested.
{"a": "b", "b": [1,2,3], "c": {"a": "b"}}
The previous json will be translated as the following ARGS:
Somehow tests sometime fails for golang 1.14: https://travis-ci.org/github/jptosso/coraza-waf/jobs/771953316
Can't figure the issue but the problem is with the actions.yaml tests
From: https://github.com/microcosm-cc/bluemonday
Implementation must be discussed, maybe a policy file or a policy string
https://github.com/SpiderLabs/ModSecurity/tree/v3/master/test/test-cases/regression
They should be transformed to go-ftw format
👋 I am currently making a coraza-waf
extension for https://github.com/caddyserver/caddy, which has been basically completed, but found that all requests after the phase2 process failed to pass the inspection.
How should I enable CRS correctly?
c.ctx = ctx
c.logger = ctx.Logger(c)
defer c.logger.Sync()
c.Waf = engine.NewWaf()
c.Waf.Init()
c.Waf.InitLogger()
crs, err := crs.NewCrs(c.Waf)
if err != nil {
return err
}
crs.TemplateDir = "/home/naiba/Desktop/coraza-waf/docs/crs/rules"
err = crs.Build()
if err != nil {
return err
}
l := len(c.Waf.Rules.GetRules())
if l == 0 {
return errors.New("No rules found")
}
if l < 500 {
return fmt.Errorf("Not enough CRS rules, found %d", l)
}
func (h *HTTPHandler) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error {
tx := h.Waf.NewTransaction()
err := tx.ParseRequestObjectHeaders(r)
if err != nil {
h.logger.Error("ParseRequestObjectHeaders", zap.Any("err", err))
return err
}
if tx.ExecutePhase(1) {
w.Write([]byte(tx.GetErrorPage()))
return errors.New("phase1")
}
err = tx.ParseRequestObjectBody(r)
if err != nil {
h.logger.Error("ParseRequestObjectBody", zap.Any("err", err))
return err
}
if tx.ExecutePhase(2) {
h.logger.Info("MatchedRules", zap.Any("disrupted", tx.Disrupted), zap.Any("rules", tx.MatchedRules))
w.Write([]byte(tx.GetErrorPage()))
return errors.New("phase2")
}
// Continue down the handler stack
if err := next.ServeHTTP(w, r); err != nil {
return err
}
return nil
}
Coraza WAF should be easy to integrate with any existing technology or language, so best way to achieve this should be an RPC interface.
JSON or binary? Golang does not support packing and it could be hard to achieve a binary structure, by the other hand, binary packages are hard to understand and less developer friendly. Maybe we could have both RPC interfaces.
Benchmark report must contain comparison for two use cases using Coreruleset:
Both must be implemented as a library and the result must contain the ms average for 100.000 requests splitted between both use cases.
Library | Version | Compilation | Phase 1 | Phase 2 | Phase 3 | Phase 4 | Phase 5 |
---|---|---|---|---|---|---|---|
Coraza | v1.0 | 100ms | 100ms | 100ms | 100ms | 100ms | 100ms |
Modsecurity | v3.x | 100ms | 100ms | 100ms | 100ms | 100ms | 100ms |
Internal wrappers were created in order to achieve this, for example:
type ruleActionParams struct {
// The name of the action, used for logging
Name string
// The action to be executed
Function RuleAction
}
Audit logger was splitted between:
Writters and formatters can be extended as plugins
Recaptcha operator is an idea to add recaptcha support, the syntaxis would look like:
#Set recaptcha key
SecRecaptchaKey key12345
#Set recaptcha template
SecRecaptchaTemplate /etc/coraza-waf/templates/recaptcha.html
# Initialize ip collection
SecAction "id:1, initcol:ip=%{remote_addr}, phase: 1"
# Check if the captcha was
SecRule REQUEST_METHOD "POST" "id:2, phase: 1, chain"
SecRule IP:recaptcha "1" "chain"
SecRule ARGS_POST:recaptcha_token "@recaptcha" "setvar:ip.recaptcha=0"
# If challenge failed, recaptcha will render the SecRecaptchaTamplate
SecRule IP:recaptcha "1" "id: 2, recaptcha, phase: 1"
nginx - > Coraza WAF -> Application
What are your plans with this integration?
Send NGINX traffic to a Coraza using a subrequest (njs or lua)?
There are many bypasses for ip addresses, like http://⑯⑨。②⑤④。⑯⑨。②⑤④
, there should be a method to detect if the string is an ip address, we could use transformations or an operator.
Must be discussed
In order to remove CGO from Coraza we have to remove two dependencies, libinjection and libpcre.
Libpcre
Libpcre can be replaced with golang regexp (re2). A wrapper has been created under utils/regex, it will pick between the CGO version (libpcre) and non CGO (RE) based on the flag CGO_ENABLED.
Libinjection
I'm currently working on a port, libinjection-go, in the meantime it will be a different repository but the final version is going to be embedded with Coraza as it won't be maintained as a importable package.
Coraza does actually support most features required by CRS, we need:
ctl action
jsdecode transformation
cssdecode transformation
OWASP CRS test suite will always fail as it relies on some behaviours provided by Apache and Nginx that the GOLANG reverse proxy fixes internally.
What's your plan about this?
I am the maintainer of ingress-nginx and one of the pain points is the WAF (ModSecurity) feature.
I'm always researching for alternatives that allow me to decouple the feature from the ingress controller and use a sidecar or maybe a socket (Shared Volume) in the same deployment?
Coverage went down for coraza v2, it must be brought back to 80. This issue is here to keep track of the coverage levels.
Something like SecGeoAutoload DATABASE APIKEY
, it should connect to maxmind site and download the proper database.
We must check licenses first, maybe they don't want us to do that.
Coraza WAF can be exported to C using CGO exports. We are going to build an Apache module that makes uses of this features. The project will be considered ready once it is fully executing the OWASP CRS test suite using go-ftw and the Coraza Apache module.
tx.Request|ResponseBodyBuffer.Write
and a write from file functionI have created some tests under https://github.com/jptosso/coraza-cexport, apxs is required to compile the module
Status of unsupported rules:
SecRule REQUEST_BASENAME "@detectSQLi" "id:942101,phase:1,block,capture,t:none,t:utf8toUnicode,t:urlDecodeUni,t:removeNulls,msg:'SQL Injection Attack Detected via libinjection',logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',tag:'application-multi',tag:'language-multi',tag:'platform-multi',tag:'attack-sqli',tag:'OWASP_CRS',tag:'capec/1000/152/248/66',tag:'PCI/6.5.2',tag:'paranoia-level/3',ver:'OWASP_CRS/3.4.0-dev',severity:'CRITICAL',setvar:'tx.sql_injection_score=+%{tx.critical_anomaly_score}',setvar:'tx.anomaly_score_pl3=+%{tx.critical_anomaly_score}'"
POST /[email protected]"%20sleep(10.to_i)%20" HTTP/1.0
Host: localhost
{"level":"debug","ts":1637549753.886731,"msg":"Evaluate rule operator","txid":"zqs9Bq9IKMZFBO0bedK","rule":942101,"event":"EVALUATE_RULE_OPERATOR","operator":"@detectSQLi","data":"[email protected]\\\" sleep(10.to_i) \\","variable":"REQUEST_BASENAME","key":"","value":"[email protected]\\\" sleep(10.to_i) \\","result":false}
Match expected
The chain of transformations is transforming "%20sleep(10.to_i)%20"
to \\\" sleep(10.to_i) \\
and libinjection is failing to test it, it requires more debug.
See test 942101-6
The issue: If you manage thousands of routes it's not resources effective to have thousands of WAF instances.
The solution: Make "recyclable" WAF instances.
How to achieve it:
Maybe we could use SecWebAppId to define an "instance ID" and register this waf instance in a hashmap ``map[string]*engine.Waf``` under the skipper package.
For example:
#routes.yaml
route1:
Path("/1) ->
corazaWAF("/etc/coraza-waf/profiles/default/rules.conf")
route2:
Path("/2") ->
corazaWAF("/etc/coraza-waf/profiles/default/rules.conf")
#/etc/coraza-waf/profiles/default/rules.conf
SecWebAppId instance1
In this case route1 and route2 would share the same instance because of their secwebappid.
This idea might work but requires more analysis
Coraza v1 has achieved 90%+ compatibility but a complete redesign on Coraza v2 broke most of the compatibility. In order to get 100% compatibility we must find the issues that are breaking CRS.
Note: the coraza testsuite includes plugins for PCRE and libinjection, so please have them installed.
SecAction "id:900005,\
phase:1,\
nolog,\
pass,\
ctl:ruleEngine=DetectionOnly,\
ctl:ruleRemoveById=910000,\
setvar:tx.paranoia_level=4,\
setvar:tx.crs_validate_utf8_encoding=1,\
setvar:tx.arg_name_length=100,\
setvar:tx.arg_length=400,\
setvar:tx.combined_file_sizes=65535"
git clone https://github.com/jptosso/coraza-testsuite
go get github.com/jptosso/coraza-waf/v2@LAST_REVISION
#Compile or run:
go run *.go run -d ../coreruleset/tests/regression -r ../coreruleset/rules/rules.conf
The current results are posted in the first comment.
collection.AddUnique
was not working, failing to create variables like REQUEST_COOKIES_NAMES, REQUEST_HEADERS_NAMES, RESPONSE_HEADERS_NAMES, ARGS_NAMES, etcA declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.