nim4 / dbshield Goto Github PK

View Code? Open in Web Editor NEW

668.0 54.0 161.0 4.66 MB

Database firewall written in Go

Home Page: http://nim4.github.io/DBShield/

License: MIT License

Go 97.46% HTML 1.25% CSS 0.72% JavaScript 0.51% Shell 0.06%

sql-injection database golang mysql db2 postgresql mariadb oracle

dbshield's Introduction

DBShield

Protects your data by inspecting incoming queries from your application server and rejecting abnormal ones.

How it works?

For example, this is how web server normally interacts with database server:

By adding DBShield in front of database server we can protect it against abnormal queries. To detect abnormal queries we first run DBShield in learning mode. Learning mode lets any query pass but it records information about it (pattern, username, time and source) into the internal database.

After collecting enough patterns we can run DBShield in protect mode. Protect mode can distinguish abnormal query pattern, user and source and take action based on configurations.

Demo

For demo, we are using sqlmap(automatic SQL injection and database takeover tool) to exploit the SQL injection vulnerability at user.php

In the first scenario, the sqlmap successfully exploits the SQL injection when web application connected directly to the database(MySQL), In the second scenario, we modify the user.php so DBShield gets between the web application and database which will drop the injection attempt and make sqlmap fail.

Sample Outputs

CLI

$ go run main.go
2016/10/15 16:25:31 [INFO]  Config file: /etc/dbshield.yml
2016/10/15 16:25:31 [INFO]  Internal DB: /tmp/model/10.0.0.21_postgres.db
2016/10/15 16:25:31 [INFO]  Listening: 0.0.0.0:5000
2016/10/15 16:25:31 [INFO]  Backend: postgres (10.0.0.21:5432)
2016/10/15 16:25:31 [INFO]  Protect: true
2016/10/15 16:25:31 [INFO]  Web interface on https://127.0.0.1:8070/
2016/10/15 16:25:33 [INFO]  Connected from: 10.0.0.20:35910
2016/10/15 16:25:33 [INFO]  Connected to: 10.0.0.21:5432
2016/10/15 16:25:33 [INFO]  SSL connection
2016/10/15 16:25:34 [DEBUG] Client handshake done
2016/10/15 16:25:34 [DEBUG] Server handshake done
2016/10/15 16:25:34 [INFO]  User: postgres
2016/10/15 16:25:34 [INFO]  Database: test
2016/10/15 16:25:34 [INFO]  Query: SELECT * FROM stocks where id=-1 or 1=1
2016/10/15 16:25:34 [WARN]  Pattern not found: [53 55 51 52 55 52 50 53 55 51 53 49 115 116 111 99 107 115 53 55 51 53 50 105 100 54 49 52 53 53 55 51 55 57 53 55 52 48 52 53 55 51 55 57 54 49 53 55 51 55 57] (SELECT * FROM stocks where id=-1 or 1=1)
2016/10/15 16:25:34 [WARN]  Dropping connection

Web Interface

Installation

Get it

$ go get -u github.com/nim4/DBShield

Then you can see help using "-h" argument:

$ $GOPATH/bin/DBShield -h
DBShield 1.0.0-beta3
Usage of DBShield:
  -a	get list of abnormal queries
  -c file
      config file (default "/etc/dbshield.yml")
  -h	show help
  -k	show parsed config and exit
  -l	get list of captured patterns
  -version
      show version

and run it with your configuration, like:

$ $GOPATH/bin/DBShield -c config.yml

see sample configuration file

⚠️ WARNING: Do NOT use default certificates in production environments!

Supports:

Database	Protect	SSL
DB2
MariaDB
MySQL
Oracle
Postgres

To Do

(Sorted by priority)

Improve documentation
Add Microsoft SQL Server
Add more command-line arguments
Get 90% test coverage
Support Oracle SSL

dbshield's People

Contributors

Stargazers

Watchers

Forkers

zhiephieforks firatozz folkevil netspar leventozen mari46 alaleramy parvaiz45 eskandar80 zhanglei cs24 songofhack codecrack3 fnzv 5up3rc tdr130 lee-0x00 wucaishi safe3 315634034 nobitanobi nonenotnull ibmendoza gophersgang cloudadic awesome-security awesome-golang butlife itsky365 liqingrikeiikyeong lewisliang82 reinhardhsu davidmr001 justintung hadoop835 zerolugithub koalacxr cntoby xxoxx youngqj fnet123 koolhazz 7758285 slimina yiyi99 mandarenmanman duzhanyuan yidanegolang post707 sdtm1016 ssdtfarm shengxinking secpersu naeimi well-it-wasnt-me qiyeboy matrixy uedev xieqiwen1993 webvul demarica dimpu47 amarcovich xymantec lx6317648 0thm4n3 johnjohnsp1 peterpt crazykid199 kakuye tonyhau secsecsec lenky0401 ivmachiavelli devopsmi v2hack nh-live zmyer flyr4nk pythonlearner7 jmptrader xieyixue cluo qiwihui praveenmunagapati winewei tom2jack qq927 flydol go-awesome stevenleeeeeee jiangrensheng cloudcache lukw00heck guoyu07 zhaoming0018 weixshen weiboyiyou oksbsb jdgcs

dbshield's Issues

DBShield run into i/o timeout when receiving response from mysql 5.7 as backend server

When test against MySQL 5.7 as backend server, DBShield failed into i/o timeout when read packets fromserver. But the server did send back packets to DBShield. But this doesn't recurrent for Mysql 5.5 or 5.6. I am not quit sure about it. Please help.

debug log:

2017/10/19 17:18:13 [INFO]  Config file: /path/to/dbshield.yml
2017/10/19 17:18:13 [INFO]  Listening: 0.0.0.0:13307
2017/10/19 17:18:13 [INFO]  Backend: mysql (10.10.10.57:3306)
2017/10/19 17:18:13 [INFO]  Protect: false
2017/10/19 17:18:13 [INFO]  Web interface on https://127.0.0.1:8070/
2017/10/19 17:18:20 [DEBUG] Connected from: 127.0.0.1:56552
2017/10/19 17:18:20 [DEBUG] Connected to: 10.10.10.57:3306
2017/10/19 17:18:20 [DEBUG] SSL connection
2017/10/19 17:18:20 [DEBUG] Client handshake done
2017/10/19 17:18:20 [DEBUG] Server handshake done
2017/10/19 17:18:20 [DEBUG] Username: root
2017/10/19 17:18:20 [DEBUG] Database: mysql_native_password
2017/10/19 17:18:20 [DEBUG] SSL bit: true
2017/10/19 17:18:20 [DEBUG] Query: show databases
2017/10/19 17:18:20 [DEBUG] action: learning
2017/10/19 17:18:20 [WARN]  data recevied: ���K��def�information_schemSCHEMATSCHEMATDatabase
                                                                                            SCHEMA_NAME
                                                                                !���information_schema��mysql��performance_schema���sys"
2017/10/19 17:18:30 [WARN]  read tcp 10.2.1.5:56553->10.10.10.57:3306: i/o timeout
2017/10/19 17:18:54 [DEBUG] Connected from: 127.0.0.1:56593
2017/10/19 17:18:54 [DEBUG] Connected to: 10.10.10.57:3306
2017/10/19 17:18:54 [DEBUG] SSL connection
2017/10/19 17:18:54 [DEBUG] Client handshake done
2017/10/19 17:18:54 [DEBUG] Server handshake done
2017/10/19 17:18:54 [DEBUG] Username: root
2017/10/19 17:18:54 [DEBUG] Database: mysql_native_password
2017/10/19 17:18:54 [DEBUG] SSL bit: true
2017/10/19 17:18:54 [DEBUG] Query: select * from first where name<1719
2017/10/19 17:18:54 [DEBUG] action: learning
2017/10/19 17:18:54 [WARN]  data recevied: �#3D000No database selected
2017/10/19 17:19:04 [WARN]  read tcp 10.2.1.5:56594->10.10.10.57:3306: i/o timeout

env

DBShield Version: 1.0.0-beta4

$ go version
go version go1.9.1 darwin/amd64

$ mysql --version
mysql  Ver 14.14 Distrib 5.7.19, for Linux (x86_64) using  EditLine wrapper

Drop action cause DBShield existing from command line

When using protect mode from command line, DBShield existed with errors. I think it may need to drop the connection without make itself existed.

DBShield version: commit b07fb6e
Go version: go version go1.9 darwin/amd64

> $GOPATH/bin/DBShield -c /etc/dbshield.yml

....

2017/09/21 18:52:14 [DEBUG] Query: select * from first where iii='中文'
2017/09/21 18:52:14 [WARN]  Invalid pattern
panic: Dropping connection

goroutine 10 [running]:
github.com/nim4/DBShield/dbshield/utils.ActionDrop(0xc42013802d, 0x26)
        /Users/qiwihui/Development/gopath/src/github.com/nim4/DBShield/dbshield/utils/action.go:5 +0x39
github.com/nim4/DBShield/dbshield/dbms.processContext(0xc42013802d, 0x26, 0x3b, 0xc420078324, 0x4, 0x39, 0xc42001438c, 0x10, 0x10, 0xc42007833e, ...)
        /Users/qiwihui/Development/gopath/src/github.com/nim4/DBShield/dbshield/dbms/utils.go:321 +0x18c
github.com/nim4/DBShield/dbshield/dbms.(*MySQL).Handler(0xc4200ba240, 0x0, 0x0)
        /Users/qiwihui/Development/gopath/src/github.com/nim4/DBShield/dbshield/dbms/mysql.go:91 +0x528
github.com/nim4/DBShield/dbshield.handleClient(0x1865d00, 0xc42000e568, 0xc420118090, 0xc420124008, 0x0)
        /Users/qiwihui/Development/gopath/src/github.com/nim4/DBShield/dbshield/utils.go:161 +0x350
created by github.com/nim4/DBShield/dbshield.mainListner
        /Users/qiwihui/Development/gopath/src/github.com/nim4/DBShield/dbshield/dbshield.go:162 +0x38a

sql statement: select * from first where iii='中文'; where there is no iii in my first table

[WARN] runtime error: slice bounds out of range

Hi nim4,

I found this issue on DBShield when my website query database on port 5000 that DBShield listening.
Do you know how to fix it or i configured something wrong ?

go run main.go

2016/12/29 09:45:41 [INFO] Config file: /etc/dbshield.yml
2016/12/29 09:45:41 [INFO] Listening: 0.0.0.0:5000
2016/12/29 09:45:41 [INFO] Backend: mysql (127.0.0.1:3306)
2016/12/29 09:45:41 [INFO] Protect: false
2016/12/29 09:45:41 [INFO] Web interface on https://XXX.XXX.XXX.XXX:8070/
2016/12/29 09:46:14 [WARN] runtime error: slice bounds out of range
2016/12/29 09:52:07 [WARN] runtime error: slice bounds out of range
2016/12/29 09:52:11 [WARN] runtime error: slice bounds out of range
2016/12/29 09:52:13 [WARN] runtime error: slice bounds out of range

/etc/dbshield.yml

##############################
### General               ###
#############################

# Operating mode
# possible values:
#       learning - add queries to training set (default)
#       protect - take action against abnormal queries
mode: learning

# DBMS type(default: mysql)
dbms: mysql

# IP address to listen on(default: 0.0.0.0)
listenIP: 0.0.0.0

# Port number to listen on (0 means dbms default port ex. 3306 for mysql)
listenPort: 5000

# IP address of the server
targetIP: 127.0.0.1

# Port number of the dbms (0 means dbms default port ex. 3306 for mysql)
targetPort: 3306

# TLS
tlsPrivateKey:  cert/server-key.pem
tlsCertificate: cert/server-cert.pem

# Directory to save internal database(default: in model directory under OS's temp directory)
dbDir: "/tmp/model/"

# Databasse synchronisation interval, 0 will force sync after each commit.
# Valid units are "ns", "us" (or "µs"), "ms", "s", "m" and "h". (default: 5s)
SyncInterval: 5s

#############################
### HTTP Service          ###
#############################
# Run http interface(default: yes)
http: yes

# Serve https (default: yes)
httpSSL: yes

# IP address to listen on(default: 127.0.0.1)
#httpIP: 127.0.0.1
httpIP: XXX.XXX.XXX.XXX

# Port number to listen on(default: 8070)
httpPort: 8070

# Password
httpPassword: admin01

#############################
### Protect               ###
#############################

# Action to take against abnormal requests
# possible values:
#       drop  - close the connection (default)
#       pass  - pass the query to server
action: drop

# Properties to check for abnormality
#
# possible values: (you can choose more than one - comma separated)
#       user     - username
#       source   - request sourece address
#
# Note: leave empty to avoid adiditional checks
additionalChecks:  user,source

#############################
### Logging               ###
#############################

# Log depth
#       1   warning
#       2   info
#       4   debug
# default: 3 (warning + info)
logLevel: 3

# Log path
# possible values:
#       /path/to/file
#       stdout
#       stderr (default)
logPath: stderr

Web Interface 404

DBShield -c dbshield.yml

404 page not found

My Config

##############################
### General               ###
#############################

# Operating mode
# possible values:
#       learning - add queries to training set (default)
#       protect - take action against abnormal queries
mode: learning

# DBMS type(default: mysql)
dbms: mariadb

# IP address to listen on(default: 0.0.0.0)
listenIP: 0.0.0.0

# Port number to listen on (0 means dbms default port ex. 3306 for mysql)
listenPort: 5000

# IP address of the server
targetIP: 127.0.0.1

# Port number of the dbms (0 means dbms default port ex. 3306 for mysql)
targetPort: 0

# TLS
tlsPrivateKey:  /etc/ssl/maxdb_key.pem
tlsCertificate: /etc/ssl/maxdb_certificate.pem

# Directory to save internal database(default: in model directory under OS's temp directory)
dbDir: "/tmp/model/"

# Connection timeout
# Valid units are "ns", "us" (or "µs"), "ms", "s", "m" and "h". (default: 5s)
timeout: 5s

# Databasse synchronisation interval, 0 will force sync after each commit.
# Valid units are "ns", "us" (or "µs"), "ms", "s", "m" and "h". (default: 5s)
SyncInterval: 5s

#############################
### HTTP Service          ###
#############################
# Run http interface(default: yes)
http: yes

# Serve https (default: yes)
httpSSL: no

# IP address to listen on(default: 127.0.0.1)
httpIP: 127.0.0.1

# Port number to listen on(default: 8070)
httpPort: 8070

# Password
httpPassword: ChangeMe

#############################
### Protect               ###
#############################

# Action to take against abnormal requests
# possible values:
#       drop  - close the connection (default)
#       pass  - pass the query to server
action: drop

# Properties to check for abnormality
#
# possible values: (you can choose more than one - comma separated)
#       user     - username
#       source   - request sourece address
#
# Note: leave empty to avoid adiditional checks
additionalChecks:  user,source

#############################
### Logging               ###
#############################

# Log depth
#       1   warning
#       2   info
#       4   debug
# default: 3 (warning + info)
logLevel: 3

# Log path
# possible values:
#       /path/to/file
#       stdout
#       stderr (default)
logPath: stderr

Was expecting something similar to the screenshot.

DBShield Version: 1.0.0-beta4
Go Version: go1.6.2 linux/amd64
OS Version: Ubuntu 16.04.2 LTS Xenial

What Oracle versions are supported

DBShield run into i/o timeout when mysql return data reaches a certain size

When try to handle return data from mysql, dbshield run into i/o timeout.

reproducing

In server A, create mysql db dbsheild and table first with

create table first (name varchar(50));

and initial data with

for i in {0..5000}; do mysql -h 127.0.0.1 -P3306 -uroot -ppassword dbshield -Bse "insert into first (name) values (${i})"; done

In server B, query data without DBShield, and it works fine even when I change 2,000 to 5,000:

mysql -h server-A-ip -P3306 -uroot -ppassword dbshield -Bse "select * from first where name<2000;"

In server B, setting up dbsheild and with default config from another server and modify follow config:

listenIP: 0.0.0.0
listenPort: 13308
targetIP: <server-A-ip>
targetPort: 3306
logLevel: 7

then it reprocuded.

$ mysql -h 127.0.0.1 -P13308 -uroot -ppassword dbshield -Bse "select * from first where name<1000;"

<return with expected data>

$ mysql -h 127.0.0.1 -P13308 -uroot -ppassword dbshield -Bse "select * from first where name<1716;"

<return with expected data>

$ mysql -h 127.0.0.1 -P13308 -uroot -ppassword dbshield -Bse "select * from first where name<1717;"

ERROR 2013 (HY000) at line 1: Lost connection to MySQL server during query

here is the debug log:

# ~/go/bin/DBShield -c ~/dbshield/dbshield.yml 
2017/09/18 09:19:59 [INFO]  Config file: /root/dbshield/dbshield.yml
2017/09/18 09:19:59 [INFO]  Listening: 0.0.0.0:13308
2017/09/18 09:19:59 [INFO]  Backend: mysql (<server-A-ip>:3306)
2017/09/18 09:19:59 [INFO]  Protect: false
2017/09/18 09:19:59 [INFO]  Web interface on https://127.0.0.1:8070/
2017/09/18 09:20:38 [DEBUG] Connected from: 127.0.0.1:38886
2017/09/18 09:20:38 [DEBUG] Connected to: <server-A-ip>:3306
2017/09/18 09:20:38 [DEBUG] Username: root
2017/09/18 09:20:38 [DEBUG] Database: dbshield
2017/09/18 09:20:38 [DEBUG] SSL bit: false
2017/09/18 09:20:38 [DEBUG] Query: select @@version_comment limit 1
2017/09/18 09:20:38 [DEBUG] Query: select * from first where name<1000
2017/09/18 09:20:56 [DEBUG] Connected from: 127.0.0.1:38892
2017/09/18 09:20:56 [DEBUG] Connected to: <server-A-ip>:3306
2017/09/18 09:20:56 [DEBUG] Username: root
2017/09/18 09:20:56 [DEBUG] Database: dbshield
2017/09/18 09:20:56 [DEBUG] SSL bit: false
2017/09/18 09:20:56 [DEBUG] Query: select @@version_comment limit 1
2017/09/18 09:20:56 [DEBUG] Query: select * from first where name<1717
2017/09/18 09:21:01 [WARN]  read tcp 127.0.0.1:13308->127.0.0.1:38892: i/o timeout

env

DBShield Version: 1.0.0-beta4

$ go version
go version go1.6.3 linux/amd64

$ uname -a
Linux qiwihui 4.10.8-041008-generic #201703310531 SMP Fri Mar 31 09:33:56 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ mysql --version 
mysql  Ver 14.14 Distrib 5.5.57, for debian-linux-gnu (x86_64) using readline 6.3

Will the project continue to be updated?

hi，
Will the project continue to be updated?