Have the ability to whitelist ASNs and regex for DNS. about threatkb HOT 8 CLOSED

For DNS: https://majestic.com/reports/majestic-million

from threatkb.

@dcuellar322 what will the API interaction look like for this when we're posting a new C2 IP or C2 DNS artifact and it matches something on the whitelist. Drop it silently? Return an error?

from threatkb.

@dcuellar322 ideally if we post a list of artifacts, we'll get a response back for the ones that were rejected with some reason (duplicate, whitelisted, etc.)

from threatkb.

@pedramamini @dcuellar322

If you're talking about mass import then it would be a function of the import api. The current output of the import api is below. we will just add a "whitelisted" key to the artifacts dictionary and pass back the information that way.

{
"artifacts": {
"duplicates": [
{
"type": "IP",
"artifact": "1.1.1.1"
}
],
"committed": [
{
"addedTags": [],
"tags": [],
"ip": "200.0.2.2",
"created_user": {
"active": true,
"admin": true,
"registered_on": "2017-08-23T00:27:31",
"email": "[email protected]",
"id": 3
},
"owner_user": null,
"reference_link": null,
"id": 49,
"asn": "CLARO S.A., BR",
"expiration_timestamp": null,
"city": null,
"date_modified": "2017-09-14T18:11:11",
"reference_text": null,
"modified_user": {
"active": true,
"admin": true,
"registered_on": "2017-08-23T00:27:31",
"email": "[email protected]",
"id": 3
},
"comments": [],
"st": null,
"state": "Imported",
"removedTags": [],
"country": null,
"date_created": "2017-09-14T18:11:11",
"expiration_type": null
}
]
}
}

from threatkb.

@pedramamini how big is this list going to get? Trying to decide if we should store it in a flat file or the database.

from threatkb.

from threatkb.

@dcuellar322

We need a table to track whitelist items. It will need CRUD operations and fields:
created_time
modified_time
created_by_user
modified_by_user
whitelist_artifact
notes

Whitelist artifacts can be:
network CIDR (EX: 100.12.12.0/24)
a single IP
regex

Add a before-insert listener for c2dns and c2ip (See: app.models.yara_rules line 171 for example):
in the before-insert listener, iterate through the whitelist and do the following:
if the whitelist artifact is a network cidr, test to see whether the IP is in the CIDR
if the whitelist artifact is a single IP, test for an exact match
if the whitelist artifact is a regex, test it against the ip or dns entry

If there is a match, bail on the insert

from threatkb.

whitelist_artifact (2048)
notes (2048)

https://netaddr.readthedocs.io/en/latest/tutorial_01.html
try to instantiate IPAddress and IPNetwork
if they fail then it is a regex

from threatkb.