Comments (11)
After a while it worked and added the events to MISP. Thank you very much Alex for your help and patience.
from otx_2_misp.
Hi there!
Thanks for posting this. I forgot to check if the config file was uploaded.
Check the config folder, in the config.ini file, you have to add your configuration (api keys and misp urls) and you are ready to go.
Thanks!
Best,
Alex.
from otx_2_misp.
Thank you very much for your answer. In the config file I have modified it by
**[otx]
otx_server = https://otx.alienvault.com/user/soc_columbus/pulses
api_key = 'XXXXXXXXXXXXXXXXXXXXXXX'
[misp]
misp_url = 'http://XXX.XXX.XXX.XXX'
key = 'XXXXXXXXXXXXXXXXXXXXX**X'
and now when I run it again it generates the following output.
root@Servidor-MISP:/home/misp/otx_2_misp# python3 otx2misp.py
otx2misp.py:62: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:62: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:137: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:137: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:155: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:155: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:158: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['title'] is not '') and (ioc['title'] is not None):
otx2misp.py:160: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['description'] is not '':
otx2misp.py:162: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['role'] is not None and ioc['role'] is not '':
otx2misp.py:175: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if pulse['modified'] is not '':
[*] Searching for Pulses on OTX:
/home/misp/otx_2_misp
..........
from otx_2_misp.
Hi!
To avoid those warnings just use python 2.7. However, I will update the syntax to adapt it to the Python 3 version.
Despite the warnings, it seems that the script is running. Just remember to add keywords or ATT&CK techniques to the config files within the config folder.
from otx_2_misp.
Yes, the script is actually executed but even with the -m option the events are not sent to MISP
from otx_2_misp.
You have to use also the -a and/or -t option for filtering the content by keywords or ATT&CK technique. Otherwise, it won't send anything.
from otx_2_misp.
Thank you very much Alex for your answers. I have tried modifying the list of techniques and words, I use -m -a -t but in the end the script ends and it does not send anything to MISP. I have the API key and MISP URs but it still fails to send anything.
from otx_2_misp.
Could you send me the error?
from otx_2_misp.
This is all the output it generates.
root@Servidor-MISP:/home/misp/otx_2_misp# python3 otx2misp.py -m -t
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:157: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['title'] is not '') and (ioc['title'] is not None):
otx2misp.py:159: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['description'] is not '':
otx2misp.py:161: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['role'] is not None and ioc['role'] is not '':
otx2misp.py:174: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if pulse['modified'] is not '':
[] Searching for Pulses on OTX:
[] Checking if the pulses gathered gathered contain any ATT&CK Technique from your list.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3/dist-packages/simplejson/init.py", line 518, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "otx2misp.py", line 300, in
start_listen_otx()
File "otx2misp.py", line 286, in start_listen_otx
pulses = search_on_otx(api, False, True, max_days)
File "otx2misp.py", line 216, in search_on_otx
pulses = api.getall()
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 404, in getall
return self.walkapi(
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 385, in walkapi
return list(self.walkapi_iter(url, max_page=max_page, max_items=max_items, method=method, body=body))
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 366, in walkapi_iter
data = self.get(next_page_url)
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 185, in get
return self.handle_response_errors(response).json()
File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 917, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value]
<style type="text/css">
* {
max-width: 100%;
margin: 0
}
body, html {
height: 100%
}
body {
font-size: 12px;
font-family: 'ATT Aleck Sans', Helvetica, Arial, sans-serif;
background: #F8FAFB
}
@-webkit-keyframes spinner-spin {
from {
transform: rotate(0)
}
to {
transform: rotate(360deg)
}
}
@-moz-keyframes spinner-spin {
from {
transform: rotate(0)
}
to {
transform: rotate(360deg)
}
}
@-o-keyframes spinner-spin {
from {
transform: rotate(0)
}
to {
transform: rotate(360deg)
}
}
@-ms-keyframes spinner-spin {
from {
transform: rotate(0)
}
to {
transform: rotate(360deg)
}
}
@keyframes spinner-spin {
from {
transform: rotate(0)
}
to {
transform: rotate(360deg)
}
}
#spinner {
-webkit-animation: spinner-spin 1s steps(8) infinite;
-moz-animation: spinner-spin 1s steps(8) infinite;
-o-animation: spinner-spin 1s steps(8) infinite;
animation: spinner-spin 1s steps(8) infinite
}
#initLoading .message, #loading .message {
position: fixed;
bottom: 50%;
left: 50%;
text-align: center;
z-index: 9002;
font-size: 14px;
text-transform: uppercase;
color: #333333;
}
#initLoading .message > div, #loading .message > div {
position: relative;
top: 98px;
left: -150px;
background-color: #fff;
width: 300px;
padding: 30px 10px;
border-radius: 2px
}
#initLoading .message > div span, #loading .message > div span {
display: block;
border-top: 1px solid #DBDBDB;
padding: 20px 0 0;
margin: 30px 0 0
}
#initLoading .message > div i, #loading .message > div i {
font-size: 77px;
letter-spacing: normal
}
#initLoading .bg, #loading .bg {
position: fixed;
top: 0;
left: 0;
background-color: rgba(0, 0, 0, .92);
width: 100%;
height: 100%;
cursor: not-allowed;
z-index: 9001
}
</style>
<!--Google Tag Manager-->
<script>(function (w, d, s, l, i) {
w[l] = w[l] || [];
w[l].push({
'gtm.start': new Date().getTime(), event: 'gtm.js'
});
var f = d.getElementsByTagName(s)[0],
j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : '';
j.async = true;
j.src =
'https://www.googletagmanager.com/gtm.js?id=' + i + dl;
f.parentNode.insertBefore(j, f);
})(window, document, 'script', 'dataLayer', 'GTM-KVJVZTK');</script>
<title>AlienVault - Open Threat Exchange</title>
<meta name="description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
<meta id="viewport" name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<!-- Schema.org markup for Google+ -->
<meta itemprop="name" content="AlienVault - Open Threat Exchange">
<meta itemprop="description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
<meta content="/assets/images/otx-logo-twitter.png" itemprop="image">
<!-- Dont send referrer data, beyond main domain !-->
<meta name="referrer" content="origin">
<!-- Twitter Card data -->
<meta name="twitter:card" content="summary">
<meta name="twitter:site" content="@alienvault">
<meta name="twitter:text:title" content="AlienVault - Open Threat Exchange">
<meta name="twitter:text:description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
<meta name="twitter:creator" content="@alienvault">
<meta content="/assets/images/otx-logo-twitter.png" name="twitter:image">
<!-- Open Graph data -->
<meta property="og:title" content="AlienVault - Open Threat Exchange">
<meta property="og:type" content="article">
<meta property="og:url" content="">
<meta content="/assets/images/otx-logo-twitter.png" property="og:image">
<meta property="og:description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
<meta property="og:site_name" content="AlienVault Open Threat Exchange">
from otx_2_misp.
That's weird. First of all, replace the otx_server with https://otx.alienvault.com instead of https://otx.alienvault.com/user/soc_columbus/pulses.
Second, when executing the script try this order:
otx2misp.py -d 30 -a -t -m
This will gather the otx pulses from the last 30 days and they will be correlated with your keywords and techniques files. Finally they will be sent to the MISP instance in case of a match.
Hope this helps.
from otx_2_misp.
Alex, I execute the command as you indicate but it keeps searching. I tried different techniques and day intervals but keep looking and no progress.
root@Servidor-MISP:/home/misp/otx_2_misp# python3 otx2misp.py -d 30 -a -t -m
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:157: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['title'] is not '') and (ioc['title'] is not None):
otx2misp.py:159: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['description'] is not '':
otx2misp.py:161: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['role'] is not None and ioc['role'] is not '':
otx2misp.py:174: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if pulse['modified'] is not '':
[] Searching for Pulses on OTX:
[] Checking if the pulses gathered contain any keyword from your list.
from otx_2_misp.
Related Issues (2)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from otx_2_misp.