AttributeError: 'ConfigParser' object has no attribute 'NoOptionError' about otx_2_misp HOT 11 CLOSED

After a while it worked and added the events to MISP. Thank you very much Alex for your help and patience.

from otx_2_misp.

Hi there!

Thanks for posting this. I forgot to check if the config file was uploaded.

Check the config folder, in the config.ini file, you have to add your configuration (api keys and misp urls) and you are ready to go.

Thanks!

Best,
Alex.

from otx_2_misp.

Thank you very much for your answer. In the config file I have modified it by

**[otx]
otx_server = https://otx.alienvault.com/user/soc_columbus/pulses
api_key = 'XXXXXXXXXXXXXXXXXXXXXXX'

[misp]
misp_url = 'http://XXX.XXX.XXX.XXX'
key = 'XXXXXXXXXXXXXXXXXXXXX**X'

and now when I run it again it generates the following output.

root@Servidor-MISP:/home/misp/otx_2_misp# python3 otx2misp.py
otx2misp.py:62: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:62: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:137: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:137: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:155: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:155: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:158: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['title'] is not '') and (ioc['title'] is not None):
otx2misp.py:160: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['description'] is not '':
otx2misp.py:162: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['role'] is not None and ioc['role'] is not '':
otx2misp.py:175: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if pulse['modified'] is not '':
[*] Searching for Pulses on OTX:
/home/misp/otx_2_misp

..........

from otx_2_misp.

Hi!

To avoid those warnings just use python 2.7. However, I will update the syntax to adapt it to the Python 3 version.

Despite the warnings, it seems that the script is running. Just remember to add keywords or ATT&CK techniques to the config files within the config folder.

from otx_2_misp.

Yes, the script is actually executed but even with the -m option the events are not sent to MISP

from otx_2_misp.

You have to use also the -a and/or -t option for filtering the content by keywords or ATT&CK technique. Otherwise, it won't send anything.

from otx_2_misp.

Thank you very much Alex for your answers. I have tried modifying the list of techniques and words, I use -m -a -t but in the end the script ends and it does not send anything to MISP. I have the API key and MISP URs but it still fails to send anything.

from otx_2_misp.

Could you send me the error?

from otx_2_misp.

This is all the output it generates.

root@Servidor-MISP:/home/misp/otx_2_misp# python3 otx2misp.py -m -t
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:157: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['title'] is not '') and (ioc['title'] is not None):
otx2misp.py:159: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['description'] is not '':
otx2misp.py:161: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['role'] is not None and ioc['role'] is not '':
otx2misp.py:174: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if pulse['modified'] is not '':
[] Searching for Pulses on OTX:
[] Checking if the pulses gathered gathered contain any ATT&CK Technique from your list.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3/dist-packages/simplejson/init.py", line 518, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "otx2misp.py", line 300, in
start_listen_otx()
File "otx2misp.py", line 286, in start_listen_otx
pulses = search_on_otx(api, False, True, max_days)
File "otx2misp.py", line 216, in search_on_otx
pulses = api.getall()
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 404, in getall
return self.walkapi(
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 385, in walkapi
return list(self.walkapi_iter(url, max_page=max_page, max_items=max_items, method=method, body=body))
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 366, in walkapi_iter
data = self.get(next_page_url)
File "/usr/local/lib/python3.8/dist-packages/OTXv2.py", line 185, in get
return self.handle_response_errors(response).json()
File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 917, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value]



<style type="text/css">
* {
max-width: 100%;
margin: 0
}

            body, html {
                    height: 100%
            }

            body {
                    font-size: 12px;
                    font-family: 'ATT Aleck Sans', Helvetica, Arial, sans-serif;
                    background: #F8FAFB
            }

            @-webkit-keyframes spinner-spin {
                    from {
                            transform: rotate(0)
                    }
                    to {
                            transform: rotate(360deg)
                    }
            }

            @-moz-keyframes spinner-spin {
                    from {
                            transform: rotate(0)
                    }
                    to {
                            transform: rotate(360deg)
                    }
            }

            @-o-keyframes spinner-spin {
                    from {
                            transform: rotate(0)
                    }
                    to {
                            transform: rotate(360deg)
                    }
            }

            @-ms-keyframes spinner-spin {
                    from {
                            transform: rotate(0)
                    }
                    to {
                            transform: rotate(360deg)
                    }
            }

            @keyframes spinner-spin {
                    from {
                            transform: rotate(0)
                    }
                    to {
                            transform: rotate(360deg)
                    }
            }

            #spinner {
                    -webkit-animation: spinner-spin 1s steps(8) infinite;
                    -moz-animation: spinner-spin 1s steps(8) infinite;
                    -o-animation: spinner-spin 1s steps(8) infinite;
                    animation: spinner-spin 1s steps(8) infinite
            }

            #initLoading .message, #loading .message {
    position: fixed;
    bottom: 50%;
    left: 50%;
    text-align: center;
    z-index: 9002;
    font-size: 14px;
    text-transform: uppercase;
    color: #333333;
}

            #initLoading .message > div, #loading .message > div {
                    position: relative;
                    top: 98px;
                    left: -150px;
                    background-color: #fff;
                    width: 300px;
                    padding: 30px 10px;
                    border-radius: 2px
            }

            #initLoading .message > div span, #loading .message > div span {
                    display: block;
                    border-top: 1px solid #DBDBDB;
                    padding: 20px 0 0;
                    margin: 30px 0 0
            }

            #initLoading .message > div i, #loading .message > div i {
                    font-size: 77px;
                    letter-spacing: normal
            }

            #initLoading .bg, #loading .bg {
                    position: fixed;
                    top: 0;
                    left: 0;
                    background-color: rgba(0, 0, 0, .92);
                    width: 100%;
                    height: 100%;
                    cursor: not-allowed;
                    z-index: 9001
            }
    </style>

    <!--Google Tag Manager-->
    <script>(function (w, d, s, l, i) {
  w[l] = w[l] || [];
  w[l].push({
      'gtm.start': new Date().getTime(), event: 'gtm.js'
  });
  var f = d.getElementsByTagName(s)[0],
      j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : '';
  j.async = true;
  j.src =
      'https://www.googletagmanager.com/gtm.js?id=' + i + dl;
  f.parentNode.insertBefore(j, f);

})(window, document, 'script', 'dataLayer', 'GTM-KVJVZTK');</script>

    <title>AlienVault - Open Threat Exchange</title>
    <meta name="description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
    <meta id="viewport" name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">

    <!-- Schema.org markup for Google+ -->
    <meta itemprop="name" content="AlienVault - Open Threat Exchange">
    <meta itemprop="description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
    <meta content="/assets/images/otx-logo-twitter.png" itemprop="image">

    <!-- Dont send referrer data, beyond main domain !-->
    <meta name="referrer" content="origin">

    <!-- Twitter Card data -->
    <meta name="twitter:card" content="summary">
    <meta name="twitter:site" content="@alienvault">
    <meta name="twitter:text:title" content="AlienVault - Open Threat Exchange">
    <meta name="twitter:text:description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
    <meta name="twitter:creator" content="@alienvault">
    <meta content="/assets/images/otx-logo-twitter.png" name="twitter:image">

    <!-- Open Graph data -->
    <meta property="og:title" content="AlienVault - Open Threat Exchange">
    <meta property="og:type" content="article">
    <meta property="og:url" content="">
    <meta content="/assets/images/otx-logo-twitter.png" property="og:image">
    <meta property="og:description" content="Learn about the latest cyber threats. Research, collaborate, and share threat intelligence in real time. Protect yourself and the community against today's emerging threats.">
    <meta property="og:site_name" content="AlienVault Open Threat Exchange">

from otx_2_misp.

That's weird. First of all, replace the otx_server with https://otx.alienvault.com instead of https://otx.alienvault.com/user/soc_columbus/pulses.

Second, when executing the script try this order:

otx2misp.py -d 30 -a -t -m

This will gather the otx pulses from the last 30 days and they will be correlated with your keywords and techniques files. Finally they will be sent to the MISP instance in case of a match.

Hope this helps.

from otx_2_misp.

Alex, I execute the command as you indicate but it keeps searching. I tried different techniques and day intervals but keep looking and no progress.

root@Servidor-MISP:/home/misp/otx_2_misp# python3 otx2misp.py -d 30 -a -t -m
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:61: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif (att is None) or (att is " ") or (att is ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:136: SyntaxWarning: "is not" with a literal. Did you mean "!="?
elif (att is not None) and (att is not " ") and (att is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:154: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['content'] is not None) and (ioc['content'] is not " ") and (ioc['content'] is not ''):
otx2misp.py:157: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (ioc['title'] is not '') and (ioc['title'] is not None):
otx2misp.py:159: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['description'] is not '':
otx2misp.py:161: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if ioc['role'] is not None and ioc['role'] is not '':
otx2misp.py:174: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if pulse['modified'] is not '':
[] Searching for Pulses on OTX:
[] Checking if the pulses gathered contain any keyword from your list.

from otx_2_misp.