eclecticiq / cabby Goto Github PK
View Code? Open in Web Editor NEWTAXII client implementation from EclecticIQ
License: Other
TAXII client implementation from EclecticIQ
License: Other
Specification defines 2 types of async polling: pulling of the results asynchronically[1] and asking TAXII server to push the results into provided Inbox Service. The first type (pulling) is not implemented.
[1] - "3.6.2.1 Pulling Asynchronous Poll Results" http://taxii.mitre.org/specifications/version1.1/TAXII_Services_Specification.pdf
Any plans for TAXII 2.0 integration? I much prefer cabby's interface and cli options to any of the other tools out there...
The build for the current master (0.1.23
) is failing -- https://travis-ci.org/github/eclecticiq/cabby/builds/774621265
Let me preface this with the fact that I am very new to OpenTAXII and Cabby.
I was attempting to set up a "subscriber" client that would subscribe to a collection. I located the inbox service that I wanted and made the subscription call as follows:
sub_status = client.subscribe(collection_name='my-collection', inbox_service=inbox, uri='<URI to collection management service>')
I then printed the information in sub_status
to verify the subscription details. I found that the delivery_params
property was empty (None
).
I ran the code in a debugger and went into the Cabby python code. In cabby/client11.py
the subscribe
function is properly populating the dictionary object that is passed to the _execute_request
function, with the details from the inbox I passed. However, the response object does not appear to have any of this information. It's as though it had been dropped.
Hello,
Trying to pull a freebie feed from https://limo.anomali.com/api/v1/taxii/taxii-discovery-service/. The published username/password is guest/guest. We're able to discover but when we try to poll any given collection it fails.
Help would be most appreciated.
Thanks,
Jeff
C:\scripts.py\cabby\discover>python -VV
Python 3.6.6 (default, Jan 29 2019, 17:21:24) [MSC v.1900 64 bit (AMD64)]
C:\scripts.py\cabby\discover>pip search cabby
cabby (0.1.20) - TAXII client library
INSTALLED: 0.1.20 (latest)
This is more of a question than a bug. My arguments to --key is returning the error unrecognized argument
taxii-poll: error: unrecognized arguments:: --key
Is there a particular private key format that is supported?
Hello,
Thank you for putting this together. Essentially, I am having trouble accessing the OpenTAXII from a different server; running commands from Cabby on the localhost everything works as expected, but when trying from a different server, I get the connection refused error.
I am running OpenTAXII on a CentOS 7 box configured on port 9003, and am using gunicorn as described in the documentation.
Running locally:
[root@server]# taxii-discovery --host localhost --port 9003 --path /services/discovery-a
2017-01-30 15:56:57,946 INFO: Sending Discovery_Request to http://localhost:9003/services/discovery-a
2017-01-30 15:56:57,960 INFO: 7 services discovered
=== Service Instance ===
Service Type: INBOX
Service Version: urn:taxii.mitre.org:services:1.1
Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
Service Address: http://localhost:9003/services/inbox-a
Message Binding: urn:taxii.mitre.org:message:xml:1.0
Message Binding: urn:taxii.mitre.org:message:xml:1.1
Inbox Service AC: []
Available: True
Message: Custom Inbox Service Description A
=== Service Instance ===
Service Type: INBOX
Service Version: urn:taxii.mitre.org:services:1.1
Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
Service Address: http://localhost:9003/services/inbox-b
Message Binding: urn:taxii.mitre.org:message:xml:1.0
Message Binding: urn:taxii.mitre.org:message:xml:1.1
Inbox Service AC: ['urn:stix.mitre.org:xml:1.1.1', 'urn:custom.example.com:json:0.0.1']
Available: True
Message: Custom Inbox Service Description B
=== Service Instance ===
Service Type: DISCOVERY
Service Version: urn:taxii.mitre.org:services:1.1
Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
Service Address: http://localhost:9003/services/discovery-a
Message Binding: urn:taxii.mitre.org:message:xml:1.0
Message Binding: urn:taxii.mitre.org:message:xml:1.1
Available: True
Message: Custom Discovery Service description
=== Service Instance ===
Service Type: DISCOVERY
Service Version: urn:taxii.mitre.org:services:1.1
Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0
Service Address: https://localhost:9003/services/discovery-a
Message Binding: urn:taxii.mitre.org:message:xml:1.0
Message Binding: urn:taxii.mitre.org:message:xml:1.1
Available: True
Message: Custom Discovery Service description
=== Service Instance ===
Service Type: COLLECTION_MANAGEMENT
Service Version: urn:taxii.mitre.org:services:1.1
Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
Service Address: http://localhost:9003/services/collection-management-a
Message Binding: urn:taxii.mitre.org:message:xml:1.0
Message Binding: urn:taxii.mitre.org:message:xml:1.1
Available: True
Message: Custom Collection Management Service description
=== Service Instance ===
Service Type: COLLECTION_MANAGEMENT
Service Version: urn:taxii.mitre.org:services:1.1
Protocol Binding: urn:taxii.mitre.org:protocol:https:1.0
Service Address: https://localhost:9003/services/collection-management-a
Message Binding: urn:taxii.mitre.org:message:xml:1.0
Message Binding: urn:taxii.mitre.org:message:xml:1.1
Available: True
Message: Custom Collection Management Service description
=== Service Instance ===
Service Type: POLL
Service Version: urn:taxii.mitre.org:services:1.1
Protocol Binding: urn:taxii.mitre.org:protocol:http:1.0
Service Address: http://localhost:9003/services/poll-a
Message Binding: urn:taxii.mitre.org:message:xml:1.0
Message Binding: urn:taxii.mitre.org:message:xml:1.1
Available: True
Message: Custom Poll Service description
Running from a different box:
[root@differentserver ~]# taxii-discovery --host [IP of OpenTAXII server] --port 9003 --path /services/discovery-a
2017-01-30 17:05:59,935 INFO: Sending Discovery_Request to http://[IP of OpenTAXII server] :9003/services/discovery-a
2017-01-30 17:05:59,939 ERROR: HTTPConnectionPool(host='[IP of OpenTAXII server]', port=9003): Max retries exceeded with url: /services/discovery-a (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7ff432873d90>: Failed to establish a new connection: [Errno 111] Connection refused',))
Trying to CURL the health check of the OpenTAXII from the 2nd box:
[root@differentserver ~]# curl http://[IP of OpenTAXII server]:9003/management/health
curl: (7) couldn't connect to host
CURL to the HTTP port of that server is fine.
[root@differentserver ~]# curl http://[IP of OpenTAXII server]:80/file.html
this is a test
Any help to point me in the right direction will be much appreciated.
Thank you!
content_blocks = client.poll(collection_name='collection-A', begin_date='2017-06-29T19:37:05+08:00', end_date='2017-06-29T20:37:05+08:00')
Traceback (most recent call last):
File "poll.py", line 17, in
for block in content_blocks:
File "/usr/lib/python2.7/site-packages/cabby/client11.py", line 485, in poll
service_type=const.SVC_POLL)
File "/usr/lib/python2.7/site-packages/cabby/abstract.py", line 205, in _execute_request
timeout=self.timeout)
File "/usr/lib/python2.7/site-packages/cabby/dispatcher.py", line 91, in send_taxii_request
raise UnsuccessfulStatusError(obj)
cabby.exceptions.UnsuccessfulStatusError: FAILURE: There was a failure while executing the message handler
Should the bin/taxii-collections
script be part of the distribution? It's currently not included in setup.py
.
Add a workaround while TAXIIProject/libtaxii#186 is fixed
The file produced by taxii-poll with -l greater than 1 is not valid xml and thus can't be used directly in taxii-push. This is not clear in the documentation and has lead to confusion in eclecticiq/OpenTAXII#169
Is it possible to push more than one file content to the collection in a single command, like [ taxii-push --path http://ip.address:port/services/inbox --content-file * --dest collection ]? I tried this command and I got the following response: [taxii-push: error: unrecognized arguments] Thanks
Hello,
I'm receiving gzip compressed data from a TAXII server. When cabby tries to process the data the following exception is raised:
Traceback (most recent call last):
File "test.py", line 14, in
discovery = ais_client.discover_services()
File "/home/harley/Workspace/github/cabby/cabby/abstract.py", line 304, in discover_services
response = self._discovery_request(uri)
File "/home/harley/Workspace/github/cabby/cabby/client11.py", line 29, in _discovery_request
response = self._execute_request(request, uri=uri)
File "/home/harley/Workspace/github/cabby/cabby/abstract.py", line 202, in _execute_request
taxii_binding=self.taxii_binding)
File "/home/harley/Workspace/github/cabby/cabby/dispatcher.py", line 81, in send_taxii_request
obj = next(gen)
File "/home/harley/Workspace/github/cabby/cabby/dispatcher.py", line 170, in _parse_response
action, root = next(gen)
File "src/lxml/iterparse.pxi", line 208, in lxml.etree.iterparse.next (src/lxml/lxml.etree.c:144543)
io.UnsupportedOperation: seek
I'm not sure exactly what the issue is but I was able to get around this by using StringIO with response.content in dispatcher.py instead of attempting to manually decompress the raw content.
This is just a reminder that the current package depends on a patched libtaxii version, which is not available from PyPI.
Is there a way to show whether the client was unsuccessful in connecting to the poll server or the push server? Here's my current code:
content_blocks = pollclient.poll(collection_name=pollcollection, uri=pollurl)
binding = 'urn:stix.mitre.org:xml:1.1.1'
gen = (block for block in content_blocks)
top100 = itertools.islice(gen, 100)
for block in top100:
pushclient.push(str(block.content.decode('ascii')), binding, uri=pushurl, collection_names=[pushcollection])
I want to except on errors when the connection is unsuccessful, so that if I have a list of servers I want to poll, I don't have to not get the entire list just because one in the middle failed. However, I'd like to know whether it was the poll server or the push server that tripped the unsuccessful error. If I do this:
content_blocks = pollclient.poll(collection_name=pollcollection, uri=pollurl)
binding = 'urn:stix.mitre.org:xml:1.1.1'
gen = (block for block in content_blocks)
top100 = itertools.islice(gen, 100)
try:
for block in top100:
pushclient.push(str(block.content.decode('ascii')), binding, uri=pushurl, collection_names=[pushcollection])
except cabby.exceptions.UnsuccessfulStatusError:
pass
It only reflects as an Unsuccessful "unauthorized" error. Let me know if this is doable. I'm working on some code in the meantime to try and mitigate this.
The setup.py
should have author/maintainer/description fields, and some trove classififers.
This will make PyPI accept the package, and also make it discoverable using e.g. its search functionality.
Sorry, wasn't sure how else to reach you. README provides the contact email [email protected]
. When sending to this address, the mail bounces with the message (apparently that address is aliased to Engineering@):
Your message to [email protected] couldn't be delivered.
The group Engineering only accepts messages from people in its organization or on its allowed senders list, and your email address isn't on the list.
Hi guys
Let's say I'm using server certificate signed by certificate authority which is not widely known.
Is there any way to specify CA certificate when I'm using CLI to connect to the TAXII server?
taxii-discovery --host taxii.example.net --port 443 --https --path services/discovery --username user --password ********
2017-09-08 11:26:39,100 ERROR: HTTPSConnectionPool(host='taxii.example.net', port=443): Max retries exceeded with url: /services/discovery (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)'),))
Instead of having host
, port
, and use_https
flags, perhaps the create_client()
factory function could take an url instead.
Hello, I am trying to subscribe to a collection within Anomali Limo (https://www.anomali.com/resources/limo). However, when using the URL (https://limo.anomali.com/taxii) I seem to be getting errors in the request and am not sure how to go about getting the STIX 2.0 data. I don't believe I am making a proper request using the cabby client correctly. How would I go about doing so?
The benefits of using console script entry points is that the actual scripts can live inside the package, and can use the module name for e.g. logger names. It will also make it easier to install on non posix platforms. Additionally it makes it easier to use the tools from within a virtualenv without first activating it. (Ideally something like pipsi install taxii-client
should make the cli programs available.)
I'm running an OpenTAXII server on port 9000, and try to use cabby to access it. Running with --port
the value is ignored:
$ taxii-push --host 127.0.0.1 --port 9000 --path /service/inbox-a --dest mycollection -f content.txt
2016-07-21 03:39:43,430 INFO: Sending Inbox_Message to http://127.0.0.1/service/inbox-a
2016-07-21 03:39:43,434 INFO: Starting new HTTP connection (1): 127.0.0.1
2016-07-21 03:39:43,435 ERROR: HTTPConnectionPool(host='127.0.0.1', port=80): Max retries exceeded with url: /service/inbox-a (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f983e592198>: Failed to establish a new connection: [Errno 111] Connection refused',))
You can see in the first log line that an attempt is made to port 80 and not port 9000.
This works OK if I add a full URL in the path as follows:
taxii-push --path http://127.0.0.1:9000/services/inbox-a --dest mycollection -f content.txt
Is that the intended usage?
Is the --port
option actually used?
Now objects in console looks like this:
<cabby.entities.ContentBlock at 0x7f26aaa90dd8>
<cabby.entities.Collection at 0x7f26aaa86470>
<cabby.client11.Client11 at 0x7f26b9287898>
It would be helpful to see in repr
for object some human-readable information. For example:
Collection(name='any-data')
Hey there,
I'd like to query the DB for specific objects for the following reasons:
Any help or recommendations, please?
I think before we publicize this code as open source, we'll have to run it against some PEP8 verifier and adjust the code where applicable.
Hey, I'm trying to poll from a taxii server but getting
2020-02-11 08:26:04,892 - ERROR - cabby.entities:192 - Unknown service protocol: urn:taxii.mitre.org:protocol:http:1.1
After peaking in this line of code I've found out that you only support
'urn:taxii.mitre.org:protocol:http:1.0'
thank you!
I'd like feedback on making the default binding for taxii-push urn:stix.mitre.org:xml:1.2. My understanding is that 1.1.1 docs should pass 1.2 schema validation, so this may remove the need to specify any --binding arg when dealing with either 1.1.1 or 1.2 docs.
just a general remark, not sure how to cleanly abstract those difference away, though...
I am trying to use taxii-proxy
to fetch some data from hailataxi.com and push into my own local collection.
Based on the size of the database file, it appears this is happening. But when I try to use taxii-poll
on my local collection it does not return any data. Why is this?
The size of the data file indicates how much data roughly is in my collection.
(opentaxii-venv)[root@source opentaxii]# ls -l data
total 1804
-rw-r--r--. 1 root root 1838080 Apr 20 20:50 data.db
-rw-r--r--. 1 root root 3072 Apr 20 20:48 auth.db
Now proxy the data from hailataxii.com
(opentaxii-venv)[root@source opentaxii]# taxii-proxy --poll-path http://hailataxii.com/taxii-data
--poll-collection guest.phishtank_com
--inbox-path http://localhost:9000/services/inbox
--inbox-collection hailataxii-phishtank
--binding urn:stix.mitre.org:xml:1.1
--inbox-username guest
--inbox-password guest
--begin 2016-04-20
2016-04-20 20:51:53,139 INFO: Sending Poll_Request to http://hailataxii.com/taxii-data
2016-04-20 20:51:53,146 INFO: Starting new HTTP connection (1): hailataxii.com
2016-04-20 20:51:53,624 INFO: Response received for Poll_Request from http://hailataxii.com/taxii-data
2016-04-20 20:51:53,800 INFO: Sending Inbox_Message to http://localhost:9000/services/inbox
...
Check the size of the data file again. Its bigger! We must have successfully received data.
(opentaxii-venv)[root@source opentaxii]# ls -l data
total 2736
-rw-r--r--. 1 root root 2791424 Apr 20 20:51 data.db
-rw-r--r--. 1 root root 3072 Apr 20 20:48 auth.db
But there is no data. Why is that?
(opentaxii-venv)[root@source opentaxii]# taxii-poll --discovery http://localhost:9000/services/discovery -c hailataxii-phishtank
2016-04-20 20:55:01,873 INFO: Polling using data binding: ALL
2016-04-20 20:55:01,874 INFO: Sending Discovery_Request to http://localhost:9000/services/discovery
2016-04-20 20:55:01,880 INFO: Starting new HTTP connection (1): localhost
2016-04-20 20:55:01,886 INFO: Response received for Discovery_Request from http://localhost:9000/services/discovery
2016-04-20 20:55:01,888 INFO: 3 services discovered
2016-04-20 20:55:01,888 INFO: Sending Poll_Request to http://localhost:9000/services/poll
2016-04-20 20:55:01,890 INFO: Starting new HTTP connection (1): localhost
2016-04-20 20:55:01,898 INFO: Response received for Poll_Request from http://localhost:9000/services/poll
2016-04-20 20:55:01,899 INFO: 0 blocks polled
services.yml
---
#
# discovery: used by a TAXII Client to discover available TAXII Service
#
- id: discovery
type: discovery
address: /services/discovery
description: Discovery service
advertised_services:
- inbox
- collection
- poll
protocol_bindings:
- urn:taxii.mitre.org:protocol:http:1.0
#
# inbox: used by a TAXII Client to push information to a TAXII Server
#
- id: inbox
type: inbox
address: /services/inbox
description: Inbox for hailataxii.com feed
destination_collection_required: yes
accept_all_content: no
authentication_required: yes
protocol_bindings:
- urn:taxii.mitre.org:protocol:http:1.0
#
# collection_management: used by a TAXII Client to request information about
# available data collections or request a subscription.
#
- id: collection
type: collection_management
address: /services/collection
description: Collection management service for hailataxii.com feed
protocol_bindings:
- urn:taxii.mitre.org:protocol:http:1.0
#
# poll: used by a TAXII Client to request information from a TAXII Server
#
- id: poll
type: poll
address: /services/poll
description: Poll for hailataxii.com feed
subscription_required: no
authentication_required: no
protocol_bindings:
- urn:taxii.mitre.org:protocol:http:1.0
collections.yml
---
- name: hailataxii-phishtank
available: true
accept_all_content: true
supported_content:
- urn:stix.mitre.org:xml:1.0
- urn:stix.mitre.org:xml:1.1
service_ids:
#- discovery
- inbox
- collection
- poll
when I try to use cabby to upload a xml file to the taxii server, I always got the response :ERROR: init() got an unexpected keyword argument 'recover'
what can I do to solve this problem? Thanks
client.poll()
method will not return anything if count_only
set to True
.
So I was trying to locate anything that on the pages that showed cabby supports certificates that require passwords. If not, this is something that many organizations are looking to have implemented.
At the moment, Cabby will throw
cabby.exceptions.UnsuccessfulStatusError: FAILURE: There was a failure while executing the message handler
when it receives an error from the server. This is not very clear and should be worded in a better way -- to explicitly say that the error is not in the client but came from the server
Cabby can not push PDF files because it expects non-binary file as input.
I have attached output of Taxii client below cabby client log.
With fulfillment_client cabby is recursively polling with next result-part-number. and it never stops...
result-part-number = 2 is not asking check result-part-number = 3 (can check the taxii client log)
this is end of first block.
2021-01-20 12:02:18,025 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" collection_name="PhishTank" more="true" result_id="1582588800404:1582675200404" result_part_number="1" in_response_to="cb280ba7-3d4b-46e1-8568-de3f394d3086" message_id="urn:uuid:2c6a068a-983e-4de7-89e1-739aedaf9e93"><taxii_11:Exclusive_Begin_Timestamp>2020-02-25T00:00:00.404174Z</taxii_11:Exclusive_Begin_Timestamp><taxii_11:Inclusive_End_Timestamp>2020-02-26T00:00:00.404174Z</taxii_11:Inclusive_End_Timestamp><taxii_11:Record_Count partial_count="false">500</taxii_11:Record_Count></taxii_11:Poll_Response>
Issue while downloading from 2 block, which is not a proper stix package, so cabbly client is downloading it again and again and waits for end of package
2021-01-20 12:02:18,027 cabby.dispatcher INFO: Sending Poll_Fulfillment to https://partnerstage.threatconnect.com/api/taxii/poll/
2021-01-20 12:02:18,027 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Fulfillment xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="5fdf99ad-471c-4ac6-8fd0-52d4c40498b9" collection_name="PhishTank" result_id="1582588800404:1582675200404" **result_part_number="2**"/>
2021-01-20 12:02:18,028 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): partnerstage.threatconnect.com
2021-01-20 12:02:22,034 urllib3.connectionpool DEBUG: https://partnerstage.threatconnect.com:443 "POST /api/taxii/poll/ HTTP/1.1" 200 662
2021-01-20 12:02:22,040 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Content_Block xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#"><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/><taxii_11:Content><stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="threatconnect:STIXPackage-c10f04c3-cf4e-46d7-be66-1e3f7a78d64f" timestamp="2021-01-20T12:02:59.008Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package></taxii_11:Content></taxii_11:Content_Block>
<stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" id="threatconnect:STIXPackage-c10f04c3-cf4e-46d7-be66-1e3f7a78d64f" timestamp="2021-01-20T12:02:59.008Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package>
2021-01-20 12:02:22,044 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" collection_name="PhishTank" more="false" result_id="1582588800404:1582675200404" result_part_number="2" in_response_to="5fdf99ad-471c-4ac6-8fd0-52d4c40498b9" message_id="urn:uuid:1a657342-562a-404b-a201-d3b88bc6e17a"><taxii_11:Exclusive_Begin_Timestamp>2020-02-25T00:00:00.404Z</taxii_11:Exclusive_Begin_Timestamp><taxii_11:Inclusive_End_Timestamp>2020-02-26T00:00:00.404Z</taxii_11:Inclusive_End_Timestamp><taxii_11:Record_Count partial_count="false">508</taxii_11:Record_Count></taxii_11:Poll_Response>
2021-01-20 12:02:22,045 cabby.dispatcher INFO: Sending Poll_Fulfillment to https://partnerstage.threatconnect.com/api/taxii/poll/
2021-01-20 12:02:22,046 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Fulfillment xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="28d56e45-b948-47d3-9b69-90887f29e3a0" collection_name="PhishTank" result_id="1582588800404:1582675200404" **result_part_number="3**"/>
2021-01-20 12:02:22,047 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): partnerstage.threatconnect.com
2021-01-20 12:02:26,097 urllib3.connectionpool DEBUG: https://partnerstage.threatconnect.com:443 "POST /api/taxii/poll/ HTTP/1.1" 200 662
2021-01-20 12:02:26,100 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Content_Block xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#"><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/><taxii_11:Content><stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="threatconnect:STIXPackage-11cd8863-f3e5-415e-959e-db9f29263745" timestamp="2021-01-20T12:03:02.914Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package></taxii_11:Content></taxii_11:Content_Block>
<stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" id="threatconnect:STIXPackage-11cd8863-f3e5-415e-959e-db9f29263745" timestamp="2021-01-20T12:03:02.914Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package>
2021-01-20 12:02:26,102 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" collection_name="PhishTank" more="false" result_id="1582588800404:1582675200404" result_part_number="3" in_response_to="28d56e45-b948-47d3-9b69-90887f29e3a0" message_id="urn:uuid:5e0116f5-ddaa-4d98-aa63-41afa0e8ce0e"><taxii_11:Exclusive_Begin_Timestamp>2020-02-25T00:00:00.404Z</taxii_11:Exclusive_Begin_Timestamp><taxii_11:Inclusive_End_Timestamp>2020-02-26T00:00:00.404Z</taxii_11:Inclusive_End_Timestamp><taxii_11:Record_Count partial_count="false">508</taxii_11:Record_Count></taxii_11:Poll_Response>
2021-01-20 12:02:26,103 cabby.dispatcher INFO: Sending Poll_Fulfillment to https://partnerstage.threatconnect.com/api/taxii/poll/
2021-01-20 12:02:26,103 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Fulfillment xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="699e0f1d-07b1-43de-a386-dfc91734f0ad" collection_name="PhishTank" result_id="1582588800404:1582675200404" **result_part_number="4"**/>
2021-01-20 12:02:26,105 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): partnerstage.threatconnect.com
2021-01-20 12:02:30,395 urllib3.connectionpool DEBUG: https://partnerstage.threatconnect.com:443 "POST /api/taxii/poll/ HTTP/1.1" 200 661
2021-01-20 12:02:30,399 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Content_Block xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#"><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/><taxii_11:Content><stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="threatconnect:STIXPackage-3b545c81-bd47-44a9-acfa-98fa6d969074" timestamp="2021-01-20T12:03:06.873Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package></taxii_11:Content></taxii_11:Content_Block>
<stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" id="threatconnect:STIXPackage-3b545c81-bd47-44a9-acfa-98fa6d969074" timestamp="2021-01-20T12:03:06.873Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package>
2021-01-20 12:02:30,401 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" collection_name="PhishTank" more="false" result_id="1582588800404:1582675200404" result_part_number="4" in_response_to="699e0f1d-07b1-43de-a386-dfc91734f0ad" message_id="urn:uuid:10850dd7-d201-43c0-829d-5140bd54ce73"><taxii_11:Exclusive_Begin_Timestamp>2020-02-25T00:00:00.404Z</taxii_11:Exclusive_Begin_Timestamp><taxii_11:Inclusive_End_Timestamp>2020-02-26T00:00:00.404Z</taxii_11:Inclusive_End_Timestamp><taxii_11:Record_Count partial_count="false">508</taxii_11:Record_Count></taxii_11:Poll_Response>
2021-01-20 12:02:30,403 cabby.dispatcher INFO: Sending Poll_Fulfillment to https://partnerstage.threatconnect.com/api/taxii/poll/
2021-01-20 12:02:30,403 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Fulfillment xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="e3938bd9-1f2c-4252-b6cd-3cb6232fc0ad" collection_name="PhishTank" result_id="1582588800404:1582675200404" **result_part_number="5**"/>
2021-01-20 12:02:30,405 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): partnerstage.threatconnect.com
2021-01-20 12:02:35,678 urllib3.connectionpool DEBUG: https://partnerstage.threatconnect.com:443 "POST /api/taxii/poll/ HTTP/1.1" 200 661
2021-01-20 12:02:35,682 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Content_Block xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#"><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/><taxii_11:Content><stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="threatconnect:STIXPackage-9a530212-be91-4aa4-8560-d7525a2cf00f" timestamp="2021-01-20T12:03:12.631Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package></taxii_11:Content></taxii_11:Content_Block>
<stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" id="threatconnect:STIXPackage-9a530212-be91-4aa4-8560-d7525a2cf00f" timestamp="2021-01-20T12:03:12.631Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package>
2021-01-20 12:02:35,683 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" collection_name="PhishTank" more="false" result_id="1582588800404:1582675200404" result_part_number="5" in_response_to="e3938bd9-1f2c-4252-b6cd-3cb6232fc0ad" message_id="urn:uuid:48b06dec-fc0a-4c8b-8ad2-61570e3a9f97"><taxii_11:Exclusive_Begin_Timestamp>2020-02-25T00:00:00.404Z</taxii_11:Exclusive_Begin_Timestamp><taxii_11:Inclusive_End_Timestamp>2020-02-26T00:00:00.404Z</taxii_11:Inclusive_End_Timestamp><taxii_11:Record_Count partial_count="false">508</taxii_11:Record_Count></taxii_11:Poll_Response>
2021-01-20 12:02:35,685 cabby.dispatcher INFO: Sending Poll_Fulfillment to https://partnerstage.threatconnect.com/api/taxii/poll/
2021-01-20 12:02:35,685 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Fulfillment xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="92353625-dcef-4629-99c6-f17a8b7a26ec" collection_name="PhishTank" result_id="1582588800404:1582675200404" **r****esult_part_number="6**"/>
2021-01-20 12:02:35,687 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): partnerstage.threatconnect.com
2021-01-20 12:02:41,216 urllib3.connectionpool DEBUG: https://partnerstage.threatconnect.com:443 "POST /api/taxii/poll/ HTTP/1.1" 200 662
2021-01-20 12:02:41,222 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Content_Block xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#"><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/><taxii_11:Content><stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="threatconnect:STIXPackage-02f61192-d97a-4b7c-9326-22c57b65cde6" timestamp="2021-01-20T12:03:18.292Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package></taxii_11:Content></taxii_11:Content_Block>
<stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" id="threatconnect:STIXPackage-02f61192-d97a-4b7c-9326-22c57b65cde6" timestamp="2021-01-20T12:03:18.292Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package>
2021-01-20 12:02:41,225 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" collection_name="PhishTank" more="false" result_id="1582588800404:1582675200404" result_part_number="6" in_response_to="92353625-dcef-4629-99c6-f17a8b7a26ec" message_id="urn:uuid:19a4aaf6-72e6-4c73-a89b-4fb269113594"><taxii_11:Exclusive_Begin_Timestamp>2020-02-25T00:00:00.404Z</taxii_11:Exclusive_Begin_Timestamp><taxii_11:Inclusive_End_Timestamp>2020-02-26T00:00:00.404Z</taxii_11:Inclusive_End_Timestamp><taxii_11:Record_Count partial_count="false">508</taxii_11:Record_Count></taxii_11:Poll_Response>
2021-01-20 12:02:41,227 cabby.dispatcher INFO: Sending Poll_Fulfillment to https://partnerstage.threatconnect.com/api/taxii/poll/
2021-01-20 12:02:41,227 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Fulfillment xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="2cb6ecfa-7647-4510-b513-17abc4753938" collection_name="PhishTank" result_id="1582588800404:1582675200404" r**esult_part_number="7"**/>
2021-01-20 12:02:41,229 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): partnerstage.threatconnect.com
2021-01-20 12:02:45,741 urllib3.connectionpool DEBUG: https://partnerstage.threatconnect.com:443 "POST /api/taxii/poll/ HTTP/1.1" 200 662
2021-01-20 12:02:45,748 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Content_Block xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#"><taxii_11:Content_Binding binding_id="urn:stix.mitre.org:xml:1.1.1"/><taxii_11:Content><stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="threatconnect:STIXPackage-adbee82f-f87e-4276-9ad4-93c1677353e3" timestamp="2021-01-20T12:03:20.561Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package></taxii_11:Content></taxii_11:Content_Block>
<stix:STIX_Package xmlns="http://xml/metadataSharing.xsd" xmlns:stix="http://stix.mitre.org/stix-1" xmlns:threatconnect="http://threatconnect.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" id="threatconnect:STIXPackage-adbee82f-f87e-4276-9ad4-93c1677353e3" timestamp="2021-01-20T12:03:20.561Z" version="1.1.1" xsi:schemaLocation="http://stix.mitre.org/stix-1 http://stix.mitre.org/XMLSchema/core/1.1.1/stix_core.xsd"><stix:STIX_Header><stix:Title>Report: PhishTank</stix:Title><stix:Package_Intent>INDICATORS</stix:Package_Intent></stix:STIX_Header><stix:Indicators/></stix:STIX_Package>
2021-01-20 12:02:45,749 cabby.dispatcher DEBUG: Stream element:
<taxii_11:Poll_Response xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:xmldsig="http://www.w3.org/2000/09/xmldsig#" collection_name="PhishTank" more="false" result_id="1582588800404:1582675200404" result_part_number="7" in_response_to="2cb6ecfa-7647-4510-b513-17abc4753938" message_id="urn:uuid:81ec21ee-b9c0-4c34-b27d-43205c8ead62"><taxii_11:Exclusive_Begin_Timestamp>2020-02-25T00:00:00.404Z</taxii_11:Exclusive_Begin_Timestamp><taxii_11:Inclusive_End_Timestamp>2020-02-26T00:00:00.404Z</taxii_11:Inclusive_End_Timestamp><taxii_11:Record_Count partial_count="false">508</taxii_11:Record_Count></taxii_11:Poll_Response>
2021-01-20 12:02:45,751 cabby.dispatcher INFO: Sending Poll_Fulfillment to https://partnerstage.threatconnect.com/api/taxii/poll/
2021-01-20 12:02:45,751 cabby.dispatcher DEBUG: Request:
<taxii_11:Poll_Fulfillment xmlns:taxii="http://taxii.mitre.org/messages/taxii_xml_binding-1" xmlns:taxii_11="http://taxii.mitre.org/messages/taxii_xml_binding-1.1" xmlns:tdq="http://taxii.mitre.org/query/taxii_default_query-1" message_id="d169300e-5261-4cae-9544-8a8209e06486" collection_name="PhishTank" result_id="1582588800404:1582675200404" **result_part_number="8"**/>
2021-01-20 12:02:45,753 urllib3.connectionpool DEBUG: Starting new HTTPS connection (1): partnerstage.threatconnect.com
^CTraceback (most recent call last):
File "/usr/bin/taxii-poll", line 11, in <module>
load_entry_point('cabby==0.1.20', 'console_scripts', 'taxii-poll')()
File "build/bdist.linux-x86_64/egg/cabby/cli/poll.py", line 160, in poll_content
File "build/bdist.linux-x86_64/egg/cabby/cli/commons.py", line 171, in run_client
File "build/bdist.linux-x86_64/egg/cabby/cli/poll.py", line 139, in _runner
File "build/bdist.linux-x86_64/egg/cabby/client11.py", line 505, in poll
File "build/bdist.linux-x86_64/egg/cabby/client11.py", line 545, in fulfilment
File "build/bdist.linux-x86_64/egg/cabby/abstract.py", line 190, in _execute_request
File "build/bdist.linux-x86_64/egg/cabby/dispatcher.py", line 54, in send_taxii_request
File "build/bdist.linux-x86_64/egg/cabby/dispatcher.py", line 89, in request_stream
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 555, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
httplib_response = conn.getresponse(buffering=True)
File "/usr/lib/python2.7/httplib.py", line 1121, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 438, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 394, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/lib/python2.7/ssl.py", line 766, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 653, in read
v = self._sslobj.read(len)
KeyboardInterrupt
Manually killed the process as its goes to never ending loop.
Issue is seen every time cabby tries to download multiple blocks.
=============================================================================
### But with the taxii client it
[root@vm21centos-1 ~]# poll_client -u https://partnerstage.threatconnect.com/api/taxii/poll --username ******** --pass *********8 --collection PhishTank --begin-timestamp 2020-02-25T00:00:00.404174+00:00 --end-timestamp 2020-02-26T00:00:00.404174+00:00
Request:
Message Type: Poll_Request
Message ID: 543160029855896038
Collection Name: PhishTank
Excl. Begin TS Label: 2020-02-25 00:00:00.404174+00:00
Incl. End TS Label: 2020-02-26 00:00:00.404174+00:00
=== Poll_Parameters ===
Response type: FULL
Response:
Message Type: Poll_Response
Message ID: urn:uuid:8766717d-5f75-4843-abaa-bbe6a499b36b; In Response To: 543160029855896038
Collection Name: PhishTank
More: True
Result ID: 1582588800404:1582675200404
Result Part Num: 1
=== Record Count ===
Record Count: 500
Excl. Begin TS Label: 2020-02-25T00:00:00.404174+00:00
Incl. End TS Label: 2020-02-26T00:00:00.404174+00:00
=== Content Block ===
Content Binding: urn:stix.mitre.org:xml:1.1.1
Content length: 805885
(Content not printed for brevity)
Message: None
Padding: None
This response has More=True, to request additional parts, use the following command:
fulfillment_client --collection PhishTank --result-id 1582588800404:1582675200404 --result-part-number 2
File created: PhishTank_STIX111_s2021_01_22T12_35_25_073632.xml
[root@vm21centos-1 ~]# fulfillment_client -u https://partnerstage.threatconnect.com/api/taxii/poll --username ********* --pass ********* --collection PhishTank --result-id 1582588800404:1582675200404 --result-part-number 2
Request:
Message Type: Poll_Fulfillment
Message ID: 8622465795543714864
Collection Name: PhishTank
Result ID: 1582588800404:1582675200404
Result Part Number: 2
Response:
Message Type: Poll_Response
Message ID: urn:uuid:3cfe6c9b-15e3-43b3-851d-63b12b3a68bd; In Response To: 8622465795543714864
Collection Name: PhishTank
More: False
Result ID: 1582588800404:1582675200404
Result Part Num: 2
=== Record Count ===
Record Count: 508
Excl. Begin TS Label: 2020-02-25T00:00:00.404000+00:00
Incl. End TS Label: 2020-02-26T00:00:00.404000+00:00
=== Content Block ===
Content Binding: urn:stix.mitre.org:xml:1.1.1
Content length: 722
(Content not printed for brevity)
Message: None
Padding: None
its not asking to download result-part-number 3
[root@vm21centos-1 ~]# fulfillment_client -u https://partnerstage.threatconnect.com/api/taxii/poll --username ********* --pass ********* --collection PhishTank --result-id 1582588800404:1582675200404 --result-part-number 3
Request:
Message Type: Poll_Fulfillment
Message ID: 842267104647120760
Collection Name: PhishTank
Result ID: 1582588800404:1582675200404
Result Part Number: 3
Response:
Message Type: Poll_Response
Message ID: urn:uuid:41eaec3e-b429-4aca-95f9-dd0d5d8aa5b9; In Response To: 842267104647120760
Collection Name: PhishTank
More: False
Result ID: 1582588800404:1582675200404
Result Part Num: 3
=== Record Count ===
Record Count: 508
Excl. Begin TS Label: 2020-02-25T00:00:00.404000+00:00
Incl. End TS Label: 2020-02-26T00:00:00.404000+00:00
=== Content Block ===
Content Binding: urn:stix.mitre.org:xml:1.1.1
Content length: 722
(Content not printed for brevity)
Message: None
Padding: None
But still don't see any error if try to download result-part-number 3...
I have a really simple python script using cabby.
import cabby
def main():
client = cabby.create_client(host='opentaxii.home',port=9000,discovery_path='/services/discovery')
client.set_auth(username='user',password='password')
try:
services=client.discover_services()
except Exception as e:
print(e)
else:
for service in services:
print('service type={s.type}, address={s.address}'.format(s=service))
if __name__ == "__main__":
main()
If i run this on my Mac, it runs perfect. If I run it in a docker container that is based on Centos 7 (with cabby installed with pip). I get init() got an unexpected keyword argument 'recover'. Obviously something is missing in my python environment inside the docker container, but I have no idea. I narrowed this down to line 85 in cabby/dispatcher.py
PR #47 was blocked because of gabrielfalcao/HTTPretty#242. As @wbolster said in that issue, using Respones instead of HTTPretty could solve the problem.
Step 5/10 : COPY ./requirements.txt ./requirements-dev.txt /cabby
When using COPY with more than one source file, the destination must be a directory and end with a /
Fixed with a modification to the Dockerfile:
Orig line 6: COPY ./requirements.txt ./requirements-dev.txt /cabby
New Line 6: COPY ./requirements.txt ./requirements-dev.txt /cabby/
here the original error gets casted to string, which is not an intended behaviour
test
There's been a few commits to this repo since the November 2020 1.23 release - should there be another release in the works?
Add the ability for using certificate-based authentication for the taxii-proxy
command.
I recommend we use a tool such as pyflakes
or pylint
to check for certain types of inconsistencies in taxii-client
. For instance:
jhg@earthenware:~/intelworks/taxii-client-master$ find -name '*.py' -exec pyflakes {} \;
./tests/test_client11.py:2: 'urllib2' imported but unused
./tests/test_client11.py:12: 'from libtaxii.constants import *' used; unable to detect undefined names
./tests/test_client11.py:13: 'from fixtures11 import *' used; unable to detect undefined names
./tests/test_client11.py:330: local variable 'response' is assigned to but never used
./tests/test_client11.py:348: local variable 'response' is assigned to but never used
./tests/test_client10.py:2: 'urllib2' imported but unused
./tests/test_client10.py:12: 'from libtaxii.constants import *' used; unable to detect undefined names
./tests/test_client10.py:13: 'from fixtures10 import *' used; unable to detect undefined names
./tests/test_client10.py:229: local variable 'response' is assigned to but never used
./taxii_client/client11.py:1: 'datetime' imported but unused
./taxii_client/client11.py:4: 'from libtaxii.constants import *' used; unable to detect undefined names
./taxii_client/client11.py:7: 'ContentBlock' imported but unused
./taxii_client/client11.py:7: 'ts_to_date' imported but unused
./taxii_client/client11.py:111: local variable 'response' is assigned to but never used
./taxii_client/client10.py:3: 'from libtaxii.constants import *' used; unable to detect undefined names
./taxii_client/client10.py:53: local variable 'response' is assigned to but never used
./taxii_client/abstract.py:6: 'ST_NOT_FOUND' imported but unused
./taxii_client/__init__.py:5: 'ContentBlock' imported but unused
Especially noticeable are the unused imports, which should be removed in a public version (imho).
Currently if you specify a JWT url, and username/password, on each request the Token is fetched again. this has a big performance impact. It would be great if there would be a (persistent) caching option for the token, and automatic expiry detection, so that a new token is fetched (UNAUTHENTICATED)
https://github.com/Intelworks/cabby/blob/master/cabby/dispatcher.py#L69
Is there any way I can specify timeout for the TAXII request ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.