Comments (9)
Yeah, proposals in the API weren't used much and are very broken, I started to work on it, but it isn't there...
I'm also working on a commit to allow pushing only one attribute instead of the whole event, but again, it isn't ready right now. And regarding the fact all the timestamps are updated to now even if the attribute isn't changed is weird. @iglocska is it expected?
from pymisp.
Thanks... it indeed is the case that the history (timestamps) are overwritten - teh large datatransfers are ugly, but I could live with it; the overwritten timestamps are a no-go for us. My current solution is to use the REST API directly to add a shadow attribute, and then to accept the proposal in a second step. This works, but of course it bypasses PyMisp. Also, I did not yet figure out how to add a (non-sample) attachment this way, just adding "data=" to the json object did not work (maybe it requires a uuid=... as well, it complains about wrong event). I will continue to try it out.
from pymisp.
Just saw this, as for the attributes being updated - yeah, an edit where the timestamp is omitted counts as a valid edit that fast forwards the timestamp. This is something we abuse for example when tagging attributes.
from pymisp.
Not sure if I understood this right - are you suggesting I should set explicitely a timestamp in my add_named_attribute call (can I do that at all?), or is it the PyMisp code itself that forgot to set the old timestamps in the newly constructed attributes? So is this something that will be fixed on your side?
from pymisp.
PS: About the other thing I mentioned... adding proposed attribtues fails... I suspect the line
response = session.post(url, data=json.dumps(query))
in __query_proposal should actually read
response = session.post(url, data=json.dumps(query, cls=EncodeUpdate))
if I understood the logic right. Otherwise, it crashes with a "TypeError: <pymisp.mispevent.MISPAttribute object at 0x10d9ff190> is not JSON serializable". But this is just speculation from my side (and not important for me right now, just wanted to mention it).
from pymisp.
I mean from the MISP core side it's expected behaviour, so I guess we need to make a small change to PyMISP.
Basically what happens: MISP will update any attributes explicitly included in an event edit that meet the timestamp criteria (timestamp > old timestamp or timestamp not set). AFAIK Raphael has reworked the way you add attributes to an event via PyMISP and it now doesn't fetch the event, make changes and push it back but instead allows you to simply add singular attributes (which should help with performance and remove any weird edit artifacts like the one you're encounterig).
from pymisp.
Adding proposals is being reworked and isn't working properly now.
Since yesterday evening, you can add attributes to an existing event by passing the event ID (and not the full event): https://github.com/MISP/PyMISP/blob/master/pymisp/api.py#L384
This will work with all the methods relying on add_named_attribute
and won't update the timestamp of all the attributes any more.
from pymisp.
Thanks.. if I get it right, I just need to pass the event ID as number instead of object? Somehow it still does not work, but maybe I'm doing something wrong?
>>> misp = pymisp.PyMISP('https://....', '....', True, 'json')
>>> evt = misp.get_event(529)
>>> evt
c7m6fx6f1tlzITywcLZ ...
>>> a=misp.add_named_attribute(529, "text", "test", category="External analysis", comment="comment")
>>> a
<Response [404]>
>>> a.text
u'{"name":"Could not save the attribute. ","message":"Could not save the attribute. ","url":"\\/attributes\\/add\\/529"}'
>>> b=misp.add_attachment(529, "/.../x.dat", category='External analysis', comment="comment")
>>> b
<Response [404]>
>>> b.text
u'{"name":"Could not save the attribute. ","message":"Could not save the attribute. ","url":"\\/attributes\\/add\\/529"}'
>>> a=misp.add_named_attribute(evt, "text", "test", category="External analysis", comment="comment")
>>> a
a
{u'Event': {u'...
The last one does add the attribute, but of course again with changed timestamps.
I'm a bit surprised about the "/attributes/add/[event_id]" URL - in the REST API documentation, I only see "/shadow_attributes/add/[event_id]" (https://circl.lu/doc/misp/automation/index.html).
I already previoiusly did wiretap a GUI session and captured this URI, but with POST parameters like
'_method': "POST",
'data[_Token][key]': '6e....',
'data[_Token][fields]': '7c....',
'data[_Token][unlocked]': '',
'data[Attribute][event_id]' : 529,
'data[Attribute][category]' : "External analysis" ,
'data[Attribute][type]' : "text",
'data[Attribute][distribution]' : 5 ,
'data[Attribute][value]' : "test" ,
'data[Attribute][comment]' : "comment" ,
'data[Attribute][to_ids]' : 0 ,
'data[Attribute][batch_import]' : 0
This does not look like REST API (it's not JSON, just standard POST parameters), but some internal, undocumented API only the GUI is using (that's my guess - is this correct, or is there any way to translate these requests to a correct REST API request?). I already previously tried using such a request directly, with the Authorization header and without the "_Token" parts, but nothing I tried worked. It seems your code tried to send a
{"category": "External analysis",
"comment": "comment",
"to_ids": false,
"value": "test",
"distribution": "5",
"type": "text",
"disable_correlation": false
}
This returns above error message. Of course it might be the MISP itself needs an update (I don't have direct access to it)? Currently I just upgraded my local PyMisp. The MISP itself is on version 2.4.65.
EDIT: We updated the MISP to 2.4.69, and now it works.. thanks ;-) Still interested how and if the GUI POST parameters can be translated to REST API.
from pymisp.
I just did a new commit in PyMISP where you can also do add a new attribute as proposal and it works the same way as adding an attribute (it isn't tagged as new version for now). It will probably make your life easier do to it.
If I got it right I can close this event then, please reopen/create a new one if you have any other remark.
from pymisp.
Related Issues (20)
- AttributeError: module 'pymisp' has no attribute 'EmailObject' when doing from pymisp import * HOT 2
- PyMISP.toggle_warninglist() sends GET method HOT 4
- PyMISP.disable_warninglist() doesn't disable, but toggle instead HOT 1
- Is it possible to change the timestamp for an attribute programatically? HOT 2
- IBM SOAR application with MISP error HOT 4
- upload_stix function return nothing when using in-memory file
- Support X-MISP-AUTH Header
- Proofpoint_tap.py script error HOT 1
- Tag for v2.4.189? HOT 4
- PyMISP and MISP version mismatches HOT 3
- PyMISP and MISP version mismatch HOT 2
- Object dropped due to validation for Event HOT 3
- Error with urllib3_with_brotli HOT 10
- Can't get search() to return attachments that have been included inside a object (e.g. report object attachment files) HOT 2
- Duplicated devices on GLPI HOT 1
- published=False returns published events HOT 2
- "The info field of the new event is required." returned when attempting to create a new event HOT 1
- Error while having the newest version of PyMISP: " This class is deprecated, use PyMISP instead warnings.warn('This class is deprecated, use PyMISP instead', FutureWarning" HOT 4
- Can't untag numeric tag name (str('1')) from an event HOT 2
- Sync Sightings: add sightings discards sighting UUID passed on HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pymisp.