c3c / adexplorersnapshot.py Goto Github PK
View Code? Open in Web Editor NEWADExplorerSnapshot.py is an AD Explorer snapshot parser. It is made as an ingestor for BloodHound, and also supports full-object dumping to NDJSON.
ADExplorerSnapshot.py is an AD Explorer snapshot parser. It is made as an ingestor for BloodHound, and also supports full-object dumping to NDJSON.
python3 ADExplorerSnapshot.py result.dat -m Objects
[◓] Collecting data: dumped 6522/212428 objects Traceback (most recent call last): File "ADExplorerSnapshot.py", line 2, in <module> adexpsnapshot.main() File "/root/ADExplorerSnapshot.py/adexpsnapshot/__init__.py", line 779, in main ades.outputObjects() File "/root/ADExplorerSnapshot.py/adexpsnapshot/__init__.py", line 102, in outputObjects wq.put((dict(obj.attributes.data))) File "/root/ADExplorerSnapshot.py/adexpsnapshot/parser/classes.py", line 98, in data self._dico[prop.propName] = self.processAttribute(prop, entry.attrOffset, self.raw) File "/root/ADExplorerSnapshot.py/adexpsnapshot/parser/classes.py", line 189, in processAttribute if self.log: AttributeError: 'AttributeDict' object has no attribute 'log'
Implement changes per dirkjanm/BloodHound.py@4e61b27 to support new BH format. Bump setuptools accordingly.
In the program the -v
flag is currently not used.
Furthermore I would like to suggest to increase the commandline options by specifying an input file and output directory for more granular control in the case of dealing with multiple files.
I have encountered a problem
python3 ADExplorerSnapshot.py tests/data/Server.dat
[*] Server: .com
[] Time of snapshot:
[] Mapping offset: 0x2853a5
[] Object count: 3632
[+] Parsing properties: 1499
[+] Parsing classes: 269
[+] Parsing object offsets: 3632
[+] Restored pre-processed information from data cache
[v] Collecting data: 3169/3632 (0 users, 0 groups, 0 computers, 0 trusts)
Traceback (most recent call last):
File "/opt/anaconda3/lib/python3.8/site-packages/dissect/cstruct/types/instance.py", line 16, in getattr
return self._values[attr]
KeyError: 'category'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "ADExplorerSnapshot.py", line 2, in
adexpsnapshot.main()
File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 603, in main
ADExplorerSnapshot(fh, log)
File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 92, in init
self.process()
File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 158, in process
ret = fun(obj)
File "ADExplorerSnapshot.py/adexpsnapshot/init.py", line 364, in processUsers
if not (('user' in entry.classes and 'person' == entry.category) or 'msds-groupmanagedserviceaccount' in entry.classes):
File "ADExplorerSnapshot.py/adexpsnapshot/parser/classes.py", line 59, in getattr
return getattr(self._data, attr)
File "/opt/anaconda3/lib/python3.8/site-packages/dissect/cstruct/types/instance.py", line 18, in getattr
raise AttributeError("Invalid attribute: %r" % attr)
AttributeError: Invalid attribute: 'category'
How do I need to solve this problem?Thanks
Hello,
The import of computers stuck at some point when importing it into BH 4.2.0
Can anyone help me resolve this error?
┌──(kali㉿kali)-[~/Bloodhound/ADExplorerSnapshot.py]
└─$ sudo pip install --user .
[sudo] password for kali:
Processing /home/kali/Bloodhound/ADExplorerSnapshot.py
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting dissect.cstruct>=2.0
Downloading dissect.cstruct-2.0-py3-none-any.whl (30 kB)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from ADExplorerSnapshot==0.1.dev42+g590b947) (2.27.1)
Collecting pwntools>=4.5.0
Downloading pwntools-4.8.0-py2.py3-none-any.whl (11.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 6.2 MB/s eta 0:00:00
Collecting frozendict
Downloading frozendict-2.3.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (113 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 113.2/113.2 KB 13.2 MB/s eta 0:00:00
Collecting bloodhound>=1.2.0
Downloading bloodhound-1.4.0-py3-none-any.whl (70 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.1/70.1 KB 7.1 MB/s eta 0:00:00
Requirement already satisfied: ldap3!=2.5.0,!=2.5.2,!=2.6,>=2.5 in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.8.1)
Requirement already satisfied: dnspython in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.2.0)
Requirement already satisfied: impacket>=0.9.17 in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (0.9.24)
Requirement already satisfied: future in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (0.18.2)
Requirement already satisfied: pyasn1>=0.4 in /usr/lib/python3/dist-packages (from bloodhound>=1.2.0->ADExplorerSnapshot==0.1.dev42+g590b947) (0.4.8)
Requirement already satisfied: sortedcontainers in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.1.0)
Collecting rpyc
Downloading rpyc-5.2.3-py3-none-any.whl (71 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.3/71.3 KB 13.3 MB/s eta 0:00:00
Collecting unicorn>=1.0.2rc1
Downloading unicorn-2.0.0-py2.py3-none-manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.1/16.1 MB 7.5 MB/s eta 0:00:00
Requirement already satisfied: pysocks in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (1.7.1)
Collecting colored-traceback
Downloading colored-traceback-0.3.0.tar.gz (3.8 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: python-dateutil in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.8.1)
Requirement already satisfied: pyserial>=2.7 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (3.5)
Requirement already satisfied: six>=1.12.0 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (1.16.0)
Requirement already satisfied: paramiko>=1.15.2 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.8.1)
Collecting psutil>=3.3.0
Downloading psutil-5.9.1-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 282.0/282.0 KB 7.5 MB/s eta 0:00:00
Collecting intervaltree>=3.0
Downloading intervaltree-3.1.0.tar.gz (32 kB)
Preparing metadata (setup.py) ... done
Collecting ropgadget>=5.3
Downloading ROPGadget-6.8-py3-none-any.whl (31 kB)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (21.3)
Requirement already satisfied: pip>=6.0.8 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (22.0.2)
Collecting pathlib2
Downloading pathlib2-2.3.7.post1-py2.py3-none-any.whl (18 kB)
Collecting capstone>=3.0.5rc2
Downloading capstone-5.0.0rc2-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (2.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 7.0 MB/s eta 0:00:00
Collecting pyelftools>=0.2.4
Downloading pyelftools-0.29-py2.py3-none-any.whl (174 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 174.3/174.3 KB 8.3 MB/s eta 0:00:00
Requirement already satisfied: pygments>=2.0 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (2.11.2)
Requirement already satisfied: mako>=1.0.0 in /usr/lib/python3/dist-packages (from pwntools>=4.5.0->ADExplorerSnapshot==0.1.dev42+g590b947) (1.1.3)
Collecting plumbum
Downloading plumbum-1.7.2-py2.py3-none-any.whl (117 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.8/117.8 KB 5.6 MB/s eta 0:00:00
Building wheels for collected packages: ADExplorerSnapshot, intervaltree, colored-traceback
Building wheel for ADExplorerSnapshot (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for ADExplorerSnapshot (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [2012 lines of output]
..
.
.
.
.
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for ADExplorerSnapshot
Building wheel for intervaltree (setup.py) ... done
Created wheel for intervaltree: filename=intervaltree-3.1.0-py2.py3-none-any.whl size=26119 sha256=c5adb48c91f8ba94ee8d0b46ad72abead60c58045e46b7efee84df4988efb963
Stored in directory: /root/.cache/pip/wheels/fa/80/8c/43488a924a046b733b64de3fac99252674c892a4c3801c0a61
Building wheel for colored-traceback (setup.py) ... done
Created wheel for colored-traceback: filename=colored_traceback-0.3.0-py3-none-any.whl size=4622 sha256=e603e3140cca08f23dc046af7df4e3d50cf4db37ba4f9bd267970339ebbe8281
Stored in directory: /root/.cache/pip/wheels/10/49/bd/750e09783fb038570efede2d03819a7141fc2350de51daf575
Successfully built intervaltree colored-traceback
Failed to build ADExplorerSnapshot
ERROR: Could not build wheels for ADExplorerSnapshot, which is required to install pyproject.toml-based projects
Probably best as an additional script.
The data is there probably so let's parse it :)
Refs:
Update output format for compatibility with BloodHound-ly4k.
└─# python3 ADExplorerSnapshot.py result.dat
[] Time of snapshot: 2023-12-04T16:10:54
[] Mapping offset: 0xbffbbc3
[*] Object count: 139228
[+] Parsing properties: 3691
[+] Parsing classes: 543
[+] Parsing object offsets: 139228
[+] Restored pre-processed information from data cache
[◣] Collecting data: 6047/139228 (0 users, 0 groups, 0 computers, 35 certtemplates, 3 CAs, 0 trusts)
Traceback (most recent call last):
File "/home/kali/desktop/ADExplorerSnapshot.py/ADExplorerSnapshot.py", line 2, in
adexpsnapshot.main()
File "/home/kali/desktop/ADExplorerSnapshot.py/adexpsnapshot/init.py", line 1129, in main
ades.outputBloodHound()
File "/home/kali/desktop/ADExplorerSnapshot.py/adexpsnapshot/init.py", line 137, in outputBloodHound
self.process()
File "/home/kali/desktop/ADExplorerSnapshot.py/adexpsnapshot/init.py", line 265, in process
ret = fun(obj)
^^^^^^^^
File "/home/kali/desktop/ADExplorerSnapshot.py/adexpsnapshot/init.py", line 744, in processUsers
MembershipEnumerator.add_user_properties(user, entry)
TypeError: MembershipEnumerator.add_user_properties() missing 1 required positional argument: 'fileNamePrefix'
User-created machine accounts (e.g. with new-adcomputer cmdlet) are not included in the parsed output from AD Explorer snapshot. Per source code, it appears the object is not parsed correctly due to a lack of the "dnshostname" attribute for Computer objects created this way.
To replicate :
First, thanks for a wonderful piece of software. Worked like a charm :)
Do you plan on supporting Bloodhound-CE in near future, as it seems like the newer features will not go into the legacy one (ADCS...).
There is a branch for bloodhound.py to support BloodHound-CE (https://github.com/dirkjanm/BloodHound.py/tree/bloodhound-ce) but I have no idea how compatible it is with ADExplorerSnapshot.
Ran across some possible ADCS ESC7 false positives which only occur when using ADExplorerSnapshot to parse ADExplorer snapshots. These results did not occur when using ly4k's Certipy.
Steps to reproduce
I have not triaged the issue but wanted to open this thread in case anyone comes across something similar. Otherwise, the massive ADCS PR #10 by @PTVB has worked great.
Import Json files to Bloodhound 4.2.0 will be stuck at 77%
Another relation I missed was the admin to queries. Not quite sure if this data is in the adexplorer snapshot?
Great work on ADExplorerSnapshot.py! Love the tool and keep up the great work!
I do have one issue, and of course it might be my lab and not an issue. But it seems that if you have multiple domains and computer (or user) accounts are members of foreign groups, these are not added to the graph.
Tested with groups that are member of foreign groups and those works, however not computer or user accounts.
Is this a known issue?
Thanks
Hi there,
Thanks for the awesome tool!
Happy New Year to you and I hope you have a good 2022 so far!
I encountered the following error when using the tool.
Would you be able to assist on this?
└─# python3 ADExplorerSnapshot.py 2Jan2022.dat
[*] Server: Corp.local
[*] Time of snapshot: 2022-01-02T11:59:17
[*] Mapping offset: XXXXXX
[*] Object count: XXXXXXX
[+] Parsing properties: XXXXXXX
[+] Parsing classes:XXXXXXX
[+] Parsing object offsets: XXXXXXX
[+] Preprocessing objects: XXXXXXX sids, XXXXXXX computers, XXXXXXX domains with XXXXXXX DCs
[▇] Collecting data: 31006/44901 (XXXXXXX users, XXXXXXX groups, XXXXXXX computers, XXXXXXX trusts)
Traceback (most recent call last):
File "/root/Desktop/Tools/ADExplorerSnapshot.py/ADExplorerSnapshot.py", line 2, in <module>
adexpsnapshot.main()
File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 603, in main
ADExplorerSnapshot(fh, log)
File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init.py", line 92, in __init_
self.process()
File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 158, in process
ret = fun(obj)
File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 293, in processComputers
aces = self.parse_acl(computer, 'computer', ADUtils.get_entry_property(entry, 'nTSecurityDescriptor', raw=True))
File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 475, in parse_acl
aces = self._parse_acl_cached(haslaps, entrytype, acl)
File "/root/Desktop/Tools/ADExplorerSnapshot.py/adexpsnapshot/_init_.py", line 466, in _parse_acl_cached
_, aces = parse_binary_acl(fake_entry, entrytype, acl, self.objecttype_guid_map)
File "/root/.local/lib/python3.9/site-packages/bloodhound/enumeration/acls.py", line 142, in parse_binary_acl
if ace_object.acedata.get_object_type().lower() == objecttype_guid_map['ms-mcs-admpwd']:
File "/usr/lib/python3/dist-packages/requests/structures.py", line 54, in _getitem_
return self._store[key.lower()][1]
KeyError: 'ms-mcs-admpwd'
Since the method "_decompose" has been removed from "enum", this no longer works:
ADExplorerSnapshot.py/adexpsnapshot/__init__.py
Line 1060 in 38e5100
Certipy has the same issue: ly4k/Certipy#108
During the execution of the processDomains
function, the script encounters a TypeError
when trying to convert a None
value to an integer. This occurs when the level_id returned by
ADUtils.get_entry_propertyfor the property
msds-behavior-versionis
None`.
When level_id
is None
, the processDomains
function should set functional_level
to 'Unknown' and then continue with the rest of the function, without trying to cast None
to an integer.
Even when level_id
is None
, the function attempts to cast it to an integer and access the FUNCTIONAL_LEVELS
dictionary, resulting in a TypeError
.
Traceback (most recent call last):
File "ADExplorerSnapshot.py", line 2, in <module>
adexpsnapshot.main()
File "adexpsnapshot/__init__.py", line 1167, in main
ades.outputBloodHound()
File "adexpsnapshot/__init__.py", line 139, in outputBloodHound
self.process()
File "adexpsnapshot/__init__.py", line 289, in process
self.processDomains()
File "adexpsnapshot/__init__.py", line 301, in processDomains
functional_level = ADUtils.FUNCTIONAL_LEVELS[int(level_id)]
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
This error prevents the successful execution of the script and appears to be a logic error in the processDomains
function.
I suspect this is caused by ancient DFLs like Windows 2000.
fixed in pull request #19
The dictionary on the following line is not defined:
Pull requests coming your way.
Another issue is that the following attribute is not used, do you want to keep in in there for future use?
Great script and possibility to gain bloodhound data without bloodhound. I have used this approach for the first time and it was really useful. I noticed a few missing things, maybe not all of them are possible to add but lets see.
Thirst missing relation is the domain trust relationship in a multi domain environment. The data for this is definitely there. Would be great if this would be parsed in future releases.
Thanks mate.
Traceback (most recent call last):
File "/home/USER/Downloads/ADExplorerSnapshot.py/ADExplorerSnapshot.py", line 2, in <module>
adexpsnapshot.main()
File "/home/USER/Downloads/ADExplorerSnapshot.py/adexpsnapshot/__init__.py", line 1133, in main
ades.outputBloodHound()
File "/home/USER/Downloads/ADExplorerSnapshot.py/adexpsnapshot/__init__.py", line 129, in outputBloodHound
self.preprocessCached()
File "/home/USER/Downloads/ADExplorerSnapshot.py/adexpsnapshot/__init__.py", line 178, in preprocessCached
Pickler(open(cachePath, "wb")).dump(dico)
_pickle.PicklingError: Can't pickle <class 'types.wchar[]'>: attribute lookup wchar[] on types failed
Temporary solution was:
init.py
@@ -126,7 +126,8 @@ class ADExplorerSnapshot(object):
self.log.success(f"Output written to {outputfile}")
def outputBloodHound(self):
- self.preprocessCached()
+ # self.preprocessCached()
+ self.preprocess()
self.numUsers = 0
self.numGroups = 0
Export to a different format that can be reimported into an LDAP server or spin up a fake server.
Hello I have .dat file (very large .dat file 6.5G). When I am running your tool in the middle of the process, I get the following error. I use python3.10 according to others issues but I do not know what is going wrong. Maybe the size of the file is very large, I don't know. This is my error trace:
[+] Parsing properties: 4058
[+] Parsing classes: 637
[+] Parsing object offsets: 716095
[+] Preprocessing objects: 554340 sids, 136241 computers, 4 domains with 102 DCs
[◢] Collecting data: 25243/716095 (0 users, 0 groups, 0 computers, 90 certtemplates, 4 CAs, 9 trusts)
Traceback (most recent call last):
File "/usr/src/app/ADExplorerSnapshot.py", line 2, in
adexpsnapshot.main()
File "/usr/src/app/adexpsnapshot/init.py", line 1129, in main
ades.outputBloodHound()
File "/usr/src/app/adexpsnapshot/init.py", line 137, in outputBloodHound
self.process()
File "/usr/src/app/adexpsnapshot/init.py", line 265, in process
ret = fun(obj)
File "/usr/src/app/adexpsnapshot/init.py", line 651, in processTrusts
trust = domtrust.to_output()
File "/usr/local/lib/python3.10/site-packages/bloodhound/ad/trusts.py", line 90, in to_output
if self.has_flag('WITHIN_FOREST'):
File "/usr/local/lib/python3.10/site-packages/bloodhound/ad/trusts.py", line 87, in has_flag
return self.flags & self.trust_flags[flag] == self.trust_flags[flag]
TypeError: unsupported operand type(s) for &: 'NoneType' and 'int'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.