Comments (2)
Hi @RandomRobbieBF, I'll look into this and see what I can do.
from iocextract.
I've included a very simple version of this in the next release which does a shallow search within the JSON structure, but for your specific case, you can do something like this:
import json, base64
content = \
"""
[
{
"data": {
".dockerconfigjson": "ewoJImF1dGhzIjogewoJCSJjZGUtZG9ja2VyLXJlZ2lzdHJ5LmVpYy5mdWxsc3RyZWFtLmFpIjogewoJCQkiYXV0aCI6ICJZMlJsTFhKbFoybHpkSEo1T21Oa1pTMXlaV2RwYzNSeWVRPT0iCgkJfQoJfQp9"
}
}
]
"""
def validate_base64(data):
try:
if isinstance(data, str):
base64_bytes = bytes(data, 'ascii')
elif isinstance(data, bytes):
base64_bytes = data
else:
raise ValueError("Data type should be a string or bytes")
return base64.b64encode(base64.b64decode(base64_bytes)) == base64_bytes
except Exception:
return False
base64_db = []
def extract_embedded_base64(data):
for d in json.loads(data):
for _, value in d.items():
if validate_base64(value):
base64_db.append(base64.b64decode(value).decode("ascii"))
else:
for _, value in value.items():
if validate_base64(value):
base64_db.append(base64.b64decode(value).decode("ascii"))
for _, value in json.loads(base64.b64decode(value).decode("ascii")).items():
if validate_base64(value):
base64_db.append(base64.b64decode(value).decode("ascii"))
for _, value in value.items():
if validate_base64(value):
base64_db.append(base64.b64decode(value).decode("ascii"))
for _, value in value.items():
if validate_base64(value):
base64_db.append(base64.b64decode(value).decode("ascii"))
# Final decoded value
return base64_db[1] # Remove the "[1]" to view all values collected
print(extract_embedded_base64(content))
Albeit a bit recursive and repetitive, but it should validate values that are base64 encoded and iterate a few objects within the JSON data structure. This script is specific to your use case, so it may not work perfectly for every condition.
If you'd like to try the iocextract
feature in the future, you can use it like this once the new version is out, but this search is a little tamer and less specific:
import iocextract
content = \
"""
[
{
"data": "ewoJImF1dGhzIjogewoJCSJjZGUtZG9ja2VyLXJlZ2lzdHJ5LmVpYy5mdWxsc3RyZWFtLmFpIjogewoJCQkiYXV0aCI6ICJZMlJsTFhKbFoybHpkSEo1T21Oa1pTMXlaV2RwYzNSeWVRPT0iCgkJfQoJfQp9"
}
]
"""
print(list(iocextract.extract_encoded_urls(content, parse_json=True)))
from iocextract.
Related Issues (20)
- Fails to parse this url correctly HOT 3
- Failed to extract the URLs from this tweet HOT 6
- Binary Extraction HOT 1
- Extracting URLs that have been base64 encoded HOT 2
- 'https' scheme values defanged as HXXPS are refanged as 'http' HOT 3
- PyPi License Mismatch HOT 1
- catastrophic backtracking in BACKSLASH_URL_RE HOT 2
- extract_unencoded_url is too greedy when parsing Windows command lines HOT 4
- Improve documentation
- Improve extraction for non-defanged URLs HOT 6
- Review documentation HOT 1
- URL is not extracted correctly HOT 1
- module 'iocextract' has no attribute 'refang_url' HOT 2
- ModuleNotFoundError: No module named 'iocextract' HOT 6
- Add the function --extract-domains and --extract-subdomains HOT 2
- BUG: --extract-ipv4s does not work HOT 3
- Add a function to import directly from a server and extract IOCs. HOT 2
- how do I add a ioc_type label with the output? HOT 1
- Found IPs being parsed as URLs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from iocextract.