qeeqbox / social-analyzer Goto Github PK

Why do the install instructions say that one should install firefox esr from a ppa instead of just using regular firefox from the normal distribution mirrors?

removed

During some testing sessions, I had some hard times to figure why the screenshot feature was not working as expected. After few hours of debugging, I've found the culprit 😅

web-automation_firefox | SE_EVENT_BUS_HOST not set, exiting!
web-automation_firefox | 2021-12-04 01:54:01,992 INFO exited: selenium-node (exit status 1; not expected)

I then did some research and found that the defined variables was not correct anymore, certainly due to version changes since last time the docker-compose.yml has been created and/or updated.

I've already created a pull request that contains the fix and resolution proofs here: #75

Hello, I've been using social analyzer programatically and it looks like, after a couple of days running 24/7, it's generated about 2TB of data in the /tmp directory. Do you know any way to avoid logs being generated automatically by the program?

it keeps hanging at google Gmail facebook and other random sites

I can open a PR to add Qvault developer profiles if you think it would be a good fit. Let me know :)

no results, but the profil exists https://www.facebook.com/jan.bor.7169/

can you help?

At https://github.com/qeeqbox/social-analyzer#linux-as-python-script the last line should be:

python3 app.py --username "johndoe" --metadata --top 100

Hi

/usr/bin/python3: No module named social-analyzer

nothing has changed still no results, what's wrong?

npm start

[email protected] start
node server.js

Server started at http://localhost:9005/app.html
Driver Session Issue
Total time 320
(node:10846) UnhandledPromiseRejectionWarning: WebDriverError: Process unexpectedly closed with status 1
at Object.throwDecodedError (/home/osint/social-analyzer/node_modules/selenium-webdriver/lib/error.js:550:15)
at parseHttpResponse (/home/osint/social-analyzer/node_modules/selenium-webdriver/lib/http.js:565:13)
at Executor.execute (/home/osint/social-analyzer/node_modules/selenium-webdriver/lib/http.js:491:26)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:10846) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:10846) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Driver Session Issue
Total time 249
(node:10846) UnhandledPromiseRejectionWarning: WebDriverError: Process unexpectedly closed with status 1
at Object.throwDecodedError (/home/osint/social-analyzer/node_modules/selenium-webdriver/lib/error.js:550:15)
at parseHttpResponse (/home/osint/social-analyzer/node_modules/selenium-webdriver/lib/http.js:565:13)
at Executor.execute (/home/osint/social-analyzer/node_modules/selenium-webdriver/lib/http.js:491:26)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:10846) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)

Server started at http://localhost:9005/app.html
2020-12-28T09:17:45.841182+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
2020-12-28T09:17:45.863001+00:00 heroku[web.1]: Stopping process with SIGKILL
2020-12-28T09:17:45.953722+00:00 heroku[web.1]: Process exited with status 137
2020-12-28T09:17:45.999542+00:00 heroku[web.1]: State changed from starting to crashed
2020-12-28T09:18:39.390728+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=shrouded-citadel-37315.herokuapp.com request_id=-6b6f--af76 fwd="" dyno= connect= service= status=503 bytes= protocol=https

I've tested on my own profile, and stefanofinding's profile, but nothing came up. I'm using the tags FindUserProfilesSlow, ShowUserProfilesSlow, FindUserProfilesSpecial on the web interface.
Selected "hackerone" on advanced options. No google API, no proxy, and default user-agent string.

I tested it beforehand with my github profile and it did detect that one.

When a service is open-source and can be self-hosted, this often leads to it being self-hosted on many different domains.

Examples are like:

• Mastodon
  • https://fosstodon.org/@{username}
  • https://mastodon.cloud/@{username}
  • https://mastodon.social/@{username}
  • https://mastodon.technology/@{username}
  • https://mastodon.xyz/@{username}
  • https://mstdn.io/@{username}
• Discourse
  • https://ask.fedoraproject.org/u/{username}
  • https://community.signalusers.org/u/{username}
  • https://community.cloudflare.com/u/{username}
  • https://community.bitwarden.com/u/{username}/summary
  • https://discourse.wicg.io/u/{username}/summary
  • https://discuss.atom.io/u/{username}/summary
  • https://forum.leasehackr.com/u/{username}/summary
  • https://discuss.elastic.co/u/{username}
  • https://forum.sublimetext.com/u/{username}
  • https://forums.whonix.org/u/{username}
• GitLab
  • https://gitlab.com/{username}
  • https://gitlab.gnome.org/{username}
• Weblate
  • https://hosted.weblate.org/user/{username}/
• MediaWiki
  • https://wikipedia.org/wiki/User:{username}

Only including links that exist in the repository.

Most Mastodon instances can share detections for example, same with all Discourse instances.
The only time this may not be the case is if a particular service is using a very outdated version of the software which then it may have to be separated.

It might help reduce the maintenance effort and total size of sites.json, by adding a way that many domains can share the same detections rather than writing/copying the same detections multiple times?

for example, run the following query in which the name is "xxxxxxxxxxxxxxxxxxxxxxx":

python3 -m social-analyzer --cli --mode "fast" --username "xxxxxxxxxxxxxxxxxxxxxx" --websites "all" --output "pretty"

will result in this:

> 
link     : https://mixcloud.com/xxxxxxxxxxxxxxxxxxxxxx
title    : Mixcloud
language : unavailable
type     : Audio Streaming
-----------------------
link     : https://nightbot.tv/t/xxxxxxxxxxxxxxxxxxxxxx/commands
title    : Nightbot
language : unavailable
type     : unavailable
....
....
edit
....
....
-----------------------
[failed] 7 Profile[s]
-----------------------
link     : https://codeberg.com/xxxxxxxxxxxxxxxxxxxxxx
-----------------------
link     : https://forum.igromania.ru/member.php?username=xxxxxxxxxxxxxxxxxxxxxx
-----------------------
link     : https://leetcode.com/xxxxxxxxxxxxxxxxxxxxxx
-----------------------
link     : https://pling.com/u/xxxxxxxxxxxxxxxxxxxxxx
-----------------------
link     : https://plug.dj/@/xxxxxxxxxxxxxxxxxxxxxx
-----------------------
link     : https://uid.me/xxxxxxxxxxxxxxxxxxxxxx
-----------------------
link     : https://xxxxxxxxxxxxxxxxxxxxxx.wix.com

> 

According to my tests, most of the results returned with confidence scores below 60 percent either point to non-existent users or are useless.

All detections fail on Ubuntu 20.04 using the Python 3-PIP version of the software.
Tested for my own accounts and returned no results.

I cannot seem to get the google key and the CS key to work, most likely I'm doing it wrong..what is the correct way to generate those keys from the console.
How do you enable the duck duck go api search option ?

Hi, is there a way to search for all the possible websites instead of just the ones you have to input? For example, instead of --websites "instagram facebook youtube" and so on, is there a way to just say --websites "all"?

This will allow us to search a person using different usernames across different platforms.

I tried the docker container but I get a bunch of unhandled errors:

Driver Session Issue
 --- Uncaught Rejection ---
  TimeoutError: Connection refused (os error 111)
  
  - error.js:550 Object.throwDecodedError
    [app]/[selenium-webdriver]/lib/error.js:550:15
  
  - http.js:565 parseHttpResponse
    [app]/[selenium-webdriver]/lib/http.js:565:13
  
  - http.js:491 Executor.execute
    [app]/[selenium-webdriver]/lib/http.js:491:26
  
  - runMicrotasks
  
  - task_queues.js:93 processTicksAndRejections
    internal/process/task_queues.js:93:5

Perhaps they should be properly handled

Just a o see if I can accomplish a project

Can you integrate with sherlock to extract even more information about a profile but with in one console, thanks.
https://github.com/sherlock-project/sherlock

I don’t see it addressed in the README, but I think it’s an important consideration.

Will users see much more captchas if they run this from their home/work IP? Will some sites block the users from doing queries, and if so: what happens to those results?

How to run as CLI on Windows?

• - How can i create a html email address or account for auto reply or costumer service section of company???

😖😭😃

README.md contains the following instructions:

git clone https://github.com/qeeqbox/social-analyzer
cd social-analyzer

However, the folder social-analyzer seems to be missing.

Is this intended?
And is there a way to pull that missing folder?

Issue

When working this week on your project, I saw that docker-compose download and install command with curl is pointing to an outdated version of docker-compose.

Where

The issue is located in the project wiki here: https://github.com/qeeqbox/social-analyzer/wiki/install#docker-compose

What's need to be changed?

According to the official documentation, the line:

sudo curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Should be replaced by:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Additional changes for docker-compose

Since you've accepted and merged my pull request, why not add the following?

# By default (2 CPU cores)
docker-compose -f docker-compose.yml up --build

# Using all available CPU cores
echo "CPU_CORES=`nproc`" > .env ; docker-compose -f docker-compose.yml up --build

# Using only some CPU cores (if more than 4)
echo "CPU_CORES=4" > .env ; docker-compose -f docker-compose.yml up --build

This would make the docker-compose install instructions from this:

sudo apt-get update
sudo apt-get install -y docker.io git curl
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
git clone https://github.com/qeeqbox/social-analyzer.git
cd social-analyzer
docker-compose -f docker-compose.yml up --build

To that:

# Install updates, docker-compose and project
sudo apt-get update
sudo apt-get install -y docker.io git curl
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
git clone https://github.com/qeeqbox/social-analyzer.git
cd social-analyzer

# Default deployment (2 CPU cores)
docker-compose -f docker-compose.yml up --build

# Deploy and use all available CPU cores
echo "CPU_CORES=`nproc`" > .env ; docker-compose -f docker-compose.yml up --build

# Deploy and use only some CPU cores (if more than 4)
echo "CPU_CORES=4" > .env ; docker-compose -f docker-compose.yml up --build

Note: My pull request has been deployed with docker-compose version 1.29.2 which is the latest release from the 1.x.x branch as of writing.

Warning

Docker has recently released the compose plugin as docker-compose V2 and compose-switch to smoothly handle the transition from the version 1.x.x (docker-compose) to the version 2.x.x (docker compose).

So, maybe it would be useful to say that this project has been tested and works on V1 (docker-compose) but probably not with V2 (docker compose) or needs additional testing to be documented.

I was unsure myself which version I would use knowing that the currently documented version 1.26.2 was outdated and so wanted to use the latest one and then I saw that a V2 where out but was working differently and was no longer a single binary but now a docker plugin and docker-switch should be used to handle the transition.

Comments

I wanted to create a pull request along this issue but I could not find any edit links in the Wiki 😅

installed this on kali linux
it was too confusing
as i could not find CLI or URL Mentioned is not opening

please do a guided setup for Kali Linux

help appreciated

kudos to you effort making life easier