shadawck / recon-archy Goto Github PK

in GUI mode the lazy loading is not a problem : just emulate a mouse scroll with selenium

FindElement as soon as the ressource is loaded : Try and retry operation every 100ms and 5 times, with Rety.Do()

Because rebuilding at each time 😒

https://medium.com/capital-one-tech/doing-well-by-doing-bad-writing-bad-code-with-go-part-1-2dbb96ce079a
https://medium.com/capital-one-tech/doing-well-by-doing-bad-writing-bad-code-with-go-part-2-e270d305c9f7
https://blog.gopheracademy.com/advent-2015/reducing-boilerplate-with-go-generate/

add Quit(), Close() and Interruption (Ctrl-C) Exit

panic: invalid session id: Tried to run command without establishing a connection

Need recover method :

• re-SignIn
• Go filter url
• Retrieve and go to last page number
• Continue crawling

happen when user identity are mask because you don't have enough relation

profileURL : #

Is this possible; Need to investigate

You will see something like this, if you want to perform search

Recon-Archy, you are a power searcher.
You’ve reached the monthly limit for profile searches. Upgrade to Premium Business to search and browse without limits.

Hi,

Hope you are all well !

Because of Linkedin's anti-scraping measures, it would be interesting to make the selenium browser look like an actual user. To do this, we need to add the li_at cookie to the selenium session.

Is it possible to add to recon-archy ?

Getting LI_AT

Navigate to www.linkedin.com and log in
Open browser developer tools (Ctrl-Shift-I or right click -> inspect element)
Select the appropriate tab for your browser (Application on Chrome, Storage on Firefox)
Click the Cookies dropdown on the left-hand menu, and select the www.linkedin.com option
Find and copy the li_at value

Setting LI_AT

$ export LI_AT=YOUR_LI_AT_VALUE

Cheers,
X

Loop through all users and all page and then export data

Certainly json