ndemir / google-site-spider Goto Github PK

Where is everything saved? There doesn't seem to be a basic manual for this.

What steps will reproduce the problem?
1. Run site spider on any website.

What is the expected output?
Full url.

What do you see instead?
Abbreviated version.


What version of the product are you using?
Latest.

On what operating system?
Chrome


Please provide any additional information below.
I'm using a netbook. The screen width is small. An easy fix would be resizable 
columns or just unabbreviated urls.

Use a mailto url on a page and it will be listed as being unable to load.  They 
should probably be ignored.

Or if feeling particularly adventurous a check can be made to see if a DNS 
record exists for the domain. 

I tried running this on my personal website using Chrome 12.0.742.53 beta for 
64 bit Linux (Natty).

It starts out okay queueing 200 items or so and filling the page as it grabs 
them but then ends up hanging (can't scroll or interact).  The second time I 
tried it didn't even get to the bottom of the page before hanging.

Going back to the tab where I pressed the button in the first place and 
pressing again has no effect unless I restart the browser.

I have no idea where the blame for this happening lies and the extension page 
doesn't list how to gather diagnostics to help this kind of issue.

I wish I could pause/stop the processes.  The only way to stop is to close the 
window and then i can't see the results.

Finer grained blacklists would be sweet as well.

This is a sweet project Thanks!

Attempting to spider any site will open the results tab but fail to get past 
initialization. It appears that the underlying problem is that 
chrome.extension.getViews({"type": "tab"}) returns an empty list rather than 
the newly created tab, which appears to be a Chrome bug at first glance. It 
might  be possible to work around this by refactoring the callback so it stores 
the new tab reference rather than needing to find it periodically.

One of my pages links to http://wiki.github.com/ook/isbg but the spider lists 
it as unable to load.

The URL is fine although retrieving it does a 301 permanent redirect to an 
https version of the URL.  Arguably the extension should also list permanent 
redirects as such so that the source can be changed to point to the new 
permanent URL.

What steps will reproduce the problem?
1. It keeps "initializing" and do not do anything else

Run scanner in: http://www.nerv.fi/
Result: No links detected
Version: 1.2

Can someone verify?

What steps will reproduce the problem?
1.Run the spider
2.Try to copy the results onto a google doc or a spreadsheet

What is the expected output? What do you see instead?
Copying the list (if it's too long) doesn't allow google docs to paste from the 
clipboard - Is there an easier way to save/export this report?


What version of the product are you using? On what operating system?
Win XP, Chrome