PhantomJS keeps timing out. about sketchy HOT 18 CLOSED

What URL are you trying to capture? Does this happen when you try www.google.com.

from sketchy.

You can also configure the PHANTOMJS_TIMEOUT in config-default.py

# Maximum time to wait for PhantomJS to generate a screenshot
PHANTOMJS_TIMEOUT = 35

Try changing that to 50 or 60 seconds.

from sketchy.

I've done as you said and changed it to 60 and it still keeps timing out. Here's an example URL I'm trying to capture from pastebin:

http://www.pastebin.com/Xm71fEcf

And the error message is:

[2015-10-01 14:48:18,910: ERROR/Worker-1] PhantomJS Capture timeout at 75 seconds

from sketchy.

Can you tell me a little bit about the host you are running on? Screenshot generation is pretty CPU intensive. That particular pastebin dump would generate a pretty large screenshot. Depending on the system's CPU you may want to set that threshold even higher.

Is Sketchy able to screenshot other sites with less content (google, etc.)?

from sketchy.

Running on Ubuntu 14.04 Trusty.
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz
cpu MHz : 2393.998
cache size : 3072 KB

Yes, sketchy is able to take screenshots of other sites with less content.

from sketchy.

I think it's just maxing out and taking too long to render. Try doubling the PHANTOMJS_TIMEOUT to 120 seconds or even a few minutes. Your Scumblr searches may take a bit longer to run but you may end up with more reliable results for really huge captures. Let me know if doubling to 120 helped.

from sketchy.

Will do and let you know how it goes tomorrow.

from sketchy.

I did double it to 120, still same thing. It keeps timing out. Instead of making Scumblr generate all screenshots including this, this time, I only requested a screenshot of the URL as to see if it was a CPU problem which I'm assuming I can conclude it isn't since this was the only screenshot requested.

from sketchy.

If it's working for other sites it just may be that it contains too much content to render a screenshot. If it's working for other sites that may indicate it's a PhantomJS screenshot generation issue, most likely related to Phantom.

from sketchy.

Understood,

Thanks.

from sketchy.

I'm having the same issue, I think. What's interesting about this particular issue is that it appears that sketchy is stuck on a particular request that is timing out and, as such, is not moving on to other requests. The request in issue has a job_status of "RETRY" and was created on 11/26. In the sketchy.log, I see:

2015-12-08 16:18:24,862 ERROR: PhantomJS Capture timeout at 60 seconds [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:114]
2015-12-08 16:18:24,862 ERROR: Command 'phantomjs capture' timed out after 60 seconds [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:354]
2015-12-08 16:18:26,816 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:18:26,822 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:18:27,121 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:18:27,126 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:18:27,538 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:18:27,543 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:18:29,146 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:18:29,153 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]

I'm not sure what to make of this yet, but I think a few things are seeming possible. First, it seems like sketchy was stuck in some weird loop as the moment I deleted the offended request (flagged as RETRY), all subsequent requests were made and completed. Second, this doesn't seem to be a timeout issue per se as the offending site had been collected before without any issue.

All that said, I don't have enough time with the sketchy code to really offer anything significant. Thoughts?

Thank you!

Josh

from sketchy.

Also, anyone know what the ERRORS are related to? If I tail sketchy.log I see output similar to below all the time.

2015-12-08 16:26:36,377 ERROR: PhantomJS Capture timeout at 60 seconds [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:114]
2015-12-08 16:26:36,378 ERROR: Command 'phantomjs capture' timed out after 60 seconds [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:354]
2015-12-08 16:26:38,531 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:38,540 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:26:38,901 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:38,908 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:26:39,388 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:39,395 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:26:43,545 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:43,552 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:26:44,028 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:44,034 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:26:44,479 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:44,486 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:26:54,482 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:54,489 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]
2015-12-08 16:26:54,975 ERROR: [in /mnt/data/apps/sketchy/sketchy/controllers/tasks.py:374]
2015-12-08 16:26:54,984 ERROR: [in /mnt/data/apps/sketchy/sketchy/init.py:68]

from sketchy.

@jwilczek Hi Josh, can you send me over the URL that was acting up? Also I think the errors are benign, I'll push a fix up to remove the logging (I think those were supposed to be informational logs not error).

from sketchy.

@jwilczek Please reopen this issue if you can provide the URL that was causing issues. w

from sketchy.

Sorry for the slow reply! In debugging the issue, I deleted the offending URL (without documenting it) and have yet to have another issue. I will reopen if this occurs again. Thanks!

from sketchy.

Ok, I'm seeing the issues again. I can share a URL (and my /api/v1.0/capture file), but I'd rather do it privately. Is that OK?

Also, when I delete the offending entries from within Scumblr, it seems sketchy is still trying to grab screenshots.

from sketchy.

Sure you can email me directly sbehrens @ gmail

On Wed, Jan 13, 2016 at 7:55 AM, jwilczek [email protected] wrote:

Ok, I'm seeing the issues again. I can share a URL (and my
/api/v1.0/capture file), but I'd rather do it privately. Is that OK?

—
Reply to this email directly or view it on GitHub
#26 (comment).

from sketchy.

Email sent. Thanks!

from sketchy.