[Feature Request] Composing Spiders about core HOT 4 CLOSED

This should be possible in the upcoming 1.0 release since a new Roach::collectSpider(...) method gets added that returns all scraped items. (Related issue #35)

from core.

Unless I’m overlooking something (on my phone right now), you should be able to call Roach::startSpider from within another spider as well, right?

from core.

Unless I’m overlooking something (on my phone right now), you should be able to call Roach::startSpider from within another spider as well, right?

Yes, but parse() is expected to return a Generator<ParseResult>.

Roach::startSpider has a void return type.

So, this would not work then, right?

foreach ($pages as $page) {
    yield Roach::startSpider(BlogPageSpider::class, overrides: new Overrides([startUrls: $page->getUri()]));
}

Or did you mean something else?

from core.

Version 1.0 got released. This should now be possible by using Roach::collectSpider instead of Roach::startSpider. Here’s the relevant section of the docs.

from core.