Giter VIP home page Giter VIP logo

Comments (7)

qinxuye avatar qinxuye commented on September 8, 2024

这种问题都是使用错误导致的,现在我推荐clone下来develop分支,然后同样命令启动。如果要退出,就CTRL+C即可(需要等待退出)。有什么问题就反馈,我会及时fix。

ps. develop分支目前仅单机模式可用。

from cola.

LiuChaofan avatar LiuChaofan commented on September 8, 2024

我也遇到了这个问题,develop分支也有。

from cola.

qinxuye avatar qinxuye commented on September 8, 2024

这个问题是因为没有可抓取的对象导致的,确定starts中添加了相应的初始对象,以及这些对象都有好友去进行接下去的抓取?

from cola.

LiuChaofan avatar LiuChaofan commented on September 8, 2024

运行第一次的时候,能够正常的抓取数据。再运行的时候就会得到这个问题。我怀疑是将已抓取过的对象作为初始对象导致的,但这种情况应该直接结束程序才对。

from cola.

qinxuye avatar qinxuye commented on September 8, 2024

初始对象如果抓取过,是不会再接着抓取了。出现这个提示,只会是因为没有可以抓取的对象。

from cola.

LiuChaofan avatar LiuChaofan commented on September 8, 2024

如果初始对象是被抓取过的,而size为1,那是不是就算是没有可以抓取的对象?这时候程序应该要自己结束吧。

from cola.

qinxuye avatar qinxuye commented on September 8, 2024

不是这种逻辑。因为size是说明总的抓取数。而抓取的对象可能不光是初始的对象,也有可能是从初始对象得到的。而且,一个调度器调度worker抓取,如果worker反馈没有抓取对象,就退出,会导致很多问题,可能一个网页正在被解析,而要抓取的对象还没有被push到mq中。

另外,这个地方我有考量到动态的修改,也就是说这个时候虽然没有对象,用户可以直接修改比如添加初始对象,这个时候程序会继续运行,这种情况在后续有web接口等等的功能时,就会存在。

如果程序退出了,就不能做到这点。而实际上,用户自己CTRL+C退出代价也不大。

from cola.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.