xianyunyh / spider_job Goto Github PK

您好，我按仓库要求装好了所有的库和redis、Mongodb，执行scrapy crawl boss毫无反应。注：打印的Action是我在类里加的调试信息，我在def parse方法中也加了，但不打印，说明没有执行def parse方法。我不知道为什么，Mongodb中也没数据。

另外，我也注意到您在Settings里写了：HTTP_PROXY = 'http://127.0.0.1:8123/' 这个代理。这个用的是什么ip代理需要我这边怎么配置呢？求教，非常感谢！

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

文档中安装命令如下：
pip install -f requirements.txt

但是上面命令不能执行。
是不是应该改成: pip install -r requirements.txt

Hi，老哥用于在线预览的服务器似乎出了点问题，接口全部返回 502 Bad Gateway，有空排查下原因吧。

File "E:\Anaconda3\envs\pachong\lib\site-packages\scrapy\downloadermiddlewares\retry.py", line 24, in
from scrapy.core.downloader.handlers.http11 import TunnelError
File "E:\Anaconda3\envs\pachong\lib\site-packages\scrapy\core\downloader\handlers\http11.py", line 26, in
from scrapy.core.downloader.webclient import _parse
File "E:\Anaconda3\envs\pachong\lib\site-packages\scrapy\core\downloader\webclient.py", line 4, in
from twisted.web.client import HTTPClientFactory
ImportError: cannot import name 'HTTPClientFactory' from 'twisted.web.client' (unknown location)