Giter VIP home page Giter VIP logo

Comments (12)

penndu avatar penndu commented on August 15, 2024

调用页面为https://dusays.com/fcircle/

from hexo-circle-of-friends.

hiltay avatar hiltay commented on August 15, 2024

yml已经弃用很久了,请按照最新文档部署https://fcircle-doc.js.cool/

from hexo-circle-of-friends.

penndu avatar penndu commented on August 15, 2024

yml已经弃用很久了,请按照最新文档部署https://fcircle-doc.js.cool/

Dockerfile如下

FROM python:3.8
MAINTAINER yyyz
COPY . /
### 在这里配置环境变量
### 通用配置
ENV RUN_PER_HOURS=1
#ENV PROXY=""
### leancloud配置
#ENV APPID=""
#ENV APPKEY=""
### mysql配置
ENV MYSQL_USERNAME="USERNAME"
ENV MYSQL_PASSWORD="PASSWORD"
ENV MYSQL_IP="IP"
ENV MYSQL_PORT="3306"
ENV MYSQL_DB="DB"
### mongodb配置
#ENV MONGODB_URI=""
EXPOSE 8000
WORKDIR /
RUN cd ./hexo_circle_of_friends && pip3 install -r requirements.txt -i https://pypi.douban.com/simple/
CMD bash ./docker.sh

from hexo-circle-of-friends.

penndu avatar penndu commented on August 15, 2024

settings.py如下:

LINK = [
    {
        "link": "https://dusays.com/friends/",  # 友链页地址1,修改为你的友链页地址
        "theme": "volantis"
BLOCK_SITE = [
    "https://dusays.com/",
    # "https://example.com/",
]

# 过期文章清除(天)
OUTDATE_CLEAN = 600

# 存储方式,可选项:leancloud,mysql,sqlite,mongodb;默认为leancloud
DATABASE = "mysql"

# 部署方式,可选项:github,server,docker;默认为github
DEPLOY_TYPE = "docker"

from hexo-circle-of-friends.

penndu avatar penndu commented on August 15, 2024

api.log如下

INFO:     Started server process [16]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     153.35.52.197:60246 - "GET /all HTTP/1.1" 200 OK
INFO:     153.35.52.191:47341 - "GET /favicon.ico HTTP/1.1" 404 Not Found
INFO:     61.147.73.164:43327 - "GET / HTTP/1.1" 404 Not Found
INFO:     61.147.73.164:49243 - "GET / HTTP/1.1" 404 Not Found
INFO:     61.147.73.164:20469 - "GET / HTTP/1.1" 404 Not Found
WARNING:  Invalid HTTP request received.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 136, in handle_events
    event = self.conn.next_event()
  File "/usr/local/lib/python3.8/site-packages/h11/_connection.py", line 485, in next_event
    exc._reraise_as_remote_protocol_error()
  File "/usr/local/lib/python3.8/site-packages/h11/_util.py", line 77, in _reraise_as_remote_protocol_error
    raise self
  File "/usr/local/lib/python3.8/site-packages/h11/_connection.py", line 467, in next_event
    event = self._extract_next_receive_event()
  File "/usr/local/lib/python3.8/site-packages/h11/_connection.py", line 409, in _extract_next_receive_event
    event = self._reader(self._receive_buffer)
  File "/usr/local/lib/python3.8/site-packages/h11/_readers.py", line 81, in maybe_read_from_IDLE_client
    raise LocalProtocolError("illegal request line")
h11._util.RemoteProtocolError: illegal request line
INFO:     68.183.234.55:36770 - "GET / HTTP/1.1" 404 Not Found
INFO:     209.17.97.10:35923 - "GET / HTTP/1.1" 404 Not Found
INFO:     83.97.20.34:26630 - "GET / HTTP/1.0" 404 Not Found
INFO:     140.205.80.137:48063 - "GET /all?rule=created&start=0&end=20 HTTP/1.1" 200 OK
INFO:     120.27.74.170:60391 - "GET /all?rule=created&start=0&end=20 HTTP/1.1" 200 OK
INFO:     120.27.74.152:42392 - "GET /all HTTP/1.1" 200 OK
INFO:     120.27.74.159:22585 - "GET /favicon.ico HTTP/1.1" 404 Not Found
INFO:     120.27.74.170:49428 - "GET /all?rule=created&start=0&end=20 HTTP/1.1" 200 OK

from hexo-circle-of-friends.

penndu avatar penndu commented on August 15, 2024

crawler.log如下

Initialization complete
----------------------
友链总数 : 0
失联友链数 : 0
共 0 篇文章
最后运行于:2022-03-28 01:46:48
done!
Initialization complete
----------------------
友链总数 : 0
失联友链数 : 0
共 0 篇文章
最后运行于:2022-03-28 07:46:57
done!
Initialization complete
----------------------
友链总数 : 0
失联友链数 : 0
共 0 篇文章
最后运行于:2022-03-28 08:46:58
done!

from hexo-circle-of-friends.

hiltay avatar hiltay commented on August 15, 2024

第一个问题:感谢反馈,是我的疏忽。mysql api的连接问题已经紧急修复,尝试重新clone一下代码再部署

第二个问题:看了你的友链页面,抓不到文章应该是css未匹配。解决方案1:尝试适配友链,参考友链页适配;解决方案2:原理同解决方案1,尝试修改仓库内./hexo-circle-of-friends/get_url.py下面的get_volantis_url方法的抓取规则以适配你的前端页面;解决方案3:尝试使用配置项友链,在settings.py中配置即可;

(第三个问题:github上暴露了你的mysql连接配置,注意安全,及时修改密码)

from hexo-circle-of-friends.

penndu avatar penndu commented on August 15, 2024

请教这种的如何写抓取规则?

<div class='content'>
	<ul class="list entry navigation">
		<li><a class="flat-box" title="http://dusays.com/" href="http://dusays.com/" active-action="action-http:dusayscom">
			<div class='name'>
				<img src="https://cdn.dusays.com/avatar.png" class="lazyload" data-srcset="https://cdn.dusays.com/avatar.png" srcset="data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==">
				杜老师说
			</div>
		</a></li>
    </ul>
</div>

from hexo-circle-of-friends.

penndu avatar penndu commented on August 15, 2024

这样是否可行?

avatar = response.css('a.name img::attr(src)').extract()
link = response.css('a.flat-box a::attr(href)').extract()
name = response.css('a.name a::text').extract()

from hexo-circle-of-friends.

hiltay avatar hiltay commented on August 15, 2024

暂时不考虑对特例做适配。如果采用解决方案2,授人以渔,可以参考CSS 选择器参考手册
另外补充解决方案4:尝试使用配置项友链的json_api同样可以达到效果;

from hexo-circle-of-friends.

penndu avatar penndu commented on August 15, 2024

暂时不考虑对特例做适配。如果采用解决方案2,授人以渔,可以参考CSS 选择器参考手册; 另外补充解决方案4:尝试使用配置项友链的json_api同样可以达到效果;

非常感谢!

from hexo-circle-of-friends.

2X-ercha avatar 2X-ercha commented on August 15, 2024

已经部署上了而且没有问题的话这个issue就可以close了

from hexo-circle-of-friends.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.