fugary / calibre-douban Goto Github PK

Calibre new douban metadata source plugin. Douban no longer provides book APIs to the public, so it can only use web crawling to obtain data. This is a calibre Douban plugin based on web crawling.

License: Apache License 2.0

Python 100.00%

calibre-douban's Introduction

calibre-douban

Calibre douban metadata download plugin. Based on https://book.douban.com web pages.

Calibre插件

最近在使用calibre-web管理电子书，不过很多时候还是需要用到Calibre桌面版软件，批量管理，编辑电子书等功能，在calibre-web上已经使用calibre-web-douban-api搜素豆瓣元数据，但是桌面版Calibre软件缺没有办法使用，不过calibre可以使用插件，而且是使用python开发，因此可以把calibre-web-douban-api改造一下包装成calibre插件，简单元数据插件还是比较容易的

安装方法

下载地址：NewDouban.zip

从release页面下载zip包，然后再calibre中安装为插件即可。

参考文档：https://fugary.com/?p=423

calibre-douban's People

Contributors

Stargazers

Watchers

calibre-douban's Issues

无法使用了，错位代码403

#################### Log for 控制论(或关于在动物和机器中控制和通信的科学) (科学元典丛书) ####################
Running identify query with parameters:
{'title': '控制论(或关于在动物和机器中控制和通信的科学) (科学元典丛书)', 'authors': ['维纳'], 'identifiers': {'asin': 'B008H0V4QE'}, 'timeout': 30}
Using plugins: New Douban Books (1, 0, 7)
The log from individual plugins is below

****************************** New Douban Books (1, 0, 7) ******************************
Found 0 results
Downloading from New Douban Books took 0.6469862461090088
Plugin New Douban Books failed
Traceback (most recent call last):
File "calibre\ebooks\metadata\sources\identify.py", line 47, in run
File "calibre_plugins.dummy1.init", line 293, in identify
File "calibre_plugins.dummy1.init", line 61, in search_books
File "calibre_plugins.dummy1.init", line 45, in load_book_urls
File "urllib\request.py", line 216, in urlopen
File "urllib\request.py", line 525, in open
File "urllib\request.py", line 634, in http_response
File "urllib\request.py", line 557, in error
File "urllib\request.py", line 496, in _call_chain
File "urllib\request.py", line 749, in http_error_302
File "urllib\request.py", line 525, in open
File "urllib\request.py", line 634, in http_response
File "urllib\request.py", line 563, in error
File "urllib\request.py", line 496, in _call_chain
File "urllib\request.py", line 643, in http_error_default
urllib.error.HTTPError: HTTP Error 403: Forbidden

The identify phase took 0.83 seconds
The longest time (0.646986) was taken by: New Douban Books
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds
Failed to download metadata for 控制论(或关于在动物和机器中控制和通信的科学) (科学元典丛书)

****************************** New Douban Books Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko')]
Failed to download valid cover
Took 0.6448569297790527 seconds
No cached cover found, running identify
Failed to download cover from New Douban Books
Traceback (most recent call last):
File "calibre\ebooks\metadata\sources\covers.py", line 46, in run
File "calibre_plugins.dummy1.init", line 227, in download_cover
File "calibre_plugins.dummy1.init", line 293, in identify
File "calibre_plugins.dummy1.init", line 61, in search_books
File "calibre_plugins.dummy1.init", line 45, in load_book_urls
File "urllib\request.py", line 216, in urlopen
File "urllib\request.py", line 525, in open
File "urllib\request.py", line 634, in http_response
File "urllib\request.py", line 557, in error
File "urllib\request.py", line 496, in _call_chain
File "urllib\request.py", line 749, in http_error_302
File "urllib\request.py", line 525, in open
File "urllib\request.py", line 634, in http_response
File "urllib\request.py", line 563, in error
File "urllib\request.py", line 496, in _call_chain
File "urllib\request.py", line 643, in http_error_default
urllib.error.HTTPError: HTTP Error 403: Forbidden

建议将豆瓣评分输出为自定义的新列

目前插件抓取到的豆瓣评分输出为calibre默认的评分一列。但是默认的评分只能设置1至5颗星。而豆瓣评分为0至10并带有一位小数的值。因此这种输出方式会造成评分精确度的丢失。
从事实上来看，一方面，calibre 默认的评分可以在图书列表中直接改动；另一方面，评分只能打1~5这五个值。个人认为这样的操作逻辑暗示着这种评分应当应用于个人基于阅读体验后的主观打分。
基于以上原因，建议将豆瓣评分输出为自定义的新列，而将默认的评分预留给读者个人打分使用。

书名+作者+年份正确，但基本上匹配不到正确信息

如题，在我已经填写好全部正确元数据的情况下，插件都匹配不到正确信息。。。有时候同一本书，连续几次会匹配到不同的结果。。。

所以你的匹配逻辑到底是什么？？

亲测豆瓣客户端，基本上title+author+year，查询结果里面最上面的就是需要的书，但你的插件竟然能在元数据完全正确的情况下完美错过正确结果。

你的这个逻辑就很迷。。

建议：
将查询关键词设定为：title+author+year+publisher+isbn

目测title+author+year匹配准确度最高，基本上搜出来的第一本就是

搜isbn只对单本书有用，很多丛书公用一个isbn，很多书的不同版本共用一个isbn，很多古书没有isbn

似乎又不可用了

豆瓣403了

麻烦修复一下

AttributeError: 'NoneType' object has no attribute 'xpath'

****************************** New Douban Books (2, 0, 1) ****************************** 
Found 0 results 
Downloading from New Douban Books took 8.697023868560791 
Random sleep time 0.013196636524950978s
Random sleep time 0.03457691943154163s
Random sleep time 0.05430206628676855s
Random sleep time 0.04697191791639523s
Random sleep time 0.0038138277025330457s
Downloaded:https://book.douban.com/subject/26636067/ Successful,Time 5161ms
Downloaded:https://book.douban.com/subject/21993088/ Successful,Time 5268ms
Plugin New Douban Books failed
Traceback (most recent call last):
  File "site-packages\calibre\ebooks\metadata\sources\identify.py", line 47, in run
  File "calibre_plugins.dummy1.__init__", line 328, in identify
  File "calibre_plugins.dummy1.__init__", line 68, in search_books
  File "concurrent\futures\_base.py", line 432, in result
  File "concurrent\futures\_base.py", line 388, in __get_result
  File "concurrent\futures\thread.py", line 57, in run
  File "calibre_plugins.dummy1.__init__", line 82, in load_book
  File "calibre_plugins.dummy1.__init__", line 113, in parse_book
AttributeError: 'NoneType' object has no attribute 'xpath'

Downloaded:https://book.douban.com/subject/1939989/ Successful,Time 5653ms
Downloaded:https://book.douban.com/subject/2154092/ Successful,Time 6515ms
Downloaded:https://book.douban.com/subject/1340758/ Successful,Time 10528ms

Calibre版本：5.1
New Douban插件已安装，并填写Cookie信息，但不知为何报了AttributeError，是插件相关的错误么？是否可以加一个Exception忽略掉？

时间线补充：

大约一年前左右初安装时，能够成功获取元数据
之后某次使用时发现豆瓣更新了安全规则，参照文档添加了Cookie后，能够成功获取元数据

如需要提供其他相关日志信息，请说明。

书籍标签

cxfksword/douban-api-rs 这个项目是可以获取书籍的tag的，请问能不能加上书籍标签的功能呢

远程端关闭连接,无响应错误

Windows PC平台下
calibre: 6.21.0版本
calibre-douban 插件: v2.0.0

错误日志:
`
#################### Log for 朗读者 ####################
Running identify query with parameters:
{'title': '朗读者', 'authors': ['[德国] 本哈德·施林克', '钱定平'], 'identifiers': {'mobi-asin': '049db565-dd65-4907-a8d6-07f2505a6ae9', 'isbn': '9787544707923'}, 'timeout': 30}
Using plugins: New Douban Books (2, 0, 0)
The log from individual plugins is below

****************************** New Douban Books (2, 0, 0) ******************************
Found 0 results
Downloading from New Douban Books took 0.051558494567871094
Plugin New Douban Books failed
Traceback (most recent call last):
File "urllib\request.py", line 1348, in do_open
File "http\client.py", line 1282, in request
File "http\client.py", line 1328, in _send_request
File "http\client.py", line 1277, in endheaders
File "http\client.py", line 1037, in _send_output
File "http\client.py", line 975, in send
File "http\client.py", line 1447, in connect
File "http\client.py", line 951, in connect
File "http\client.py", line 920, in _tunnel
File "http\client.py", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\ebooks\metadata\sources\identify.py", line 47, in run
File "calibre_plugins.dummy1.init", line 326, in identify
File "calibre_plugins.dummy1.init", line 63, in search_books
File "calibre_plugins.dummy1.init", line 48, in load_book_urls_new
File "urllib\request.py", line 216, in urlopen
File "urllib\request.py", line 519, in open
File "urllib\request.py", line 536, in _open
File "urllib\request.py", line 496, in _call_chain
File "urllib\request.py", line 1391, in https_open
File "urllib\request.py", line 1351, in do_open
urllib.error.URLError:

The identify phase took 0.21 seconds
The longest time (0.051558) was taken by: New Douban Books
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds
Failed to download metadata for 朗读者

****************************** New Douban Books Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36 Edg/87.0.664.75')]
Failed to download valid cover
Took 0.014216899871826172 seconds
No cached cover found, running identify
Failed to download cover from New Douban Books
Traceback (most recent call last):
File "urllib\request.py", line 1348, in do_open
File "http\client.py", line 1282, in request
File "http\client.py", line 1328, in _send_request
File "http\client.py", line 1277, in endheaders
File "http\client.py", line 1037, in _send_output
File "http\client.py", line 975, in send
File "http\client.py", line 1447, in connect
File "http\client.py", line 951, in connect
File "http\client.py", line 920, in _tunnel
File "http\client.py", line 287, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\ebooks\metadata\sources\covers.py", line 46, in run
File "calibre_plugins.dummy1.init", line 261, in download_cover
File "calibre_plugins.dummy1.init", line 326, in identify
File "calibre_plugins.dummy1.init", line 63, in search_books
File "calibre_plugins.dummy1.init", line 48, in load_book_urls_new
File "urllib\request.py", line 216, in urlopen
File "urllib\request.py", line 519, in open
File "urllib\request.py", line 536, in _open
File "urllib\request.py", line 496, in _call_chain
File "urllib\request.py", line 1391, in https_open
File "urllib\request.py", line 1351, in do_open
urllib.error.URLError:

请问这个ISSUE是什么原因造成的呢?谢谢

豆瓣把tag关了，页面里找了个内容不知道能不能替代

元数据乱下书

下载元数据总是下载成另一本书

ids一栏会出现new douban的编号

今天试用了一下新插件，很不错，也很方便，以前一直使用[simple-boot-douban-api]，现在两个一起用，发现一个问题：
IDS一栏中会出现一个new douban的编号，这个编号是与douban编号重复的。找了一下设置，也没发现如何不添加这个编号的设置。
希望要么能关掉这个新增编号，要么也显示为douban:XXXXX的形式覆盖掉原来的豆瓣编号，可以吗？

右键下载元数据如何下载多项，并选择某项后进行编辑确认？

在编辑元数据界面下下载元数据可获得多个结果，但是不具备比较功能；使用右键下载元数据只提供了一个结果，这个结果可能不是最优的，可有解决方法？

ids一栏会出现new douban的编号

搜索词存在全角括号或者冒号时搜索不到内容

搜索词存在全角括号或者冒号时搜索不到，改为半角符号就能搜索结果了。

calibre, version 5.44.0
错误：无匹配项:

无法为你找到书籍元数据。请尝试扩大搜索范围。比如：只使用作者的名称或只包含标题中具有特征性的词组。

要查看完整的日志，请点击“查看详细信息”。

Running identify query with parameters:
{'title': '深入浅出数据分析', 'authors': ['（美）米尔顿著'], 'identifiers': {}, 'timeout': 30}
Using plugins: New Douban Books (1, 0, 7)
The log from individual plugins is below

****************************** New Douban Books (1, 0, 7) ******************************
Found 0 results
Downloading from New Douban Books took 1.368743896484375
Plugin New Douban Books failed
Traceback (most recent call last):
File "calibre/ebooks/metadata/sources/identify.py", line 47, in run
File "calibre_plugins.dummy1.init", line 293, in identify
books = self.book_searcher.search_books(isbn or title, log)
File "calibre_plugins.dummy1.init", line 65, in search_books
book = future.result()
File "concurrent/futures/_base.py", line 432, in result
File "concurrent/futures/_base.py", line 388, in __get_result
File "concurrent/futures/thread.py", line 57, in run
File "calibre_plugins.dummy1.init", line 79, in load_book
res = urlopen(Request(url, headers={'user-agent': random_user_agent()}))
File "urllib/request.py", line 222, in urlopen
File "urllib/request.py", line 531, in open
File "urllib/request.py", line 640, in http_response
File "urllib/request.py", line 569, in error
File "urllib/request.py", line 502, in _call_chain
File "urllib/request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 418:

The identify phase took 1.43 seconds
The longest time (1.368744) was taken by: New Douban Books
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds

异常IP无法下载元数据怎么办

也就用了3-4次，都挺成功的，在PC端5.x.x版本的Calibre，但是再用就用不了了，用浏览器打开豆瓣读书提示异常IP需要登录，登录以后再用Calibre来获取也不行，怎么办