hjyssg / shigureader Goto Github PK

View Code? Open in Web Editor NEW

386.0 4.0 45.0 55.09 MB

硬核宅宅资源管理器. Ultimate Manga Resource Manager

License: MIT License

HTML 0.16% JavaScript 87.57% Dockerfile 0.20% SCSS 12.08%

comics-reader zip react full-stack doujin ipad mobile-web manga doujin-manager video-player

shigureader's Introduction

ShiguReader

English Version

ShiguReader是一款可在电脑或iPad上使用的漫画浏览器，它还支持整理资源、播放音乐和观看视频等多种功能。只需前往Release，下载后便可立即开始使用。

Screenshots

功能特色

可在电脑和iPad上使用。
显示每个漫画压缩包的封面，方便浏览。
支持播放音乐和视频。
提供各种排序和筛选功能。包括根据喜欢排序。
可一键压缩压缩包内的图片，节约硬盘空间。
进行特定作者或同人类型的全部文件展示。
可移动、删除文件。
可制作统计图表，统计文件大小和各时期的文件数量。
接近于旧版熊猫网的配色，让你感受亲切熟悉。
同时支持Windows和lunix系统。

支持的文件格式

支持的压缩包格式取决于7Zip。支持常见的zip、rar、7z。图片、音乐和视频的支持格式取决于浏览器。图片格式常见的包括jpg、png和gif，视频格式常见的包括mp4和avi，音乐格式支持mp3和wav等多种格式。

演示视频

ShiguReader的演示视频有些过时了，不过我们会尽快更新新版本的演示视频。你可以通过以下链接找到过去的演示视频
iPad使用
 PC使用
 iPhone使用

快捷键

漫画页面
enter: 全屏
A、D、左右方向键: 翻页
W、S、上下方向键: 上下
+和-: 缩放图片
x：移动到no good文件夹
v：移到good文件夹
g：快速跳页

第三方依赖

虽然ShiguReader可不安装依赖也能使用，但强烈建议安装image magick。这样可以使用它来压缩图片，提高软件的性能。

注意事项

部分文件名带汉字或日语假名的图片无法正常加载，你可能需要进行以下语言设置。请注意，这可能会导致其他非Unicode软件出现乱码问题。

Windows 语言设置如下所示：:

压缩包内图片压缩功能

介绍视频
一些漫画图片过于庞大，比如下载了一本24页共640MB的漫画，但关键画面与一本仅30MB的漫画并没有太大区别。因此，我们添加了压缩包内图片压缩功能。首先，你需要确认是否可以通过cmd运行magick命令。然后就可以通过网页以启动压缩程序，压缩后的文件默认保存在workspace\minified_zip_cache目录里。

配合TamperMonkey使用

把EhentaiHighighliger.js添加到TamperMonkey。
在你上绅士网的时候，该脚本会与后端服务器通信。显示文件下载过与否。

FAQ

问： 点击exe后软件无法启动，怎么办？
答:  默认的3000端口可能已被占用，请尝试更改端口号。  
     比如 ShiguReader_Backend.exe --port 5000   

问：某些视频无法播放，怎么办？
答：视频只是附件功能，支持的格式有限

问：电脑可以正常使用，但是扫描二维码后无法在手机上打开，请问如何解决？
答：请首先确认电脑和手机是否在同一局域网Wifi下。如果仍然无法打开，请检查电脑防火墙设置。

问： ShiguReader是啥意思？
答： Shigure(しぐれ) + Reader。当年的舰C的同人可真好看。

开发环境设置

开发人员请阅读Readme_Env_Setup

反馈与建议

如果你仍有疑问或者需要帮助，请在Github上反馈issue。同时，我们也欢迎任何关于改善ShiguReader的建议。

DOCKER 使用方法（过时）

docker pull liwufan/shigureader
docker run -d -p hostport:3000 -v comicpath:/data liwufan/shigureader

# hostport 是主机要开放的端口
# comicpath 是要扫描的文件目录

有问题阅读 docker配置说明

NAS 使用方法（过时）

热心人总结的

shigureader's People

Contributors

Stargazers

Watchers

shigureader's Issues

node src/server/index.js 可不可以跑在自定义端口呢？

因为我在8080端口开了nginx，端口被占用的时候会报错无法运行程序。

[file-iterator] scan: 0
[0] events.js:292
[0] throw er; // Unhandled 'error' event
[0] ^
[0]
[0] Error: listen EADDRINUSE: address already in use :::8080
[0] at Server.setupListenHandle [as _listen2] (net.js:1314:16)
[0] at listenInCluster (net.js:1362:12)
[0] at Server.listen (net.js:1448:7)
[0] at Function.listen (/home/liwufan/codes/ShiguReader/node_modules/express/lib/application.js:618:24)
[0] at init (/home/liwufan/codes/ShiguReader/src/server/index.js:195:24)
[0] at Object. (/home/liwufan/codes/ShiguReader/src/server/index.js:1005:1)
[0] at Module._compile (internal/modules/cjs/loader.js:1185:30)
[0] at Object.Module._extensions..js (internal/modules/cjs/loader.js:1205:10)
[0] at Module.load (internal/modules/cjs/loader.js:1034:32)
[0] at Function.Module._load (internal/modules/cjs/loader.js:923:14)
[0] Emitted 'error' event on Server instance at:
[0] at emitErrorNT (net.js:1341:8)
[0] at processTicksAndRejections (internal/process/task_queues.js:84:21) {
[0] code: 'EADDRINUSE',
[0] errno: -98,
[0] syscall: 'listen',
[0] address: '::',
[0] port: 8080
[0] }
[0] [nodemon] app crashed - waiting for file changes before starting...

[performance] single-thread problem

express server make a lot of search and calculation when receiving request.
single-thread make it do it one by one.

need to speed up

crawler

const HCCrawler = require('headless-chrome-crawler');
const JSONLineExporter = require('headless-chrome-crawler/exporter/json-line');

const FILE_PATH = 'C:\\git\\examples\\result.csv';

const exporter = new JSONLineExporter({
   file: FILE_PATH,
   // fields: ['options', 'response.url'],
   fields: ['response.url'],
   // separator: '\n',
});

//todo 
//two crawler
//one for index page
//the other for detail page

(async () => {

  const crawler = await HCCrawler.launch({
  	maxConcurrency: 1,
    // maxDepth: 2,  //Maximum depth for the crawler to follow links automatically, default to 1. Leave default to disable following links.
    exporter,

    evaluatePage: () => {
    	return {
    		title: $('title').text()
    	}
    },
    onSuccess: result => {
      // console.log(`Screenshot is saved as ${PATH}${result.options.saveAs} for ${result.options.url}.`);
      debugger
      console.log(result);
    },
    onError: error => {
    	console.error(error)
    },
    preRequest: options => {
      //return fale to skip url
      return true;
    },
  });



  await crawler.queue({
  	url:'https://www.doujinshi.org/browse/circle/31071/', 
  	screenshot: '31071.png',
  	obeyRobotsTxt: false,
  	delay: 1500
  });


  await crawler.onIdle();
  await crawler.close();


})();


```js

when module.exports.home_pathes has non-exist path, will js error

node-sass 安装失败

https://segmentfault.com/a/1190000010984731

npm i node-sass --sass_binary_site=https://npm.taobao.org/mirrors/node-sass/

// 也可以设置系统环境变量的方式。示例
// linux、mac 下
SASS_BINARY_SITE=https://npm.taobao.org/mirrors/node-sass/ npm install node-sass

// window 下
set SASS_BINARY_SITE=https://npm.taobao.org/mirrors/node-sass/ && npm install node-sass

服务端nodemon 起不来

之前的 156f29d 可以运行，升级到 fffc87c 报错

0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/node', '/usr/bin/npm', 'run', 'server' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'preserver', 'server', 'postserver' ]
5 info lifecycle [email protected]preserver: [email protected]
6 info lifecycle [email protected]server: [email protected]
7 verbose lifecycle [email protected]server: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]server: PATH: /usr/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/liwufan/codes/ShiguReader/node_modules/.bin:/usr/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/liwufan/codes/ShiguReader/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
9 verbose lifecycle [email protected]server: CWD: /home/liwufan/codes/ShiguReader
10 silly lifecycle [email protected]server: Args: [ '-c', 'nodemon src/server/index.js' ]
11 silly lifecycle [email protected]server: Returned: code: 1 signal: null
12 info lifecycle [email protected]server: Failed to exec server script
13 verbose stack Error: [email protected] server: nodemon src/server/index.js
13 verbose stack Exit status 1
13 verbose stack at EventEmitter. (/usr/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack at EventEmitter.emit (events.js:315:20)
13 verbose stack at ChildProcess. (/usr/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack at ChildProcess.emit (events.js:315:20)
13 verbose stack at maybeClose (internal/child_process.js:1051:16)
13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
14 verbose pkgid [email protected]
15 verbose cwd /home/liwufan/codes/ShiguReader
16 verbose Linux 5.5.10-arch1-1
17 verbose argv "/usr/bin/node" "/usr/bin/npm" "run" "server"
18 verbose node v14.0.0
19 verbose npm v6.14.4
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] server: nodemon src/server/index.js
22 error Exit status 1
23 error Failed at the [email protected] server script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

内网穿透

允许外网用户也可以访问。安全性，性能都需要考虑。尤其是后端安全性。不要被人删除任意文件。
是否需要可以跑在NAS上。
限定仅部分文件展示。我可不想被请去喝茶。

chokidar内存使用

用了chokidar。watch 50,000个文件需要1GB内存

关掉chokidar就只要100mb了。

pure javascript zip library

https://www.npmjs.com/package/jszip

https://www.npmjs.com/package/extract-zip

https://www.npmjs.com/package/yauzl

use lokijs for even fast lsdir and search

const file_db = new loki();
const file_collection = loki_db.addCollection("fileInfo", { indices: ['filePath'] });


//each entry will be 
filePath, author, group, tag, fileName, fileStat

function getData(filePath){
    file_collection.findOne({filePath: filePath});
}

const has = module.exports.has = function(filePath){
    const data = getData(filePath);
    return !!data;
}



const deleteFromFileDb = function(filePath){
    if(has(filePath)){
        let data = getData(filePath);
        file_collection.remove(data);
    }
}

const updateFileDb = function(filePath, stat){
    //maybe: name parser and name pick to save tags into db
    //or let search to record this info
    if(has(filePath)){
        let data = getData(filePath);
        data.stat = stat;
        file_collection.update(data);
    }else{
        file_collection.insert({
            filePath,
            stat
        });
    }
}

module.exports.initFileToInfo = function(obj){
    db.fileToInfo = obj;
    loopEachFileInfo(e => {
        const fileName = path.basename(e);
        if (isDisplayableInExplorer(fileName)) {
            serverUtil.parse(fileName);
        }

        file_collection.updateFileDb(e)
    })
}


module.exports.updateStatToDb =  function(path, stat){
    const result = {};
    result.isFile = stat.isFile();
    result.isDir = stat.isDirectory();
    result.mtimeMs = stat.mtimeMs;
    result.mtime = stat.mtime;
    result.size = stat.size;
    db.fileToInfo[path] = result;
    file_collection.updateFileDb(path, result);
}

module.exports.deleteFromDb = function(path){
    delete db.fileToInfo[path];
    deleteFromFileDb(path);
}


function escapeRegExp(string) {
    return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
}

str = escapeRegExp(str)
var reg = new RegExp(str, 'i');
var results = file_collection.find({'Name': { '$regex' : reg }});

// .where(queryFunc).simplesort('name').data();

[Todo]replace old video player

https://filmora.wondershare.com/video-player/12-best-html5-video-players.html

https://www.sitepoint.com/5-great-html5-video-players/

release 1.5.0 draft

[Bug fixing] fix a bug that causes server not able to list any zip file content.
[UI] Easier to tell which book is zip, which one is folder.
[New Feature] allow user to delete/zip folder
[code quality] client uses await/async for post. F**K you, callback function. Server always sends back json.
[New Feature] better ehentai user script
[UI] able to display why error message from server
[UI] busy indicator for img loading
[Experiment New Feature] utilize Everything search HTTP server to boost performance.

Not for this release

[New Feature] use ehentai metadata to display for UI

utilize everything http api

https://www.voidtools.com/forum/viewtopic.php?f=2&t=8703&p=29464&hilit=http+path#p29464

http://localhost:80/?search="C:\Users\jh\Downloads\1111"&offset=0&json=1&path_column=1&size_column=1&date_modified_column=1

[Todo] detect good or not by machine learning

https://github.com/karpathy/svmjs

先尝试用js实现看看效果。
不行就用python，然后js和python之间用http通信。

Is there default comic folder?

I am using Synology Docker and not very familar with customizing the run command.
It would be great if the default comic path could be set, so I can map it direct by mounting the NAS volume folder.

文件名包含某些字符的图片显示不出来

大概是大于 7MB 的, 一般的漫画没有这么大的, 但是有的扫图组会搞很大的封面

我搭在自己的家用服务器上, 用 nginx 反代内网服务, dev 模式下, 把 / 转到 localhost:3000, 把 /api 转到 localhost:4000/api (我把 src 里面的 8080 全改成 4000 了), 可以正常工作. 但是在 Production 模式下, 貌似只有一个端口 4000, 把所有请求转到 localhost:4000, 会出现阅读页 404.
Nginx 的日志:

 [03/May/2020:10:00:11 +0800] "GET /onebook/2691019969 HTTP/2.0" 404 152 "https://*****:444/explorer/1381105937" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"

访问 localhost:4000/onebook/2691019969 也是 404

请教哪里有问题?

[potential bug] path seperator

linux uses /
windows uses \

the client code is becoming confusing.
should receive a universal solution.

这一行有时会报错

machineLearning.js#L238

bayes 可能会等于 undefined

(node:2198) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'predict' of undefined

然后 lsdir 调用就无下文了

[code review] relative path for util.js is annoying

https://stackoverflow.com/questions/10860244/how-to-make-node-js-require-absolute-instead-of-relative

todo

[todo]create cache folder when first time init

[todo] UI改进

explorer点可以toggle一次显示更多的thumbnail
onebook连续点击可以更快跳页

[Todo] windows console不设置成unicode，listZipContent()会乱码

看看可不可以decoding

[Bug] 文件夹名字有. 会被当成文件

比如文件夹

xxx NO.003

ShiguReader is only for LAN

First, the server is only safe at home LAN environment.
The express server provides API that users can list/download/move/delete files.
Currently, server don't do any security check, just do what the request asks.
However, if exposing ShiguReader to public web, malicious hacker can easily destroy your computer.
e.g. They can list all you zip files and delete them all.

If any dev who want to improve security, they need to refactor server code a lot.
e.g. Creating a user login system, only authorized user can do certain file operation.
That will be a huge workload to implement. My estimation is that, 8 hours per day, at least one month for coding and testing.

Second, if you run ShiguReader on your home pc and want to access it when going outside. You have to make its IP publicly accessible. Google how to do and you will find out it is troublesome and costs money.
If you run ShiguReader on cloud server, you don't have to worry about IP.
But you still must pay a monthly fee for cloud server.

Third, the server sends uncompressed image to the client. Sometimes, each image can be >20MB.
20MB image is totally okay in LAN. But it will be too slow to load in public web.

In conclusion, accessing Shigureader from public web is a bad idea. It requires a huge workload to implement security. And It still costs money. DropBox or Google Drive will be cheaper and more reliable. Setting up a WordPress website is also a good choice.

目前支持的启动方式

开发者模式：npm run dev
production mode： npm run start

pkg的snapshot搞的太坑
还有谜之bootstrap错误
再也不想研究exe打包了

avoid extracting the same file mulitple times when the user keeps sending the same request

const  extractQueue = {  };

//after 7zip
extractQueue[fileName] = true;


finally {
	extractQueue[fileName] = false;
}

[todo] allow user to raname file and folder

also want to delete folder

页面全都显示Bad request 400

[0]
[0] > [email protected] server E:\ShiguReader
[0] > nodemon src/server/index.js
[0]
[1]
[1] > [email protected] client E:\ShiguReader
[1] > webpack-dev-server --mode development --devtool inline-source-map --hot
[1]
[0] [nodemon] 1.18.9
[0] [nodemon] to restart at any time, enter rs
[0] [nodemon] watching: E:\ShiguReader\src\server/**/* src/util.js src/port-config.js src/user-config.js src/path-config
[0] [nodemon] starting node src/server/index.js
[0] --------------------
[0] process.cwd() E:\ShiguReader
[0] __filename E:\ShiguReader\src\server\index.js
[0] __dirname E:\ShiguReader\src\server
[0] rootPath E:\ShiguReader
[0] log path: E:\ShiguReader\workspace\log\2020-05-15 21-02.log
[0] sevenZipPath E:\ShiguReader\resource\7zip
[0] ----------------------
[0] [chcp] ���ҳ: 936
[0] Please switch you console encoding to utf8 in windows language setting
[0] scanning local files
[0] [file-iterator] D:_Happy_Lesson_Going_to_sort_good\good_2020_05_01 does not exist! Please check you path-config and user-config.js
[0] [file-iterator] D:_Happy_Lesson_Going_to_sort_good\good_2020_05_01 不存在! 检查一下你的path-config和user-config.js
[0] [file-iterator] D:_Happy_Lesson_Going_to_sort_good does not exist! Please check you path-config and user-config.js
[0] [file-iterator] D:_Happy_Lesson_Going_to_sort_good 不存在! 检查一下你的path-config和user-config.js
[0] [file-iterator] D:_Happy_Lesson_Going_to_sort_Compressed_2020 does not exist! Please check you path-config and user-config.js
[0] [file-iterator] D:_Happy_Lesson_Going_to_sort_Compressed_2020 不存在! 检查一下你的path-config和user-config.js
[0] [file-iterator] scan: 0
[0] 0.012s to read local dirs
[0] Analyzing local files
[0] There are 25 files
[0] ----------scan cache------------
[0] [file-iterator] scan: 0
[0] ----------------------------------------------------------------
[0] 2020-05-15 21:02
[0] Express Server listening on port 8080
[0] You can open ShiguReader from Browser now!
[0] http://localhost:3000
[0] http://10.10.10.3:3000
[0] Scan the QR code to open on mobile devices

[1] clean-webpack-plugin: E:\ShiguReader\dist has been removed.
[1] i ｢wds｣: Project is running at http://0.0.0.0:3000/
[1] i ｢wds｣: webpack output is served from /
[1] i ｢wds｣: 404s will fallback to /index.html
[1] Browserslist: caniuse-lite is outdated. Please run next command npm update caniuse-lite browserslist
[1] i ｢wdm｣: Hash: 8013aaedb73142abd6bb
[1] Version: webpack 4.29.0
[1] Time: 17137ms
[1] Built at: 2020-05-15 21:02:38
[1] Asset Size Chunks Chunk Names
[1] bundle.js 10.2 MiB main [emitted] main
[1] favicon-96x96.png 15.4 KiB [emitted]
[1] index.html 560 bytes [emitted]
[1] Entrypoint main = bundle.js
[1] [0] multi (webpack)-dev-server/client?http://0.0.0.0:3000 (webpack)/hot/dev-server.js babel-polyfill ./src/client/index.js 64 bytes {main} [built]
[1] [./node_modules/babel-polyfill/lib/index.js] 833 bytes {main} [built]
[1] [./node_modules/core-js/fn/regexp/escape.js] 108 bytes {main} [built]
[1] [./node_modules/core-js/shim.js] 8.03 KiB {main} [built]
[1] [./node_modules/loglevel/lib/loglevel.js] 7.68 KiB {main} [built]
[1] [./node_modules/react-dom/index.js] 1.33 KiB {main} [built]
[1] [./node_modules/react-router-dom/es/index.js] 1010 bytes {main} [built]
[1] [./node_modules/react/index.js] 190 bytes {main} [built]
[1] [./node_modules/regenerator-runtime/runtime.js] 23.9 KiB {main} [built]
[1] [./node_modules/url/url.js] 22.8 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/index.js?http://0.0.0.0:3000] (webpack)-dev-server/client?http://0.0.0.0:3000 7.78 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/overlay.js] (webpack)-dev-server/client/overlay.js 3.58 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/socket.js] (webpack)-dev-server/client/socket.js 1.05 KiB {main} [built]
[1] [./node_modules/webpack/hot/dev-server.js] (webpack)/hot/dev-server.js 1.61 KiB {main} [built]
[1] [./src/client/index.js] 295 bytes {main} [built]
[1] + 910 hidden modules
[1] Child html-webpack-plugin for "index.html":
[1] 1 asset
[1] Entrypoint undefined = index.html
[1] [./node_modules/html-webpack-plugin/lib/loader.js!./public/index.html] 661 bytes {0} [built]
[1] [./node_modules/lodash/lodash.js] 528 KiB {0} [built]
[1] [./node_modules/webpack/buildin/global.js] (webpack)/buildin/global.js 472 bytes {0} [built]
[1] [./node_modules/webpack/buildin/module.js] (webpack)/buildin/module.js 497 bytes {0} [built]
[1] i ｢wdm｣: Compiled successfully.

centos设置

根据b站的fdhgsdrfg，在centos7.1下path-config要写绝对路径

比如
/mnt/abc

然后需要安装 p7zip-plugins才能使用7z命令

'npm run dev' command failed

Hi hjyssg,
Thanks for your hard work. I followed your instructions and installed node.js and git bash on my Win10 laptop. I was coding in visual studio code. All went well until the final command. It seems something got wrong when executing 'npm run dev'. You can check more details in below. It'll be greatly appreciated if you can give me some help.

$npm run dev

[email protected] dev D:\ShiguReader\ShiguReader
concurrently "npm run server" "npm run client"

[1]
[1] ┌────────────────────────────────────────────────────────────┐
[1] │ npm update check failed │
[1] │ Try running with sudo or get access │
[1] │ to the local update config store via │
[1] │ sudo chown -R $USER:$(id -gn $USER) C:\Users\syuny.config │
[1] └────────────────────────────────────────────────────────────┘
[0]
[0] > [email protected] server D:\ShiguReader\ShiguReader
[0] > nodemon src/server/index.js
[0]
[1]
[1] > [email protected] client D:\ShiguReader\ShiguReader
[1] > webpack-dev-server --mode development --devtool inline-source-map --hot
[1]
[0] [nodemon] 1.18.9
[0] [nodemon] to restart at any time, enter rs
[0] [nodemon] watching: D:\ShiguReader\ShiguReader\src\server//* D:\ShiguReader\ShiguReader\src\common//* D:\ShiguReader\ShiguReader\src\config/**/* src/path-config
[0] [nodemon] starting node src/server/index.js
[0] D:\ShiguReader\ShiguReader\node_modules\internal-ip\index.js:26
[0] } catch {}
[0] ^
[0]
[0] SyntaxError: Unexpected token {
[0] at createScript (vm.js:80:10)
[0] at Object.runInThisContext (vm.js:139:10)
[0] at Module._compile (module.js:617:28)
[0] at Object.Module._extensions..js (module.js:664:10)
[0] at Module.load (module.js:566:32)
[0] at tryModuleLoad (module.js:506:12)
[0] at Function.Module._load (module.js:498:3)
[0] at Module.require (module.js:597:17)
[0] at require (internal/module.js:11:18)
[0] at Object. (D:\ShiguReader\ShiguReader\src\server\index.js:8:20)
[0] [nodemon] app crashed - waiting for file changes before starting...
[1] clean-webpack-plugin: D:\ShiguReader\ShiguReader\dist has been removed.
[1] i ｢wds｣: Project is running at http://0.0.0.0:3000/
[1] i ｢wds｣: webpack output is served from /
[1] i ｢wds｣: Content not from webpack is served from D:\ShiguReader\ShiguReader
[1] i ｢wds｣: 404s will fallback to /index.html
[1] Browserslist: caniuse-lite is outdated. Please run next command npm update caniuse-lite browserslist
[1] i ｢wdm｣: Hash: 1939f3c57830ddb4f096
[1] Version: webpack 4.43.0
[1] Time: 7286ms
[1] Built at: 2020-07-31 00:17:53
[1] Asset Size Chunks Chunk Names
[1] bundle.js 10.8 MiB main [emitted] main
[1] favicon-96x96.png 15.4 KiB [emitted]
[1] index.html 576 bytes [emitted]
[1] Entrypoint main = bundle.js
[1] [0] multi (webpack)-dev-server/client?http://0.0.0.0:3000 (webpack)/hot/dev-server.js babel-polyfill ./src/client/index.js 64 bytes {main} [built]
[1] [./node_modules/babel-polyfill/lib/index.js] 833 bytes {main} [built]
[1] [./node_modules/core-js/fn/regexp/escape.js] 108 bytes {main} [built]
[1] [./node_modules/core-js/shim.js] 8.03 KiB {main} [built]
[1] [./node_modules/react-dom/index.js] 1.33 KiB {main} [built]
[1] [./node_modules/react-router-dom/es/index.js] 1010 bytes {main} [built]
[1] [./node_modules/react/index.js] 190 bytes {main} [built]
[1] [./node_modules/regenerator-runtime/runtime.js] 23.9 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/index.js?http://0.0.0.0:3000] (webpack)-dev-server/client?http://0.0.0.0:3000 4.29 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/overlay.js] (webpack)-dev-server/client/overlay.js 3.51 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/socket.js] (webpack)-dev-server/client/socket.js 1.53 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/utils/createSocketUrl.js] (webpack)-dev-server/client/utils/createSocketUrl.js 2.91 KiB {main} [built]
[1] [./node_modules/webpack-dev-server/client/utils/log.js] (webpack)-dev-server/client/utils/log.js 964 bytes {main} [built]
[1] [./node_modules/webpack/hot/dev-server.js] (webpack)/hot/dev-server.js 1.59 KiB {main} [built]
[1] [./src/client/index.js] 295 bytes {main} [built]
[1] + 858 hidden modules
[1] Child html-webpack-plugin for "index.html":
[1] 1 asset
[1] Entrypoint undefined = index.html
[1] [./node_modules/html-webpack-plugin/lib/loader.js!./public/index.html] 693 bytes {0} [built]
[1] [./node_modules/lodash/lodash.js] 528 KiB {0} [built]
[1] [./node_modules/webpack/buildin/global.js] (webpack)/buildin/global.js 472 bytes {0} [built]
[1] [./node_modules/webpack/buildin/module.js] (webpack)/buildin/module.js 497 bytes {0} [built]
[1] i ｢wdm｣: Compiled successfully.

About thumbnail display of folders

Long time no see!
I will write about the points that I found inconvenient after using shigureader for a while.

When I search for a cartoon folder from "home" in the current shigureader, the folders are listed as strings.
But in many cases, it's more convenient to see folders as thumbnails rather than strings.
My suggestion is to display thumbnails instead of strings if there are files such as .zip under the folder.

utilize hentai metadata

更新以后 production 模式访问 /onebook/xxxx 报 404

用 npm run dev 启动, 可正常使用
用 npm start 启动, 访问 /onebook/xxxx 报 404, Can't get /onebook/xxxx, 第一次点进 /explorer/xxxx 正常, 刷新也是报 404.

test with Node.js 14

the current version is 12

need to update

async function mkdir(path, quiet){
    try {
        if(!(await isExist(path))){
            const err = await pfs.mkdir(path, { recursive: true});
            if(err){
                throw err;
            }
        }
    }catch(err){
        console.log(err)
        if(!quiet){
            throw err;
        }
    }
}

log-update

https://github.com/sindresorhus/log-update

在visual studio code不work。故此以后不再研究优化server init的console.log了

[todo]启动太慢了

file-iterator太慢了，一个一个scan。看看能不能用上everything

[windows cmd] cannot not start 'npm run dev'

npm ERR! code ELIFECYCLE
npm ERR! syscall spawn bash
npm ERR! file bash
npm ERR! path bash
npm ERR! errno ENOENT

[Todo] allow user to rename file

文件名大小写问题

在 wsl 里面启动报错

Module not found: Error: Can't resolve './style/App.scss' in '/**********/ShiguReader/src/client

实际的文件名是 app.scss, windows 不区分大小写所以没发现吧

todo

video player
ehentai script

imageMagickHelp.js improve

function isConertable() =>单独做成一个 post
在client先问可不可以转换，然后再转换。两个请求。

module.exports.minifyOneFile

不在通过 oldAvgImgSize判断

async function convertImage(imgFilePath, outputImgPath, oldAvgImgSize){
    try{
        let opt;

         const stat = await pfs.stat(p);
         const oldImgSize =    stat.size;
        if(oldImgSize > img_reduce_resolution_threshold){

            opt = [imgFilePath, "-strip", "-quality", img_convert_quality, "-resize", `${img_reduce_resolution_dimension}\>`, outputImgPath ];
        }else{
            opt = [imgFilePath, "-strip", "-quality", img_convert_quality, outputImgPath ];
        }

        let {stdout, stderr} = await execa("magick", opt);
        return {stdout, stderr};
    }catch(e){
        logFail("[convertImage]", e);
    }
}

test template

// https://github.com/mochajs/mocha
// npm i mocha chai puppeteer

//  "scripts": {
//     "test": "mocha test.js"
// },

const puppeteer = require('puppeteer');
https://stackoverflow.com/questions/25678063/whats-the-difference-between-assertion-library-testing-framework-and-testing-e#:~:text=Assertion%20libraries%20are%20tools%20to,do%20thousands%20of%20if%20statements.
const { expect }  = require('chai');


describe('Duck Duck Go search using basic Puppeteer', function(){

    let browser;
    let page;

    //https://github.com/mochajs/mocha/issues/2586 
    //不用arrow function
    // Passing arrow functions (aka “lambdas”) to Mocha is discouraged. Lambdas lexically bind this and cannot access the Mocha context. 
    this.timeout(30*1000);

    before(async function() {
        // runs once before the first test in this block
        browser = await puppeteer.launch({
            headless: false
        });
    });


    after(async function() {
        // runs once after the last test in this block
        await browser.close();
    });

    beforeEach(async function() {
        page = await browser.newPage();
        await page.goto('https://duckduckgo.com');
    });

    afterEach(async function() {
       await page.close();
    });

    it('should have the correct page title', async function() {
        const title = await page.title();
        expect(title).to.eql('DuckDuckGo — Privacy, simplified.');
    });

    // it('should show a list of results when searching actual word', async () => {
    //     await page.type('input[id=search_form_input_homepage]', 'puppeteer');
    //     await page.click('input[type="submit"]');
    //     await page.waitForSelector('h2 a');
    //     const links = await page.evaluate(() => {
    //         return Array.from(document.querySelectorAll('h2 a'));
    //     });
    //     expect(links.length).to.be.greaterThan(0);
    // });

    // it('should show a warning when searching fake word', async () => {
    //     await page.type('input[id=search_form_input_homepage]', 'pupuppeppeteerteer');
    //     await page.click('input[type="submit"]');
    //     await page.waitForSelector('div[class=msg__wrap]');
    //     const text = await page.evaluate(() => {
    //         return document.querySelector('div[class=msg__wrap]').textContent;
    //     });
    //     expect(text).to.contain('Not many results contain');
    // });

});

[dev notes] run with only bundle.js

SET NODE_TLS_REJECT_UNAUTHORIZED=0
pkg -t x86 ./src/server/index.js

//不work，没有bable的样子
app.use('/explorer/', express.static('dist'));
app.use('/explorer/*', express.static('dist'));
app.use('/explorer', express.static('dist'));
app.use('explorer', express.static('dist'));

https://stackoverflow.com/questions/34105183/uncaught-syntaxerror-unexpected-token-in-nodejs
//不work，没有babel的样子
app.get('/*', (req, res) => {
    const as = path.resolve(__dirname, "..", "..", 'dist', 'index.html');
    console.log(as)
    res.sendFile(as);
})



/* final catch-all route to index.html defined last */
app.get('*', (req, res) => {
    const as = path.resolve(__dirname, "..", "..", 'dist', 'index.html');
    console.log(as)
    res.sendFile(as);
})


//js 要按js传
//html要按html传


var history = require('connect-history-api-fallback');
app.use(history({
    verbose: true
}));

app.get('/index.html', (req, res) => {
    const as = path.resolve(__dirname, "..", "..", 'dist', 'index.html');
    console.log(as)
    res.sendFile(as);
})

Rewriting GET /explorer/2085920753 to /index.html
C:\Users\jhuang342\Downloads\ShiguReader-dev\dist\index.html
Not rewriting GET /fontawesome-free-5.7.2-web/css/all.min.css because the path includes a dot (.) character.
Not rewriting GET /explorer/bundle.js because the path includes a dot (.) character.

pagination

class AjiPagination extends Component {
    constructor (props) {
      super()
      this.state = {
        textValue: props.currentPage
      }
    }
  
    componentWillReceiveProps (nextProps) {
      if (this.props.currentPage !== nextProps.currentPage) {
        this.setState({ textValue: nextProps.currentPage })
      }
    }
  
    onChange = (e) => {
      const {onChange} = this.props;
      const value = typeof e === "number"? e : parseInt(e.target.textContent.trim());
      onChange && onChange(value);
    }
  
  
    prev = () => {
      const {onChange, currentPage} = this.props;
      onChange && onChange(Math.max(currentPage - 1, 1));
    }
  
    next = () => {
      const {onChange, currentPage} = this.props;
      onChange && onChange(Math.min(currentPage + 1, this.getTotalPage()));
    }
  
    getTotalPage(){
      const {
        itemPerPage,
        totalItemNum,
      } = this.props
      return Math.ceil(totalItemNum/itemPerPage);
    }
  
    getSafePage (page) {
      if (Number.isNaN(page)) {
        page = this.props.currentPage
      }
      return Math.min(Math.max(page, 1), this.getTotalPage());
    }
  
    render(){
      const {
        currentPage,
        itemPerPage,
        totalItemNum,
        onChange,
        className,
      } = this.props
  
      const {textValue} = this.state;
  
      const BUFFER_SIZE = 3; //the items will 1 + BUFFER_SIZE*2
      const totalPage = this.getTotalPage();
  
      if(totalPage <= 1){
        return null;
      }
  
      const prevButton = (<div className="pagination-item prev" onClick={this.prev}> prev </div>)
      const nextButton = (<div className="pagination-item next" onClick={this.next}> next </div>)
  
      let right = Math.max(currentPage - BUFFER_SIZE, 1);
      let left = Math.min(currentPage + BUFFER_SIZE, totalPage);
  
      if(currentPage - right < BUFFER_SIZE){
        const toLeft = BUFFER_SIZE - (currentPage - right);
        left = Math.min(currentPage + BUFFER_SIZE + toLeft, totalPage);
      }
  
      if(left - currentPage < BUFFER_SIZE){
        const toRight = BUFFER_SIZE - (left - currentPage);
        right = Math.max(currentPage - BUFFER_SIZE - toRight, 1);
      }
  
      const one = (<li className="pagination-item" key={1} onClick={this.onChange}> 1 </li>);
      const ellipsis1 =  (<li className="pagination-item ellipsis" key="ellipsis-1"> ... </li>);
      const ellipsis2 =  (<li className="pagination-item ellipsis" key="ellipsis-2"> ... </li>);
      const end = (<li className="pagination-item" key={totalPage} onClick={this.onChange} > {totalPage} </li>);
  
  
      let contentList = [];
      contentList.push(prevButton);
      if(right > 1){
        contentList.push(one);
        if(right > 2){
          contentList.push(ellipsis1);
        }
      }
  
      for(let ii = right; ii <= left; ii++){
        contentList.push(<li className="pagination-item" key={ii} onClick={this.onChange}> {ii} </li>);
      }
  
      if(left < totalPage){
        if(left < totalPage - 1){
          contentList.push(ellipsis2);
        }
        contentList.push(end);
      }
  
      contentList.push(nextButton);
  
      const pageInput = (
        <div className="-pageJump">
          <input
            type='text'
            value={textValue||currentPage}
            onKeyPress={e => {
              if (e.which === 13 || e.keyCode === 13) {
                //enter key
                this.onChange(parseInt(textValue));
              }
            }}
  
            onChange={e => {
              const val = e.target.value
              this.setState({ textValue: val })
            }}
          />
          <div>{`/${totalPage}`}</div>
        </div>)
  
      return (<ul className="pagination">
                {contentList}
                {pageInput}
              </ul>);
    }
  }

调整图片大小后一翻页就恢复默认了，如何保存设置？

Succeeded in running on synology NAS

Below are the steps I took.

Nodejs is installed on the synology NAS.

Install Java on your NAS.
However, my NAS did not need to be installed because the CPU is arm v7.

If you don’t already have a ‘shared folder’ for your Comics, create one.

Enter administrator mode with "sudo -i".

sudo -i

Install entware-ng. It's a package manager and you can easily install imagemagick.
https://github.com/Entware/Entware-ng

Install imagemagick with entware-ng.

opkg install imagemagick

Install git.

opkg install git
opkg install git-http

git clone https://github.com/hjyssg/ShiguReader

modify src/path-config.
replace "D:" to "/volume1"
for example "/volume1/Comic" if shared folder is "Comic".

modify src/config/user-config.js.
replace "D:" to "/volume1"
replace "\" to "/"
for example "/volume1/Comic/_Happy_Lesson/_Going_to_sort/_good/"

7z is included in Synology NAS as standard, but there was a problem, so install it separately.

opkg install p7zip

cd ShiguReader

npm install
If some packages fail to install with a warning, force them to install with the "--force" command.

run.

npm run dev

It's works.

snip

const pathUtil = require("../pathUtil");
const {
        isExist
} = pathUtil;
const pfs = require('promise-fs');
const fs = require('fs');
const execa = require('execa');
const userConfig = global.requireUserConfig();
const isWindows = require('is-windows');
const express = require('express');
const router = express.Router();
const logger = require("../logger");
const path = require('path');

const util = global.requireUtil();
const { isImage, isCompress, isMusic, arraySlice, isDisplayableInOnebook } = util;

const sevenZipHelp = require("../sevenZipHelp");

function getReason(e){
    return e.stderr || e.message || e;
}


router.post('/api/renameFile', (req, res) => {
    const src = req.body && req.body.src;
    const dest = req.body && req.body.dest;

    if(!src || !dest){
        res.sendStatus(404);
        return;
    }

    (async () =>{
        try{
            let err = await pfs.rename(src, dest);

            if(err){ throw err; }

            logger.info(`[rename] ${src} to ${dest}`);
            res.sendStatus(200);
        }catch(err){
            console.error(err);
            res.status(500).send(getReason(err));
        }
    })();
});

router.post('/api/moveFile', (req, res) => {
    const src = req.body && req.body.src;
    const dest = req.body && req.body.dest;

    if(!src || !dest){
        res.sendStatus(404);
        return;
    }

    (async () =>{
        try{
            let err;
            if(!(await isExist(dest))){
                err = await pfs.mkdir(dest, {recursive: true});
            }

            if(err){ throw "fail to create dest folder";}

            const cmdStr = isWindows()? "move" : "mv";
            const {stdout, stderr} = await execa(cmdStr, [src, dest]);
            err = stderr;

            if(err){ throw err;}
         
            logger.info(`[MOVE] ${src} to ${dest}`);
            res.sendStatus(200);
        }catch(err){
            console.error(err);
            res.status(500).send(getReason(err));
        }
    })();
});

function deleteThing(src){
   if(userConfig.move_file_to_recyle){
        const trash = require('trash');
        await trash([src]);
        if(!(await isExist(src))){
            res.sendStatus(200);
            logger.info(`[DELETE] ${src}`);
        } else {
            //missing is delete
            res.sendStatus(200);
        }
    }else{
        const err = await pfs.unlink(src) 
        if (err){ throw err; }
    }
}

async function isSimpleFolder(src){
    let content_pathes = await pfs.readdir(src);
    const otherTypes = content_pathes.filter(e => !isDisplayableInOnebook(e));

    return otherTypes.length === 0;
}

const _folder_waring_ = "This folder is not a one-level img/music folder";
const file_occupy_warning = "File may be used by another process"

router.post('/api/deleteFile', async (req, res) => {
    const src = req.body && req.body.src;

    if(!src || !(await isExist(src))){
        res.sendStatus(404);
        return;
    }

    try{
        deleteThing(src);
        res.sendStatus(200);
        logger.info(`[DELETE] ${src}`);
    } catch(e) {
        console.error(e);
        res.status(500).send(file_occupy_warning);
    }
});


router.post('/api/deleteFolder', async (req, res) => {
    const src = req.body && req.body.src;

    if(!src || !(await isExist(src))){
        res.sendStatus(404);
        return;
    }

    if(!(await isSimpleFolder(src))){
        res.status(500).send(_folder_waring_);
        return;
    }

    //below is duplicate code as /api/deleteFile
    //need to improve
    try{
        deleteThing(src);
        res.sendStatus(200);
        logger.info(`[DELETE] ${src}`);
    } catch(e) {
        console.error(e);
        res.status(500).send(file_occupy_warning);
    }
});

router.post('/api/zipFolder', async (req, res) => {
    const src = req.body && req.body.src;

    if(!src || !(await isExist(src))){
        res.sendStatus(404);
        return;
    }

    if(! (await isSimpleFolder(src))){
        res.status(500).send(_folder_waring_);
        return;
    }


    try{
        let {stdout, stderr, resultZipPath} = await sevenZipHelp.zipOneFolder(src);
        if(stderr){
            throw stderr;
        }
        res.sendStatus(200);
        logger.info(`[zipFolder] ${src}`);
    } catch(e) {
        console.error(e);
        res.status(500).send("fail to zip");
    }
});

module.exports = router;