Giter VIP home page Giter VIP logo

dipu-bd / lightnovel-crawler Goto Github PK

View Code? Open in Web Editor NEW
1.3K 40.0 244.0 32.2 MB

Generate and download e-books from online sources.

Home Page: https://pypi.org/project/lightnovel-crawler/

License: GNU General Public License v3.0

Python 99.16% Shell 0.29% CSS 0.25% Batchfile 0.08% HTML 0.04% Dockerfile 0.09% Procfile 0.01% JavaScript 0.09%
lightnovel termux web-scraper console-app python lightnovel-crawler discord telegram kindle-books

lightnovel-crawler's Introduction

Lightnovel Crawler

download win download linux Discord
PyPI version Python version Downloads License Build and Publish

An app to download novels from online sources and generate e-books.

Discord: https://discord.gg/wMECG2Q

Telegram: https://t.me/epub_smelter_bot

Table of contents

Installation

This application uses Calibre to convert ebooks.
Install it from https://calibre-ebook.com/download
Without it, you will only get output in epub, text, and web formats.

For macOS, you need to manually add the path to Calibre.
Before starting lncrawl, use the command:

$ export PATH="$PATH:/Applications/calibre.app/Contents/MacOS"

If you used a folder other than Applications during installation, replace /Applications/ with your path to Calibre.

Standalone Bundle (Windows, Linux)

Windows: lncrawl.exe ~ 25MB

In Windows 8, 10 or later versions, it might say that lncrawl.exe is not safe to dowload or execute. You should bypass/ignore this security check to execute this program.

Linux: lncrawl ~ 30MB

It is recommended to install via pip if you are on Linux

To get older versions visit the Releases page

PIP (Windows, Mac, and Linux)

📦 A python package named lightnovel-crawler is available at pypi.

Make sure you have installed Python v3.8 or higher and have pip enabled. Visit these links to install python with pip in Windows, Linux and MacOS. Feel free to ask on the Discord server if you are stuck.

To install this app or to update installed one via pip, just run:

$ pip install -U lightnovel-crawler

In some cases you have to use python3 -m pip or pip3 or python -m pip. And you do not need --user option, if you are running from root.

Next, open your terminal and enter:

$ lncrawl

To view extra logs, use: lncrawl -lll

If you want to get the cutting-edge (sometimes unstable) from the dev branch, you can get it by:

PIP (Directly from GitHub)

The master branch contains the latest stable code. If you can not wait for it to be released in the PyPi, you can get it like this:

$ pip install -U git+https://github.com/dipu-bd/lightnovel-crawler.git#egg=lightnovel-crawler

The dev branch contains cutting-edge, sometimes unstable changes. To install it:

$ pip install -U https://github.com/dipu-bd/lightnovel-crawler/tarball/refs/heads/dev#egg=lightnovel-crawler

Docker

Docker is a convenient way to run it anywhere.

  • First clone the project.
$ git clone https://github.com/dipu-bd/lightnovel-crawler
  • Build docker:
$ cd lightnovel-crawler
$ docker build -t lncrawl -f ./scripts/Dockerfile .
  • Run commands using docker:
$ mkdir ~/Lightnovels
$ docker run -v ~/Lightnovels:/home/appuser/app/Lightnovels -it lncrawl

You can setup alias to the above command in your terminal's profile to run using single a single-word command.

Termux (Android)

Please read before proceeding:

  • It is not guaranteed that the app will run smoothly in all devices.
  • It may take a long time to install depending on your mobile processor.
  • It is recommended to use the bots on either Discord or Telegram if you are on mobile.

📱 Using Termux, you can run this app in your android phones too. Follow this instructions:

  • Install Termux from playstore.
  • Open the app and run these commands one by one:
    • pkg upgrade
    • pkg install python libxml2 libxslt libjpeg-turbo rust python-lxml python-grpcio
    • pip install -U pip wheel setuptools
    • pip install lightnovel-crawler
    • termux-setup-storage
    • cd ~/storage/downloads
    • lncrawl
  • You can navigate up using Vol UP + W and down using Vol UP + S.

When there is a new update available, you can install it just by running pip install -U lightnovel-crawler. You will not have to run all the above commands again.

PyDroid

You can also use PyDroid in Android phones. Check this discussion for a custom script to run the app: #1137

Chatbots

Discord

Join our server: https://discord.gg/7A5Hktx

Or, visit this link to install discord bot to your own server: https://discordapp.com/oauth2/authorize?client_id=537526751170002946&permissions=51264&scope=bot

Telegram

Visit this link to get started with the telegram bot: https://t.me/epub_smelter_bot

Send !help to open the bot help message.

Heroku Deployment

Simply fill out the environment variables and you get a running instance.

Deploy

Running from source

  • First clone the repository:
$ git clone https://github.com/dipu-bd/lightnovel-crawler
  • Open command prompt inside of the project folder and install requirements:
$ pip install -r requirements.txt
  • Run the program (use python v3.8 or higher):
$ python lncrawl

Running the Bots

There are two chatbots available at this moment: Telegram and Discord. To run your own bot server, follow these instructions:

  • Clone this repository
$ git clone https://github.com/dipu-bd/lightnovel-crawler
$ pip3 install -r requirements.txt
  • Copy .env.example file to .env file. Edit this file and give your API credentials here.

  • To run the discord bot:

$ python3 lncrawl --bot discord --shard-id 0 --shard-count 1
  • To run the telegram bot
$ python3 lncrawl --bot telegram

There is a start.sh script to run a bot in ubuntu servers. It will basically execute the python3 lncrawl and send the task to run in background. I use it to run my discord bot in the server.

General Usage

Available options

$ lncrawl -h
================================================================================
╭╮╱╱╱╱╱╱╭╮╱╭╮╱╱╱╱╱╱╱╱╱╱╱╱╭╮╱╭━━━╮╱╱╱╱╱╱╱╱╱╭╮
┃┃╱╱╱╱╱╱┃┃╭╯╰╮╱╱╱╱╱╱╱╱╱╱╱┃┃╱┃╭━╮┃╱╱╱╱╱╱╱╱╱┃┃
┃┃╱╱╭┳━━┫╰┻╮╭╋━╮╭━━┳╮╭┳━━┫┃╱┃┃╱╰╋━┳━━┳╮╭╮╭┫┃╭━━┳━╮
┃┃╱╭╋┫╭╮┃╭╮┃┃┃╭╮┫╭╮┃╰╯┃┃━┫┃╱┃┃╱╭┫╭┫╭╮┃╰╯╰╯┃┃┃┃━┫╭╯
┃╰━╯┃┃╰╯┃┃┃┃╰┫┃┃┃╰╯┣╮╭┫┃━┫╰╮┃╰━╯┃┃┃╭╮┣╮╭╮╭┫╰┫┃━┫┃
╰━━━┻┻━╮┣╯╰┻━┻╯╰┻━━╯╰╯╰━━┻━╯╰━━━┻╯╰╯╰╯╰╯╰╯╰━┻━━┻╯
╱╱╱╱╱╭━╯┃ v3.6.0
╱╱╱╱╱╰━━╯ 🔗 https://github.com/dipu-bd/lightnovel-crawler
--------------------------------------------------------------------------------
usage: lncrawl [options...]
       lightnovel-crawler [options...]

options:
  -h, --help            show this help message and exit

  -v, --version         show program's version number and exit
  -l                    Set log levels. (-l = warn, -ll = info, -lll = debug).
  --log-file [FILE]     To store application logs to a file.
  --list-sources        Display a list of available sources.
  --crawler [FILES ...]
                        Load additional crawler files.
  -s URL, --source URL  Profile page url of the novel.
  -q STR, --query STR   Novel query followed by list of source sites.
  -x [REGEX], --sources [REGEX]
                        Filter out the sources to search for novels.
  --login USER PASSWD   User name/email address and password for login.
  --format E [E ...]    Define which formats to output. Default: all.
  --add-source-url      Add source url at the end of each chapter.
  --single              Put everything in a single book.
  --multi               Build separate books by volumes.
  -o PATH, --output PATH
                        Path where the downloads to be stored.
  --filename NAME       Set the output file name
  --filename-only       Skip appending chapter range with file name
  -f, --force           Force replace any existing folder.
  -i, --ignore          Ignore any existing folder (do not replace).
  --all                 Download all chapters.
  --first [COUNT]       Download first few chapters (default: 10).
  --last [COUNT]        Download last few chapters (default: 10).
  --page START [STOP. ...]
                        The start and final chapter urls.
  --range FROM TO., --index FROM TO., --chapter FROM TO.
                        The start and final chapter indexes.
  --volumes [N ...]     The list of volume numbers to download.
  --chapters [URL ...]  A list of specific chapter urls.
  --proxy-file FILE     Proxies as SCHEME://HOST:PORT@USER:PASSWORD format in
                        each line. All except HOST are optional
  --auto-proxy          Use some free proxies from https://free-proxy-
                        list.net/
  --bot {console,telegram,discord,lookup}
                        Select a bot. Default: console.
  --shard-id [SHARD_ID]
                        Discord bot shard id (default: 0)
  --shard-count [SHARD_COUNT]
                        Discord bot shard counts (default: 1)
  --selenium-grid URL   Selenium Grid URL for Chrome Webdriver
  --suppress            Suppress all input prompts and use defaults.
  --ignore-images       Ignore images in chapters when downloading.
  --close-directly      Do not prompt to close at the end for windows
                        platforms.
  --resume [NAME/URL]   Resume download of a novel containing in
                        /home/runner/work/lightnovel-crawler/lightnovel-
                        crawler/Lightnovels
  ENV                   [chatbots only] Pass query string at the end of all
                        options. It will be use instead of .env file. Sample:
                        "BOT=discord&DISCORD_TOKEN=***&LOG_LEVEL=DEBUG"

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~

--------------------------------------------------------------------------------
 🔗  https://github.com/dipu-bd/lightnovel-crawler/issues 
================================================================================

Example Usage

Open your console and type lncrawl --version first to check if you have installed it properly. Here are some example usage of the app:

  • To start an interactive session: lncrawl

  • To download using an url: lncrawl -s https://boxnovel.com/novel/reincarnation-of-the-strongest-sword-god/

  • To search novels: lncrawl -q "Strongest Sword God"

  • To search novels from selected sources: lncrawl -q "Strongest Sword God" --sources

  • To download all chapters: lncrawl --all

  • To download first 25 chapters: lncrawl --first 25

  • To download all between two chapters: lncrawl --range 10 30

  • To download all between two chapter links: lncrawl -s https://novelfull.com/release-that-witch.html --chapters https://novelfull.com/release-that-witch/chapter-6-training-part-i.html https://novelfull.com/release-that-witch/chapter-8-months-of-the-demons-part-1.html

  • To download a specific volumes: lncrawl --volumes 2 3

  • To define output path: lncrawl -o "D:\Lightnovels\reincarnation-of-the-strongest-sword-god"

  • To delete the output folder if exists: lncrawl -f

  • To ignore the output folder if exists: lncrawl -i

  • To resume download where is has been left previously: lncrawl -i

  • To specify output formats: lncrawl --format epub pdf mobi

  • To display list of supported sources: lncrawl --list-sources

  • If you provide an option in the argument, it will skip it in the interactive session. If you want to disable all interactive prompts, pass --suppress at the end.

  • You can stack up options like this: lncrawl -s https://boxnovel.com/novel/reincarnation-of-the-strongest-sword-god/ -o "D:\Lightnovels\reincarnation-of-the-strongest-sword-god" --last 50 -i --format pdf --suppress

Additional Help

Visit the discussions page for more information. You can also post your query there too.

Follow this guide to know how to login: #1360

Development

You are very welcome to contribute in this project. You can:

  • create new issues pointing out the bugs.
  • solve existing issues.
  • add your own sources.
  • add new output formats.
  • create new bots.

Adding new source

  • Use lncrawl --bot lookup first to auto-generate your crawler from an existing template.
  • Check inside the sources/_examples. Read all the comments of all the files. And pick the one you like.
  • You can find plenty examples in the sources folder. Try to check the latest ones
  • Put your source file inside the language folder. The en folder has too many files, therefore it is grouped using the first letter of the domain name.
  • Before making commit format files using blake formatter, and use scripts/lint.sh or scripts/lint.bat to check linting issues.

Adding new Bot

Supported sources

Request new one by creating a new issue.

We are supporting 394 sources and 346 crawlers.

~ Unknown

Source URL Version Contributors
🤖🔍 http://es.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🤖🔍 http://fr.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🤖🔍 http://id.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🤖🔍 http://www.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🔍 https://18.foxaholic.com/ 80 dipu-bd SirGryphin yudilee watzeedzad mesmerlord
🤖🔍 https://es.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🔍 https://foxaholic.com/ 80 dipu-bd SirGryphin yudilee watzeedzad mesmerlord
🤖🔍 https://fr.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🔍 https://global.foxaholic.com/ 80 dipu-bd SirGryphin yudilee watzeedzad mesmerlord
🤖🔍 https://id.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🔑 https://my.w.tt/ 70 dipu-bd SirGryphin yudilee
🤖🔍 https://ncode.syosetu.com/ 17 dipu-bd zGadli
🤖🔍 https://wtr-lab.com/ 5 zGadli
🔍 https://www.foxaholic.com/ 80 dipu-bd SirGryphin yudilee watzeedzad mesmerlord
🤖🔍 https://www.mtlnovel.com/ 25 dipu-bd yudilee CryZFix kuwoyuki Galunid
🔍 https://www.novelupdates.com/ 1 dipu-bd
https://www.quotev.com/ 4 idMysteries
🔑 https://www.wattpad.com/ 70 dipu-bd SirGryphin yudilee
🔍 https://www.webfic.com/ 4

ar Arabic

Source URL Version Contributors
🔍 https://arnovel.me/ 23 dipu-bd SirGryphin
🔍 https://kolnovel.com/ 1
https://rewayat.club/ 8 dipu-bd

en English

Source URL Version Contributors
🔍 http://lightnovels.live/ 8 dipu-bd SirGryphin Seven0492
🔍 http://lightnovels.me/ 8 dipu-bd SirGryphin Seven0492
🔍 http://novelfull.com/ 50 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki Galunid
🤖🔍 http://novelhall.com/ 63 dipu-bd yudilee jere344 Seven0492
🔍 http://ranobes.net/ 23 dipu-bd zerty
🔍 http://ranobes.top/ 23 dipu-bd zerty
🔍 http://readlightnovel.online/ 17 dipu-bd SirGryphin jere344 mesmerlord Camfurbush
🔍 http://readonlinenovels.com/ 67 dipu-bd SirGryphin yudilee idMysteries jere344 amritoo
🤖 http://wnmtl.org/ 21 dipu-bd SirGryphin
http://www.fujitranslation.com/ 62 dipu-bd SirGryphin yudilee
🤖🔍 http://www.novelhall.com/ 63 dipu-bd yudilee jere344 Seven0492
🤖 http://www.wnmtl.org/ 21 dipu-bd SirGryphin
http://zenithnovels.com/ 17 dipu-bd
🤖🔍 https://1stkissnovel.love/ 82 dipu-bd SirGryphin yudilee idMysteries jere344 zGadli kuwoyuki
🤖🔍 https://1stkissnovel.org/ 82 dipu-bd SirGryphin yudilee idMysteries jere344 zGadli kuwoyuki
https://88tangeatdrinkread.wordpress.com/ 72 dipu-bd SirGryphin yudilee
🔍 https://allnovel.org/ 48 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki Galunid
🔍 https://allnovelfull.com/ 7 dipu-bd SirGryphin idMysteries
🔍 https://allnovelfull.net/ 7 dipu-bd SirGryphin idMysteries
🔍 https://allnovelxo.com/ 48 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki Galunid
https://americanfaux.com/ 2 dipu-bd SirGryphin
https://amnesiactl.com/ 14 dipu-bd SirGryphin
https://ancientheartloss.com/ 74 dipu-bd SirGryphin yudilee idMysteries
https://ancientheartloss.wordpress.com/ 74 dipu-bd SirGryphin yudilee idMysteries
🔍🔑 https://api.babelnovel.com/ 30 dipu-bd yudilee idMysteries kuwoyuki Galunid
🔍🖼️ https://aquamanga.com/ 75 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki
🔍🖼️ https://aquamanga.org/ 75 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki
🔍 https://asadatranslations.com/ 68 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🤖 https://automtl.wordpress.com/ 68 dipu-bd SirGryphin yudilee idMysteries
🔍🔑 https://babelnovel.com/ 30 dipu-bd yudilee idMysteries kuwoyuki Galunid
https://bakapervert.wordpress.com/ 73 dipu-bd SirGryphin yudilee
🔍🖼️ https://bato.to/ 9 dipu-bd idMysteries CryZFix
🔍🖼️ https://batocc.com/ 9 dipu-bd idMysteries CryZFix
🔍🖼️ https://batotoo.com/ 9 dipu-bd idMysteries CryZFix
🔍🖼️ https://batotwo.com/ 9 dipu-bd idMysteries CryZFix
🔍🖼️ https://battwo.com/ 9 dipu-bd idMysteries CryZFix
🔍🖼️ https://beautymanga.com/ 6 dipu-bd jere344
🔍 https://bednovel.com/ 29 dipu-bd SirGryphin idMysteries CryZFix
🔍 https://bestlightnovel.com/ 25 dipu-bd SirGryphin yudilee jere344 kuwoyuki
🔍 https://bonnovel.com/ 81 dipu-bd SirGryphin yudilee idMysteries jere344 kuwoyuki
🔍 https://booknet.com/ 6 dipu-bd
🔍 https://boxnovel.com/ 68 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://bronovel.com/ 68 dipu-bd SirGryphin yudilee kuwoyuki
🖼️ https://chapmanganato.com/ 61 dipu-bd SirGryphin yudilee
🔑 https://chrysanthemumgarden.com/ 18 dipu-bd jere344 zGadli
🔍🖼️ https://coffeemanga.io/ 17 dipu-bd SirGryphin CryZFix
🔍🖼️ https://comiko.net/ 9 dipu-bd idMysteries CryZFix
🤖🔍 https://comrademao.com/ 14 dipu-bd SirGryphin
https://creativenovels.com/ 32 dipu-bd yudilee kuwoyuki
https://crescentmoon.blog/ 58 dipu-bd yudilee
🔍 https://daonovel.com/ 16 dipu-bd SirGryphin jere344
🤖🔍 https://daotranslate.com/ 20 dipu-bd SirGryphin jere344 zGadli CryZFix
🤖🔍 https://daotranslate.us/ 20 dipu-bd SirGryphin jere344 zGadli CryZFix
https://demontranslations.com/ 67 dipu-bd SirGryphin yudilee idMysteries
https://dmtranslationscn.com/ 62 dipu-bd SirGryphin yudilee
🤖 https://dobelyuwai.wordpress.com/ 75 dipu-bd SirGryphin yudilee
🔍 https://dragontea.ink/ 17 dipu-bd SirGryphin mesmerlord
🔍🖼️ https://dto.to/ 9 dipu-bd idMysteries CryZFix
🔍 https://dummynovels.com/ 4 dipu-bd idMysteries
🤖🔍 https://earlynovel.net/ 6 jere344 hoangphuc05
🔍 https://engnovel.com/ 1 jere344
https://exiledrebelsscanlations.com/ 68 dipu-bd SirGryphin yudilee jere344
🔍 https://f-w-o.com/ 68 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://fanstranslations.com/ 14 dipu-bd SirGryphin idMysteries
🤖🔍 https://faqwiki.us/ 8
🔍 https://freefullnovel.com/ 78 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍🖼️ https://freemanga.me/ 75 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍 https://freewebnovel.com/ 29 dipu-bd SirGryphin idMysteries CryZFix
https://fujitranslation.com/ 62 dipu-bd SirGryphin yudilee
🔍🖼️ https://harimanga.com/ 75 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍 https://home.novel-gate.com/ 22 dipu-bd SirGryphin jere344 zGadli
https://hostednovel.com/ 6 dipu-bd idMysteries zGadli
🔍 https://hotnovelfull.com/ 77 dipu-bd SirGryphin yudilee idMysteries kuwoyuki Galunid
🔍🖼️ https://hto.to/ 9 dipu-bd idMysteries CryZFix
https://hui3r.wordpress.com/ 63 dipu-bd SirGryphin yudilee
🔍🖼️ https://imperfectcomic.org/ 5 dipu-bd idMysteries
https://inadequatetranslations.wordpress.com/ 71 dipu-bd SirGryphin yudilee
https://infinitenoveltranslations.net/ 67 dipu-bd SirGryphin yudilee zGadli
🔍 https://innnovel.com/ 29 dipu-bd SirGryphin idMysteries CryZFix
🔍 https://innread.com/ 29 dipu-bd SirGryphin idMysteries CryZFix
🔍🖼️ https://isekaiscan.com/ 76 dipu-bd SirGryphin yudilee idMysteries jere344 kuwoyuki
🔍🖼️ https://isekaiscan.eu/ 76 dipu-bd SirGryphin yudilee idMysteries jere344 kuwoyuki
https://isotls.com/ 63 dipu-bd SirGryphin yudilee idMysteries
🤖 https://jpmtl.com/ 63 dipu-bd yudilee
https://justatranslatortranslations.com/ 66 dipu-bd SirGryphin yudilee
🔍🖼️ https://king-manga.com/ 74 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍🖼️ https://kissmanga.in/ 76 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍 https://latestnovel.net/ 76 dipu-bd SirGryphin yudilee watzeedzad
https://lazybirdtranslations.wordpress.com/ 66 dipu-bd SirGryphin yudilee
https://lazygirltranslations.com/ 12 dipu-bd idMysteries
https://lemontreetranslations.wordpress.com/ 70 dipu-bd SirGryphin yudilee
🔍 https://librarynovel.com/ 7 dipu-bd SirGryphin
🔍 https://libread.com/ 29 dipu-bd SirGryphin idMysteries CryZFix
🔍 https://libread.org/ 29 dipu-bd SirGryphin idMysteries CryZFix
https://lightnovel.world/ 62 dipu-bd SirGryphin yudilee
🔍 https://lightnovelbastion.com/ 10 dipu-bd idMysteries
🔍 https://lightnovelheaven.com/ 66 dipu-bd SirGryphin yudilee jere344 kuwoyuki
🔍 https://lightnovelreader.me/ 17 dipu-bd SirGryphin jere344 mesmerlord Camfurbush
🔍 https://lightnovels.live/ 8 dipu-bd SirGryphin Seven0492
🔍 https://lightnovels.me/ 8 dipu-bd SirGryphin Seven0492
🔍 https://lightnovelsonl.com/ 20 dipu-bd SirGryphin yudilee kuwoyuki
https://lightnovelstranslations.com/ 12 dipu-bd SirGryphin idMysteries zerty
🔍 https://listnovel.com/ 11 dipu-bd SirGryphin
🤖🔑 https://lnmtl.com/ 97 dipu-bd yudilee kuwoyuki
🔍 https://lnreader.org/ 17 dipu-bd SirGryphin jere344 mesmerlord Camfurbush
🔍 https://luminarynovels.com/ 3 zGadli
https://lunarletters.com/ 15 dipu-bd SirGryphin
🔍 https://m.readlightnovel.cc/ 13 dipu-bd SirGryphin yudilee
🔍 https://m.webnovel.com/ 89 dipu-bd yudilee idMysteries
🔍 https://m.wuxiaworld.co/ 69 dipu-bd SirGryphin yudilee Galunid
🔍🖼️ https://manga-tx.com/ 76 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍🖼️ https://mangabuddy.com/ 7 dipu-bd idMysteries
🔍🖼️ https://mangachill.io/ 12 dipu-bd SirGryphin
🔍🖼️ https://mangachill.love/ 12 dipu-bd SirGryphin
🔍🖼️ https://mangarockteam.com/ 74 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍🖼️ https://mangarosie.love/ 77 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki
🔍🖼️ https://mangarosie.me/ 77 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki
🔍🖼️ https://mangastic.net/ 16 dipu-bd SirGryphin watzeedzad
https://mangatoon.mobi/ 8 dipu-bd yudilee
🔍🖼️ https://mangatoto.com/ 9 dipu-bd idMysteries CryZFix
🔍🖼️ https://mangatoto.net/ 9 dipu-bd idMysteries CryZFix
🔍🖼️ https://mangatoto.org/ 9 dipu-bd idMysteries CryZFix
🖼️ https://mangatx.com/ 76 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍🖼️ https://mangaweebs.in/ 75 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍🖼️ https://manhuaplus.online/ 17 dipu-bd SirGryphin
🔍🖼️ https://manhwachill.com/ 11 dipu-bd SirGryphin
🔍 https://meownovel.com/ 16 dipu-bd SirGryphin idMysteries
https://miraslation.net/ 63 dipu-bd SirGryphin yudilee idMysteries
🤖🔍 https://mixednovel.net/ 6 jere344 hoangphuc05
🤖🔍 https://mltnovels.com/ 4 dipu-bd idMysteries
🔍 https://mostnovel.com/ 8 dipu-bd SirGryphin
🤖🔍🔑 https://mtlnation.com/ 13 dipu-bd idMysteries CryZFix
🤖🔍 https://mtlreader.com/ 4 dipu-bd
🔍🖼️ https://mto.to/ 9 dipu-bd idMysteries CryZFix
🔍 https://myboxnovel.com/ 69 dipu-bd SirGryphin yudilee kuwoyuki
🔍🖼️ https://myreadingmanga.fit/ 6 dipu-bd jere344
🔍 https://mysticalmerries.com/ 68 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://newnovel.org/ 67 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://noblemtl.com/ 10 dipu-bd idMysteries
🔍🖼️ https://noobchan.xyz/ 78 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍 https://novel-bin.com/ 2 dipu-bd SirGryphin
🔍 https://novel-bin.net/ 1 SirGryphin
🔍 https://novel27.com/ 68 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://novel35.com/ 1 idMysteries
🔍 https://novelbin.com/ 76 dipu-bd SirGryphin yudilee idMysteries zerty kuwoyuki Galunid
🔍 https://novelbin.net/ 1 dipu-bd
🔍 https://novelcake.com/ 15 dipu-bd SirGryphin idMysteries
🔍 https://novelfull.com/ 50 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki Galunid
🔍 https://novelfull.me/ 2 idMysteries jere344
🔍 https://novelfull.net/ 50 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki Galunid
🔍 https://novelfullplus.com/ 11 dipu-bd idMysteries
🔍 https://novelgate.net/ 22 dipu-bd SirGryphin jere344 zGadli
🤖🔍 https://novelhall.com/ 63 dipu-bd yudilee jere344 Seven0492
🔍 https://novelhard.com/ 66 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://novelhi.com/ 6 dipu-bd idMysteries zimorok
🔍 https://novelhulk.com/ 74 dipu-bd SirGryphin yudilee idMysteries kuwoyuki Galunid
🤖 https://novelmao.com/ 6 dipu-bd
🖼️ https://novelmic.com/ 20 dipu-bd SirGryphin
🔍 https://novelnext.com/ 8 dipu-bd SirGryphin idMysteries
🔍 https://novelnextz.com/ 9 dipu-bd SirGryphin idMysteries
🔍 https://novelonlinefree.com/ 23 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://novelonlinefull.com/ 19 dipu-bd yudilee kuwoyuki
🔍 https://novels.pl/ 6 dipu-bd jere344
🔍 https://novelsala.com/ 11 dipu-bd idMysteries
🔍 https://novelsemperor.com/ 12 dipu-bd idMysteries jere344 zGadli
🔍 https://novelsite.net/ 14 dipu-bd SirGryphin
🔍 https://novelsonline.net/ 74 dipu-bd SirGryphin yudilee zerty
🤖🔍 https://noveltranslate.com/ 12 dipu-bd SirGryphin jere344
🤖 https://novelww.com/ 8 dipu-bd idMysteries jere344
https://novelzec.com/ 2 dipu-bd SirGryphin
https://nyx-translation.com/ 2
https://nyxtranslation.home.blog/ 2
🔍 https://pandanovel.org/ 2 dipu-bd SirGryphin
🔍🖼️ https://pianmanga.com/ 76 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
https://puretl.com/ 9 dipu-bd idMysteries
🔍 https://randomnovel.com/ 78 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍 https://ranobes.net/ 23 dipu-bd zerty
🔍 https://ranobes.top/ 23 dipu-bd zerty
https://re-library.com/ 7 dipu-bd zerty CryZFix
https://readlightnovel.me/ 76 dipu-bd yudilee jere344
https://readlightnovel.today/ 76 dipu-bd yudilee jere344
🔍 https://readlightnovels.net/ 32 dipu-bd yudilee
🔍 https://readlitenovel.com/ 17 dipu-bd SirGryphin jere344 mesmerlord Camfurbush
🖼️ https://readmanganato.com/ 61 dipu-bd SirGryphin yudilee
🤖🔍 https://readmtl.com/ 1 dipu-bd
🔍 https://readnovelfull.com/ 73 dipu-bd SirGryphin yudilee idMysteries kuwoyuki Galunid
🔍 https://readonlinenovels.com/ 67 dipu-bd SirGryphin yudilee idMysteries jere344 amritoo
🔍 https://readwebnovels.net/ 68 dipu-bd SirGryphin yudilee kuwoyuki
https://reaperscans.com/ 21 dipu-bd SirGryphin idMysteries kuwoyuki
https://reincarnationpalace.com/ 64 dipu-bd SirGryphin yudilee
https://rpgnoob.wordpress.com/ 72 dipu-bd SirGryphin yudilee
https://rpgnovels.com/ 72 dipu-bd SirGryphin yudilee
🔍 https://scribblehub.com/ 39 dipu-bd yudilee idMysteries zerty zGadli
https://secondlifetranslations.com/ 5 idMysteries
https://shalvationtranslations.wordpress.com/ 68 dipu-bd SirGryphin yudilee
https://shanghaifantasy.com/ 2 CryZFix
https://skydemonorder.com/ 6 dipu-bd
🔍 https://skynovel.org/ 14 dipu-bd SirGryphin
https://sleepytranslations.com/ 14 dipu-bd SirGryphin
https://smnovels.com/ 60 dipu-bd SirGryphin yudilee
https://snowycodex.com/ 3 dipu-bd idMysteries
https://steambunlightnovel.com/ 14 dipu-bd SirGryphin
🔍 https://supernovel.net/ 65 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://systemtranslation.com/ 11 dipu-bd idMysteries
🤖🔍 https://tamagotl.com/ 6 dipu-bd idMysteries
🔍 https://tigertranslations.org/ 4
🔍 https://tipnovel.com/ 67 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://toc.qidianunderground.org/ 11 dipu-bd idMysteries
🔍 https://tocqidianunderground.blogspot.com/ 11 dipu-bd idMysteries
https://tomotranslations.com/ 11 dipu-bd
🔍🖼️ https://toon69.com/ 77 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki
🖼️ https://toonily.com/ 78 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
https://totallytranslations.com/ 6 dipu-bd
https://travistranslations.com/ 4 dipu-bd SirGryphin
🔍 https://usefulnovel.com/ 80 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
https://veratales.com/ 71 dipu-bd SirGryphin yudilee zerty
🔍 https://vipnovel.com/ 67 dipu-bd SirGryphin yudilee kuwoyuki
https://vistranslations.wordpress.com/ 72 dipu-bd SirGryphin yudilee
https://wanderinginn.com/ 65 dipu-bd SirGryphin yudilee CryZFix
https://webnovelonline.com/ 12 dipu-bd kuwoyuki
🔍 https://webnovelonline.net/ 69 dipu-bd SirGryphin yudilee jere344 kuwoyuki
https://whatsawhizzerwebnovels.com/ 1 idMysteries
🔍 https://whitemoonlightnovels.com/ 2 jere344 zGadli
🤖 https://wnmtl.org/ 21 dipu-bd SirGryphin
🔍 https://wondernovels.com/ 13 dipu-bd SirGryphin
🔍 https://woopread.com/ 9 dipu-bd SirGryphin
🔍 https://wordexcerpt.com/ 13 dipu-bd SirGryphin yudilee
https://wordrain69.com/ 4 dipu-bd SirGryphin
🔍🖼️ https://wto.to/ 9 dipu-bd idMysteries CryZFix
https://wujizun.com/ 75 dipu-bd SirGryphin yudilee
🤖🔍 https://wuxia.city/ 5 dipu-bd
🔍 https://wuxiaworld.name/ 25 dipu-bd SirGryphin yudilee kuwoyuki
https://wuxiaworld.online/ 29 dipu-bd SirGryphin yudilee
https://wuxiaworldsite.co/ 6 dipu-bd
🔍 https://www.allnovel.org/ 48 dipu-bd SirGryphin yudilee idMysteries zGadli kuwoyuki Galunid
https://www.asianhobbyist.com/ 15 dipu-bd SirGryphin
https://www.blackbox-tl.com/ 4 dipu-bd SirGryphin
https://www.box-novel.com/ 16 dipu-bd SirGryphin
https://www.chickengege.org/ 4 idMysteries
https://www.divinedaolibrary.com/ 3 dipu-bd
🔍 https://www.f-w-o.com/ 68 dipu-bd SirGryphin yudilee kuwoyuki
🔍 https://www.fanfiction.net/ 15 dipu-bd
🔍 https://www.fictionpress.com/ 16 dipu-bd watzeedzad
https://www.flying-lines.com/ 11 dipu-bd
https://www.foxteller.com/ 7 dipu-bd
https://www.freelightnovel.com/ 11 dipu-bd SirGryphin
https://www.fringecapybara.com/ 66 dipu-bd SirGryphin yudilee
https://www.fuyuneko.org/ 65 dipu-bd SirGryphin yudilee
🔍 https://www.inkitt.com/ 2 idMysteries
https://www.isotls.com/ 63 dipu-bd SirGryphin yudilee idMysteries
https://www.kitenovel.com/ 3 dipu-bd SirGryphin
🤖 https://www.koreanmtl.online/ 12 dipu-bd
🔍 https://www.lightnovelmeta.com/ 4 dipu-bd idMysteries zerty
🔍 https://www.lightnovelpub.com/ 26 dipu-bd SirGryphin idMysteries jere344 zerty Galunid
🔍 https://www.lightnovelreader.me/ 17 dipu-bd SirGryphin jere344 mesmerlord Camfurbush
🔍 https://www.lightnovelworld.com/ 1 dipu-bd
🔍 https://www.lnreader.org/ 17 dipu-bd SirGryphin jere344 mesmerlord Camfurbush
🔍 https://www.ltnovel.com/ 7 dipu-bd SirGryphin zerty watzeedzad
🤖🔍 https://www.machine-translation.org/ 14 dipu-bd yudilee kuwoyuki
🔍🖼️ https://www.mangaread.org/ 76 dipu-bd SirGryphin yudilee idMysteries jere344 kuwoyuki
🔍🖼️ https://www.mangaweebs.in/ 75 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
https://www.miraslation.net/ 63 dipu-bd SirGryphin yudilee idMysteries
🤖🔍🔑 https://www.mtlnation.com/ 13 dipu-bd idMysteries CryZFix
🤖🔍 https://www.mtlreader.com/ 4 dipu-bd
🔍 https://www.neosekaitranslations.com/ 75 dipu-bd SirGryphin yudilee watzeedzad
https://www.newsnovel.net/ 30 dipu-bd SirGryphin yudilee
🔍 https://www.novelall.com/ 61 dipu-bd yudilee
🖼️ https://www.novelcool.com/ 33 dipu-bd SirGryphin yudilee mesmerlord
🤖🔍 https://www.novelhall.com/ 63 dipu-bd yudilee jere344 Seven0492
🔍 https://www.novelhunters.com/ 69 dipu-bd SirGryphin yudilee idMysteries
🤖🔍 https://www.novelmt.com/ 7 dipu-bd zerty watzeedzad
🔍 https://www.novelmtl.com/ 7 dipu-bd zerty watzeedzad
🔍 https://www.novelmultiverse.com/ 17 dipu-bd SirGryphin
🔍 https://www.novelpassion.com/ 8 dipu-bd SirGryphin
🔍 https://www.novelpub.com/ 18 dipu-bd SirGryphin idMysteries Galunid
🔍 https://www.novels.pl/ 6 dipu-bd jere344
🔍 https://www.novelupdates.cc/ 14 dipu-bd SirGryphin yudilee
https://www.oppatranslations.com/ 65 dipu-bd SirGryphin yudilee
https://www.ornovel.com/ 62 dipu-bd SirGryphin yudilee
🔍 https://www.pandamanga.xyz/ 9 dipu-bd idMysteries
🔍 https://www.readlightnovel.cc/ 13 dipu-bd SirGryphin yudilee
https://www.readlightnovel.me/ 76 dipu-bd yudilee jere344
https://www.readlightnovel.today/ 76 dipu-bd yudilee jere344
🤖🔍 https://www.readwn.com/ 11 dipu-bd SirGryphin jere344 zerty mesmerlord
🔍 https://www.royalroad.com/ 75 dipu-bd yudilee kuwoyuki needKVAS
🔍 https://www.scribblehub.com/ 39 dipu-bd yudilee idMysteries zerty zGadli
https://www.tapread.com/ 57 dipu-bd yudilee
https://www.teanovel.com/ 2 idMysteries
https://www.teanovel.net/ 2 idMysteries
🔍🖼️ https://www.topmanhua.com/ 13 dipu-bd SirGryphin
https://www.virlyce.com/ 66 dipu-bd SirGryphin yudilee
🔍 https://www.volarenovels.com/ 63 dipu-bd yudilee kuwoyuki
🔍 https://www.webnovel.com/ 89 dipu-bd yudilee idMysteries
🔍 https://www.webnovelpub.com/ 1 dipu-bd
🔍🖼️ https://www.webtoons.com/ 3 zGadli
🤖 https://www.wnmtl.org/ 21 dipu-bd SirGryphin
https://www.wuxia.blog/ 4 idMysteries
🔍 https://www.wuxiahub.com/ 3 dipu-bd zerty
https://www.wuxialeague.com/ 8 dipu-bd
🤖🔍 https://www.wuxiamtl.com/ 3 dipu-bd zerty
🔍 https://www.wuxianovelhub.com/ 3 dipu-bd idMysteries zerty
🤖🔍 https://www.wuxiap.com/ 11 dipu-bd SirGryphin jere344 zerty mesmerlord
🔍 https://www.wuxiapub.com/ 3 dipu-bd zerty
🔍 https://www.wuxiar.com/ 2 dipu-bd zerty
🔍 https://www.wuxiau.com/ 3 dipu-bd zerty
🔍 https://www.wuxiav.com/ 2 dipu-bd zerty
🔍🔑 https://www.wuxiaworld.com/ 105 dipu-bd yudilee idMysteries kuwoyuki alzamer2
🔍 https://www.wuxiax.com/ 2 dipu-bd zerty
🔍 https://www.wuxiaz.com/ 3 dipu-bd zerty
https://www.xiainovel.com/ 57 dipu-bd SirGryphin yudilee
🔍 https://zetrotranslation.com/ 81 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍🖼️ https://zinmanga.com/ 77 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
🔍 https://zinnovel.com/ 12 dipu-bd SirGryphin

es Spanish; Castilian

Source URL Version Contributors
https://cclawtranslations.home.blog/ 71 dipu-bd SirGryphin yudilee
https://domentranslations.wordpress.com/ 71 dipu-bd SirGryphin yudilee

fr French

Source URL Version Contributors
🔍🖼️ https://anime-sama.fr/ 4 jere344
🔍 https://chireads.com/ 10 dipu-bd jere344
🔍 https://lightnovelfr.com/ 9 dipu-bd idMysteries
🤖🔍 https://lnmtlfr.com/ 9 dipu-bd jere344
https://noveldeglace.com/ 2
https://xiaowaz.fr/ 3 idMysteries

id Indonesian

Source URL Version Contributors
http://zhi-end.blogspot.co.id/ 63 dipu-bd SirGryphin yudilee
http://zhi-end.blogspot.com/ 63 dipu-bd SirGryphin yudilee
https://darktranslation.com/ 68 dipu-bd SirGryphin yudilee
https://grensia.blogspot.com/ 5 dipu-bd idMysteries
https://indowebnovel.id/ 61 dipu-bd yudilee
https://meionovel.id/ 62 dipu-bd SirGryphin yudilee
🔍 https://morenovel.net/ 9 dipu-bd SirGryphin
https://novelgo.id/ 14 dipu-bd yudilee
https://novelku.id/ 62 dipu-bd yudilee
https://novelringan.com/ 59 dipu-bd SirGryphin yudilee
🔍 https://noveltoon.mobi/ 6 dipu-bd
🔍 https://wbnovel.com/ 62 dipu-bd SirGryphin yudilee
https://webnovelindonesia.com/ 7 dipu-bd
https://www.idqidian.us/ 46 dipu-bd yudilee
🔍 https://www.webnovelover.com/ 67 dipu-bd SirGryphin yudilee
https://www.worldnovel.online/ 81 dipu-bd yudilee
https://yukinovel.id/ 56 dipu-bd yudilee
https://yukinovel.me/ 56 dipu-bd yudilee

pt Portuguese

Source URL Version Contributors
🔍 https://blnovels.net/ 6 dipu-bd SirGryphin
🔍 https://centralnovel.com/ 9 dipu-bd idMysteries jere344
https://www.ceunovel.com/ 1 idMysteries

ru Russian

Source URL Version Contributors
🔍🖼️ https://bestmanga.club/ 75 dipu-bd SirGryphin yudilee idMysteries kuwoyuki
https://bookhamster.ru/ 3 CryZFix
https://ifreedom.su/ 3 CryZFix
https://jaomix.ru/ 9 dipu-bd idMysteries CryZFix watzeedzad
🔍 https://litnet.com/ 6 dipu-bd
https://renovels.org/ 2 CryZFix
🔑 https://tl.rulate.ru/ 7 CryZFix needKVAS

vi Vietnamese

Source URL Version Contributors
🤖🔍 https://docln.net/ 11 dipu-bd jere344
🤖🔍 https://ln.hako.vn/ 11 dipu-bd jere344
🤖🔍 https://truyenfull.vn/ 5 dipu-bd
🤖🔍 https://truyentr.info/ 5 dipu-bd

zh Chinese

Source URL Version Contributors
🔍 https://ixdzs8.com/ 6 dipu-bd junqili259
🔍 https://ixdzs8.tw/ 6 dipu-bd junqili259
🔍 https://sj.uukanshu.net/ 8 dipu-bd idMysteries
https://trxs.cc/ 1 CryZFix
🔍 https://tw.m.ixdzs.com/ 6 dipu-bd junqili259
https://tw.uukanshu.net/ 10 dipu-bd idMysteries
🔍 https://www.69shu.com/ 19 dipu-bd jere344 zGadli
🔍 https://www.69shu.pro/ 19 dipu-bd jere344 zGadli
🔍 https://www.69shuba.com/ 19 dipu-bd jere344 zGadli
🔍 https://www.69xinshu.com/ 19 dipu-bd jere344 zGadli
🔍 https://www.aixdzs.com/ 6 dipu-bd junqili259
https://www.bq99.cc/ 19 dipu-bd CryZFix
https://www.daocaorenshuwu.com/ 58 dipu-bd SirGryphin yudilee
🔍 https://www.ddxsss.com/ 2
https://www.p2wt.com/ 19 dipu-bd CryZFix
https://www.shw5.cc/ 19 dipu-bd CryZFix
https://www.soxs.cc/ 6 dipu-bd junqili259
https://www.uukanshu.net/ 10 dipu-bd idMysteries
🔍 https://www.xbanxia.com/ 21 dipu-bd SirGryphin zGadli

Rejected sources

Source URL Rejection Cause
http://boxnovel.org/ No longer operational
http://fullnovel.live/ This site can not be reached
http://gravitytales.com/ Domain is expired
http://hs2ppe.co.uk/ No longer provides lightnovels
http://wspadancewichita.com/ Site closed and moved to https://readnovelfull.com/
http://www.hanyunovels.site/ Site is down
https://4scanlation.com/ Domain expired
https://888novel.com/ Gets IP banned for using crawler
https://anythingnovel.com/ The domain is for sale
https://arangscans.com/ This site can not be reached
https://bestoflightnovels.com/ This site can not be reached
https://boxnovel.online/ This site can not be reached
https://boxnovel.org/ No longer operational
https://clicknovel.net/ The domain has expired
https://dsrealmtranslations.com/ Domain expired
https://fsapk.com/ No longer provides lightnovels
https://indomtl.com/ Not crawler friendly
https://instadoses.com/ This site can not be reached
https://kiss-novel.com/ Site is down
https://light-novel.online/ The domain has expired
https://lightnovel.tv/ This site can not be reached
https://lightnovelkiss.com/ This site can not be reached
https://lightnovelshub.com/ No longer provides lightnovels
https://mtled-novels.com/ Domain is expired
https://myoniyonitranslations.com/ This site can not be reached
https://newsite.kolnovel.com/ Site is down
https://novelcrush.com/ Site is down
https://novelplanet.com/ Site is closed
https://novelraw.blogspot.com/ Site closed down
https://novelsrock.com/ Web server is down
https://omgnovels.com/ Site is down
https://overabook.com/ Site is down
https://pery.info/ Site is down
https://ranobelib.me/ Access denied
https://read.asianovel.com/ Connection timed out
https://readnovelz.net/ Redirects to webnovelonline.net
https://tunovelaligera.com/ Broken. Chapters does not load
https://viewnovel.net/ Site is down
https://wordexcerpt.org/ This site can not be reached
https://writerupdates.com/ Site is down
https://wuxia.click/ Access denied
https://wuxiaworld.io/ Cloudflare Error 522, can not connect to host
https://wuxiaworld.live/ The domain has expired
https://wuxiaworld.site/ Access denied
https://www.centinni.com/ Site is down
https://www.novelspread.com/ Site is down
https://www.noveluniverse.com/ Site is down
https://www.novelv.com/ Site is down
https://www.rebirth.online/ Redrects to https://foxteller.com/
https://www.shinsori.com/ This site can not be reached
https://www.translateindo.com/ Site is down
https://www.wuxiaworld.co/ This site can not be reached

Supported output formats

  • JSON
  • EPUB
  • TEXT
  • WEB
  • DOCX
  • MOBI
  • PDF
  • RTF
  • TXT
  • AZW3
  • FB2
  • LIT
  • LRF
  • OEB
  • PDB
  • RB
  • SNB
  • TCR

Sponsors


Wuxiaworld

lightnovel-crawler's People

Contributors

alzamer2 avatar amritoo avatar ancientcatz avatar camfurbush avatar cryzfix avatar dependabot[bot] avatar dipu-bd avatar drewbitt avatar galunid avatar hoangphuc05 avatar idmysteries avatar jere344 avatar junqili259 avatar kuwoyuki avatar mesmerlord avatar mymro avatar nd2024 avatar needkvas avatar nilanekanayake avatar nntin avatar pmosko avatar seven0492 avatar sirgryphin avatar typotami avatar watzeedzad avatar wuxmes avatar yudilee avatar zerty avatar zgadli avatar zimorok avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lightnovel-crawler's Issues

LNMTL login issues

I'm trying to download from this url https://lnmtl.com/novel/forty-millenniums-of-cultivation
This is one of the novels on the site that needs logging in to read.
exact command I'm using
lncrawl --login <username> <password> -s https://lnmtl.com/novel/forty-millenniums-of-cultivation
username and password redacted for obvious reasons.
followed onscreen prompts for output directory and selected option for first 10 chapters
"body is empty" for every chapter; the generated file are indeed empty
lncrawl --version returns 2.7.6

also i tried downloading the above novel by just going through prompts instead of using option flags and i never got prompted to login despite the novel requiring it.

LNMTL not working

LNMTL doesnt seem to be working. For example I trried the last 10 chapters of Martial God Asura and it came back with this:

"? Enter an url or novel name to find: https://lnmtl.com/novel/martial-god-asura
Retrieving novel info...
NOVEL: Martial God Asura
? Which chapters to download? Last 10 chapters
Getting cover image...
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3623
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3621
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3624
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3620
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3622
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3625
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3628
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3627
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3626
�[KBody is empty: https://lnmtl.com/chapter/martial-god-asura-book-8-chapter-3629
�[KDownloading chapters |████████████████████████████████| 10/10
�[?25hCreated: 10 text files
Created: 10 html files
Created: Martial God Asura.epub
Created: Martial God Asura.mobi"

And the files contain no text. I have been able to get other sites to work fine just not LNMTL.
I have been trying with and without the login option. If it works than can you please give an example of the commands to get it to work, thanks.

Novel binding fails

Processing: _novel\8093990805004205
!! Failed to bind: _novel\8093990805004205

This is with when providing the chapter numbers.

python3 main.py webnovel 8093990805004205
Getting CSRF Token from https://www.webnovel.com/book/8093990805004205
CSRF Token = BEwNDFH7yoADt2uvgWH9Y2ZdxMHKanalcugCh9WI
Getting book name and chapter list...
1646 chapters found
Processing: _novel\8093990805004205
Creating: NA\NA_v.epub


Amazon kindlegen(Windows) V2.9 build 1029-0897292
A command line e-book compiler
Copyright Amazon.com and its Affiliates 2014


Info:I9007:option: -c2: Kindle Huffdic compression
Error(opfparser):E20004: the id in the spine does not match any item in the manifest: cover

That is without giving chapter numbers.

Discord bot file size limit

Discord won't allow uploading files larger than 8MB. But most of the compressed file has size > 8MB. Need another solution to send files to discord.

Readlightnovel.com problems

I think there is a problem with readlightnovel.com
the rest of the sites are okay.
i Tried: book_crawler readln full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1830 https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1835

and this is what i get from terminal.
Visiting https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband
Getting book name and chapter list...
[Full Marks Hidden Marriage: Pick Up a Son, Get a Free Husband] 1842 chapters found
Downloading https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1830
Downloading https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1831
Downloading https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1832
Downloading https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1833
Downloading https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1834
Saving Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19/01836.json
Saving Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19/01835.json
Downloading https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1835
Downloading https://www.readlightnovel.org/full-marks-hidden-marriage-pick-up-a-son-get-a-free-husband/chapter-1841
Saving Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19/01834.json
Saving Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19/01837.json
Saving Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19/01833.json
Saving Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19/01838.json
Saving Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19/01839.json
complete
Processing: Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/json/19
Creating: Full Marks Hidden Marriage Pick Up a Son, Get a Free Husband/epub/Full Marks Hidden Marriage: Pick Up a Son, Get a Free Husband_v19.epub
Traceback (most recent call last):
File "/anaconda3/lib/python3.5/site-packages/ebook_crawler/binding.py", line 128, in novel_to_mobi
generator(KINDLEGEN_PATH_MAC)
File "/anaconda3/lib/python3.5/site-packages/ebook_crawler/binding.py", line 123, in
generator = lambda kindlegen: call([kindlegen, epub_file])
File "/anaconda3/lib/python3.5/subprocess.py", line 247, in call
with Popen(*popenargs, **kwargs) as p:
File "/anaconda3/lib/python3.5/subprocess.py", line 676, in init
restore_signals, start_new_session)
File "/anaconda3/lib/python3.5/subprocess.py", line 1289, in _execute_child
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/anaconda3/lib/python3.5/site-packages/ebook_crawler/binding.py", line 136, in novel_to_mobi
generator('kindlegen')
File "/anaconda3/lib/python3.5/site-packages/ebook_crawler/binding.py", line 123, in
generator = lambda kindlegen: call([kindlegen, epub_file])
File "/anaconda3/lib/python3.5/subprocess.py", line 247, in call
with Popen(*popenargs, **kwargs) as p:
File "/anaconda3/lib/python3.5/subprocess.py", line 676, in init
restore_signals, start_new_session)
File "/anaconda3/lib/python3.5/subprocess.py", line 1289, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'kindlegen'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/anaconda3/bin/ebook_crawler", line 11, in
sys.exit(main())
File "/anaconda3/lib/python3.5/site-packages/ebook_crawler/init.py", line 51, in main
volume=volume,
File "/anaconda3/lib/python3.5/site-packages/ebook_crawler/readln.py", line 49, in start
novel_to_mobi(self.output_path)
File "/anaconda3/lib/python3.5/site-packages/ebook_crawler/binding.py", line 138, in novel_to_mobi
if err[1].errno == errno.ENOENT:
TypeError: 'FileNotFoundError' object is not subscriptable

after the result i get is only an epub with all the titles with no content.

include function/parameter to bypass "Press Enter to exit"

I'm trying to integrate and automate using the windows version of the crawler. when I try to batch out the executable, it stops after each execution due to the requirement for the "Enter" key to be pressed. I tried the -f and --suppress flags, but execution still requires the "Enter" key press. Adding this flag/parameter would help automation.

Telegram Bot when upload file finished not destroy crawler instance

Hi Sudipto,

After trying and testing the telegram bot i found that after upload zip file finished session not closed, and crawler instance not destroyed so when i call /start again, i have to call command /cancel first so bot can accept new job, is that the right flow?

And the other problem i found is even after call /cancel and i can call command /start again volume and chapter number is counted (past session + this session).

Is crawler instance passed betwen session?

Windows Binary Does nothing after inputting the novel link

the provided windows binary simply exits after entering the login information with your github link reference
Checked on a fresh install of windows with no python or any other possible dependency installed but pure windows
screenshot

UPDATE---
calling the exe with all the proper parameters instead of using the interactive menu seems to make it work absolutely fine

Implement novel searching feature

Make it more intelligence by following this workflow:

  • Take any string as input
  • If it a supported URL. start the crawler immediately
  • Otherwise display a list of websites that supports searching. User has to choose one.
  • Display search results from selected website. User will select one from the result list.
  • Start the crawler using the lightnovel URL

Implemented it for:

X represents the sites that does not support searching or could not implement searching

output_path referenced before assignment

Installed using pip on windows. Created the folders manually. See #10 .

.\ebook_crawler.exe webnovel 8093990805004205
Getting CSRF Token from https://www.webnovel.com/book/8093990805004205
CSRF Token = LI1UErwQVWiDGnvCmmpgBamfOpavmDGUCZJqglkP
Getting book name and chapter list...
1646 chapters found
Traceback (most recent call last):
File "c:\program files\python36\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\program files\python36\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\Karl\AppData\Roaming\Python\Python36\Scripts\ebook_crawler.exe_main
.py", line 9, in
File "C:\Users\Karl\AppData\Roaming\Python\Python36\site-packages\ebook_crawler_init
.py", line 34, in main
end_chapter=sys.argv[4] if len(sys.argv) > 4 else ''
File "C:\Users\Karl\AppData\Roaming\Python\Python36\site-packages\ebook_crawler\webnovel.py", line 44, in start
novel_to_kindle(self.output_path)
File "C:\Users\Karl\AppData\Roaming\Python\Python36\site-packages\ebook_crawler\binding.py", line 83, in novel_to_kindle
for file_name in sorted(os.listdir(output_path)):
UnboundLocalError: local variable 'output_path' referenced before assignment

Request: novelfull.com

I've noticed it isn't supported I was wondering if you could add it. It has very similar layout to some of the other supported sites like BoxNovel, Readinglightnovel and Novelplanet.

Need New Source

I think 4 source available is really amazing but more source available is great. I think wuxiaworld.co. and novelplanet is perfect candidate for new source to be packed into epub.

Add Intro Page to generated book

Many of source book has synopsis and infor regarding the book. maybe we can crawled it and app intro pages to generated book, maybe adding info that this book generated using this script, etc

Windows cygwin: FileNotFoundError: [WinError 3] The system cannot find the path specified

Laptop@Lenovo ~/novel
$ ebook_crawler webnovel 7931338406001705 1 10 false
Traceback (most recent call last):
  File "c:\users\laptop\appdata\local\programs\python\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\laptop\appdata\local\programs\python\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\laptop\AppData\Roaming\Python\Python36\Scripts\ebook_crawler.exe\__main__.py", line 9, in <module>
  File "C:\Users\laptop\AppData\Roaming\Python\Python36\site-packages\ebook_crawler\__init__.py", line 44, in main
    volume=volume,
  File "C:\Users\laptop\AppData\Roaming\Python\Python36\site-packages\ebook_crawler\webnovel.py", line 49, in start
    novel_to_mobi(self.output_path)
  File "C:\Users\laptop\AppData\Roaming\Python\Python36\site-packages\ebook_crawler\binding.py", line 110, in novel_to_mobi
    for file_name in sorted(os.listdir(epub_path)):
FileNotFoundError: [WinError 3] The system cannot find the path specified: '7931338406001705\\epub'
Getting CSRF Token from  https://www.webnovel.com/book/7931338406001705
CSRF Token = mNpxvSI9mDTA9EixtmCelAFwmC3Ifgv4TzZOQRqM
Getting book name and chapter list...
1159 chapters found
7931338406001705 does not exists

I get this error. I am using cygwin 64bit on Windows10. Install using pip install ebook-crawler

Add uploader for google drive

For issue #52 we can add uploader for google drive and share google drive link using send message. I have already create function to do that in my forked repository. should i pull request to master or to other branch repository?

readlightnovel.org spider duplicate text

readlightnovel.org crawler is duplicating the text. There is a hidden div in the HTML source, which does not contain ads, and a visible DIV. The crawler get's both and is not filtering the ADS sub-div either.

code from sekindo

I've noticed it before but thought it was a one off. Sometimes I get this kind of error/message in my books.

This is a sentence in the book. "Hello, I'm a example." said Me.
code from sekindo - Readlightnovel.org In-article - outstream
code from sekindo
/339474670/ReadLightNovel/InStory_1
This is a sentence in the book. "Hello, I'm a example." said Me.

As you can see I get this weird message and duplicate sentences, there's also others through out as well at random points, I've listed them below.

/339474670/ReadLightNovel/InStory_3
/339474670/ReadLightNovel/InStory_2
/339474670/ReadLightNovel/BottomStory

I don't know what causes it, I was downloading this novel https://www.readlightnovel.org/mo-tian-ji
If you need anymore info let me know and I will try and provide it.

lnmtl.com terminates when chapter_no not purely numeric

Visiting: https://lnmtl.com/chapter/the-amber-sword-book-3-chapter-531-1
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "./__main__.py", line 74, in <module>
    main()
  File "./__main__.py", line 28, in main
    end_url=sys.argv[4] if len(sys.argv) > 4 else ''
  File "./EbookCrawler/lnmtl.py", line 57, in start
    self.crawl_chapters(browser)
  File "./EbookCrawler/lnmtl.py", line 89, in crawl_chapters
    self.parse_chapter(browser)
  File "./EbookCrawler/lnmtl.py", line 112, in parse_chapter
    chapter_no = re.search(r'chapter-\d+$', url).group().strip('chapter-')
AttributeError: 'NoneType' object has no attribute 'group'

There are two chapters labeled # 531 on the website.

Add new sources

MAc user

Hi, A beginner here,
Can you please provide us an instruction for mac users? thank you

Webnovel error invalid literal for int() with base 10:

While i try to create epub from webnovel i got error invalid literal for int() with base 10:

python3 main.py webnovel 10377938706023605 https://www.webnovel.com/book/10377938706023605/27858104469219628/Last-Wish-System/Yale-Roanmad https://www.webnovel.com/book/10377938706023605/30154170363336851/Last-Wish-System/Crossing-the-Border
Getting CSRF Token from https://www.webnovel.com/book/10377938706023605
CSRF Token = 9eJJFX5txT0r9s3004p1rDY61DZrTfvslGGHmp61
Getting book name and chapter list...
148 chapters found
Traceback (most recent call last):
File "main.py", line 2, in
main()
File "/home/yudi/book/Web Scrapper/ebook_crawler/init.py", line 34, in main
end_chapter=sys.argv[4] if len(sys.argv) > 4 else ''
File "/home/yudi/book/Web Scrapper/ebook_crawler/webnovel.py", line 43, in start
self.get_chapter_bodies()
File "/home/yudi/book/Web Scrapper/ebook_crawler/webnovel.py", line 83, in get_chapter_bodies
start = int(self.start_chapter)
ValueError: invalid literal for int() with base 10: 'https://www.webnovel.com/book/10377938706023605/27858104469219628/Last-Wish-System/Yale-Roanmad'

Thanks for helping

UnicodeEncodeError: 'cp932' codec can't encode character '\xa0' in position 48: illegal multibyte sequence

�[?25hCreated: I’m in Hollywood.epub
Failed to generate mobi for I’m in Hollywood.epub
Traceback (most recent call last):
File "c:\program files (x86)\python36-32\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\program files (x86)\python36-32\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Program Files (x86)\Python36-32\Scripts\lightnovel-crawler.exe_main
.py", line 9, in
File "c:\program files (x86)\python36-32\lib\site-packages\lightnovel_crawler_init
.py", line 65, in main
start_app(crawler_list)
File "c:\program files (x86)\python36-32\lib\site-packages\lightnovel_crawler\app_init_.py", line 34, in start_app
Program().run(crawler())
File "c:\program files (x86)\python36-32\lib\site-packages\lightnovel_crawler\app\program.py", line 38, in run
bind_books(self)
File "c:\program files (x86)\python36-32\lib\site-packages\lightnovel_crawler\app\bind_books.py", line 56, in bind_books
file.write(text)
UnicodeEncodeError: 'cp932' codec can't encode character '\xa0' in position 48: illegal multibyte sequence

LNMTL partially missing output when splitting by volume

all chapters download fine, but some chapters are missing from output formats when opting to generate separate files for each volume.

on version 2.7.7
was downloading https://lnmtl.com/novel/forty-millenniums-of-cultivation
opted to generate separate file for each volume
should have 34 volumes
only 33 volumes (for all output formats except json)

with the json format
all the volume_title fields show 1 volume higher than they are supposed to be (except the very last volume)
ex. the json file for the very first chapter:
{"id": 1, "url": "https://lnmtl.com/chapter/forty-millenniums-of-cultivation-chapter-1", "volume": 1, "title": "Chapter #1 - Magical Artifact Graveyard", "volume_title": "Volume 2",...

LNMTL chapter numbers are wrong

When I compile chapters from LNMTL the chapter numbers don't match what they actually are on the site. For instance I downloaded the last 10 chapters of Martial God Asura and it had them as 3123-3132 when they should be 3620-3629 which is what they are on the site. The chapters them selves match, just not the chapter numbers.

Discord bot closed when processing

When i use discord bot. if i ask it to generate book for novel with more than 200 chapter it usually generate error like this
[ERROR] (asyncio) Task was destroyed but it is pending! task: <Task pending coro=<Client._run_event() running at /home/yudi/.local/lib/python3.6/site-packages/discord/client.py:307> wait_for=<Future pending cb=[BaseSelectorEventLoop._sock_connect_done(15)(), <TaskWakeupMethWrapper object at 0x7f52f3c029a8>()]>>
and bot will destroyed(closed). But i think this not happened with lightnovel-crawler bot in readme link. Is there something i miss when deploying discord bot?

Chapter Title from TOC and from chapter page is different

While converting scrapper to new style, i found that in the new style template chapter title is scrapped from TOC page not from chapter page. In old template the crawler get it from chapter page. In some source in toc they only write the title in number like in boxnovel and novel planet. I think we need to get chapter title from chapter page rather than from toc page.

Discord bot cannot get novel url in channel chat

Discord bot works great in 1 on 1 chat but not on channel chat because it can read novel url or search item in chat. Unlike telegram bot, in discord while in channel they don't have reply feature. Maybe we need shorter command feature for chat in channel but also retain conversation like request for 1 on 1 chat. shorter comman is similar to argument in console bot for example
for searching we can do :
!lncrawl search novel_title novel_source --> to generate novel url
for generating book
!lncrawl format_book novel_url pack_by_volume all -> to generate all chapter in format_book format
etc

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.