zehina / webtoon-downloader Goto Github PK
View Code? Open in Web Editor NEWWebtoons Scraper able to download all chapters of any series wanted.
Home Page: https://zehina.github.io/Webtoon-Downloader/
License: MIT License
Webtoons Scraper able to download all chapters of any series wanted.
Home Page: https://zehina.github.io/Webtoon-Downloader/
License: MIT License
Hello,
I would like to request an option(maybe something like --createseperatefolder(--createsepfol)) so that when someone defines a destination with --dest it still downloads the webtoon into it's own folder like when not providing --dest.
I think that would make it a lot more useful to people who set an alias for this program and want all their Webtoons downloaded into another folder,[edit:] but still have them each in their own folder.
I'm not too sure what this means
$ webtoon-downloader "https://www.webtoons.com/en/horror/ghost-teller/list?title_no=1307"
C:\Users\J\AppData\Local\Programs\Python\Python312\Lib\site-packages\PIL\Image.py:1000: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
warnings.warn(
Hey there! really great project got to know about it from https://pythonawesome.com/webtoons-scraper-able-to-download-all-chapters-of-any-series-wanted/
Wanted to ask is there any way to download webtoons like "All of us are Dead" which have "Daily pass" system and do not show the rest chapters on website url but only on their official app?
I'm willing to help because whatever you do use a desktop browser or mobile browser those chapters will only be accessible on their official apps on playstore/appstore. I primarily use Tachiyomi with webtoon extension but even it shows upto 9 chapters. which also has an open unattended issue https://github.com/tachiyomiorg/tachiyomi-extensions/issues/8705
Do yout think incorporating elements from the webtoon API provide any help?
I want to be able to save them as a PDF instead of having to turn them all into one.
I tried using it and everything went fine but after using the prompt " webtoon-downloader [https://www.webtoons.com/en/fantasy/1hp-club/list?title_no=2960]" I received "AttributeError: 'NoneType' object has no attribute 'split" error. What's wrong?
Hello,
I tried to download the prettiest platypus with this program, however, it didn't work. This is the link I used(https://www.webtoons.com/en/challenge/the-prettiest-platypus/chapter-1-the-new-cat-in-town/viewer?title_no=463063&episode_no=2).
I do see that it is a little bit different than the one in the example, however, I do not have an idea how to correct it if that is the problem.
My OS is Linux(Arch-based) with python 3.10.4, the error message when executing this(python webtoon_downloader.py "https://www.webtoons.com/en/challenge/the-prettiest-platypus/chapter-1-the-new-cat-in-town/list?title_no=463063&episode_no=2") command I get is :
Traceback (most recent call last):
File "/home/username/Documents/Webtoon-Downloader/src/webtoon_downloader.py", line 438, in
main()
File "/home/username/Documents/Webtoon-Downloader/src/webtoon_downloader.py", line 435, in main
download_webtoon(series_url, args.start, args.end, args.dest, args.images_format, args.latest, separate)
File "/home/username/Documents/Webtoon-Downloader/src/webtoon_downloader.py", line 358, in download_webtoon
viewer_url = get_chapter_viewer_url(soup)
File "/home/username/Documents/Webtoon-Downloader/src/webtoon_downloader.py", line 165, in get_chapter_viewer_url
return html.find('li', attrs={'data-episode-no': True}).find('a')['href'].split('&')[0]
AttributeError: 'NoneType' object has no attribute 'find
.
Any help would be appreciated. Sorry if this is the wrong tab in github for this kind of issue, I'm pretty new to this. Many thanks in advance.
Hello there,
I just found a minor issue and wanted to let you know. utils.py needs aiohttp but it's not included in the requirements.txt. Quickly fixed with pip, but probably worth adding. There might also be other missed imports but I'm not sure.
Is it possible to download "Fast pass" and "locked" chapters?
There is no flag to skip existing chapters. Script redownload them every time it runs
I understand, that previous downloads could be unsuccessful, but i want to have an option, while running
The other webtoons were all downloaded successfully,but this link don't,why?
link--> https://www.webtoons.com/zh-hant/heartwarming/meinanyuyelang/list?title_no=1486
Occasionally the downloaded images have the incorrect extension . PNG instead of JPEG for example.
Could you please support the korean version of the website?
Sorry for the wrong label....
[12/20/23 00:24:10] ERROR Couldn't find a tree webtoon_downloader.py:842
builder with the
features you requested:
lxml. Do you need to
install a parser
library?
┌─ Traceback (most rec─┐
│ C:\Users\Qmeri\Pycha │
│ rmProjects\Webtoon-D │
│ ownloader\src\webtoo │
│ n_downloader.py:829 │
│ in main │
│ │
│ 826 │ separate = │
│ 827 │ exporter = │
│ 828 │ try: │
│ > 829 │ │ downlo │
│ 830 │ │ │ se │
│ 831 │ │ │ ar │
│ 832 │ │ │ ar │
│ │
│ C:\Users\Qmeri\Pycha │
│ rmProjects\Webtoon-D │
│ ownloader\src\webtoo │
│ n_downloader.py:688 │
│ in download_webtoon │
│ │
│ 685 │ session.co │
│ 686 │ session.co │
│ 687 │ resp = ses │
│ > 688 │ soup = Bea │
│ 689 │ viewer_url │
│ 690 │ series_tit │
│ 691 │ if not (de │
│ │
│ C:\Users\Qmeri\AppDa │
│ ta\Local\Programs\Py │
│ thon\Python310\lib\s │
│ ite-packages\bs4_i │
│ nit_.py:250 in │
│ init │
│ │
│ 247 │ │ │ │ │
│ 248 │ │ │ bu │
│ 249 │ │ │ if │
│ > 250 │ │ │ │ │
│ 251 │ │ │ │ │
│ 252 │ │ │ │ │
│ 253 │ │ │ │ │
└──────────────────────┘
FeatureNotFound:
Couldn't find a tree
builder with the
features you requested:
lxml. Do you need to
install a parser
library?
Describe the bug
Slice-of-life comic Daybreak failing to download. After multiple attempts it downloaded the first chapter.
To Reproduce
Steps to reproduce the behavior:
command webtoon-downloader --start 2 --save-as cbz https://www.webtoons.com/en/slice-of-life/daybreak/list?title_no=4089
Expected behavior
Download chapters.
System [please complete the following information]:
Additional context
Error:
C:\Users\1\Desktop\Webtoon_Downloader>webtoon-downloader --start 2 --save-as cbz https://www.webtoons.com/en/slice-of-life/daybreak/list?title_no=4089
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:148 in handle_async_request │
│ │
│ 145 │ │ │ async with Trace( │
│ 146 │ │ │ │ "receive_response_headers", logger, request, kwargs │
│ 147 │ │ │ ) as trace: │
│ ❱ 148 │ │ │ │ status, headers = await self._receive_response( │
│ 149 │ │ │ │ │ request=request, stream_id=stream_id │
│ 150 │ │ │ │ ) │
│ 151 │ │ │ │ trace.return_value = (status, headers) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:292 in _receive_response │
│ │
│ 289 │ │ Return the response status code and headers for a given stream ID. │
│ 290 │ │ """ │
│ 291 │ │ while True: │
│ ❱ 292 │ │ │ event = await self._receive_stream_event(request, stream_id) │
│ 293 │ │ │ if isinstance(event, h2.events.ResponseReceived): │
│ 294 │ │ │ │ break │
│ 295 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:333 in _receive_stream_event │
│ │
│ 330 │ │ Will read more data from the network if required. │
│ 331 │ │ """ │
│ 332 │ │ while not self._events.get(stream_id): │
│ ❱ 333 │ │ │ await self._receive_events(request, stream_id) │
│ 334 │ │ event = self._events[stream_id].pop(0) │
│ 335 │ │ if isinstance(event, h2.events.StreamReset): │
│ 336 │ │ │ raise RemoteProtocolError(event) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:361 in _receive_events │
│ │
│ 358 │ │ │ # block until we've available flow control, event when we have events │
│ 359 │ │ │ # pending for the stream ID we're attempting to send on. │
│ 360 │ │ │ if stream_id is None or not self._events.get(stream_id): │
│ ❱ 361 │ │ │ │ events = await self._read_incoming_data(request) │
│ 362 │ │ │ │ for event in events: │
│ 363 │ │ │ │ │ if isinstance(event, h2.events.RemoteSettingsChanged): │
│ 364 │ │ │ │ │ │ async with Trace( │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:435 in _read_incoming_data │
│ │
│ 432 │ │ timeout = timeouts.get("read", None) │
│ 433 │ │ │
│ 434 │ │ if self._read_exception is not None: │
│ ❱ 435 │ │ │ raise self._read_exception # pragma: nocover │
│ 436 │ │ │
│ 437 │ │ try: │
│ 438 │ │ │ data = await self._network_stream.read(self.READ_NUM_BYTES, timeout) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\defau │
│ lt.py:67 in map_httpcore_exceptions │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\defau │
│ lt.py:371 in handle_async_request │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\connect │
│ ion_pool.py:216 in handle_async_request │
│ │
│ 213 │ │ │ │ closing = self._assign_requests_to_connections() │
│ 214 │ │ │ │
│ 215 │ │ │ await self._close_connections(closing) │
│ ❱ 216 │ │ │ raise exc from None │
│ 217 │ │ │
│ 218 │ │ # Return the response. Note that in this case we still have to manage │
│ 219 │ │ # the point at which the response is closed. │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\connect │
│ ion_pool.py:196 in handle_async_request │
│ │
│ 193 │ │ │ │ │
│ 194 │ │ │ │ try: │
│ 195 │ │ │ │ │ # Send the request on the assigned connection. │
│ ❱ 196 │ │ │ │ │ response = await connection.handle_async_request( │
│ 197 │ │ │ │ │ │ pool_request.request │
│ 198 │ │ │ │ │ ) │
│ 199 │ │ │ │ except ConnectionNotAvailable: │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\connect │
│ ion.py:101 in handle_async_request │
│ │
│ 98 │ │ │ self._connect_failed = True │
│ 99 │ │ │ raise exc │
│ 100 │ │ │
│ ❱ 101 │ │ return await self._connection.handle_async_request(request) │
│ 102 │ │
│ 103 │ async def _connect(self, request: Request) -> AsyncNetworkStream: │
│ 104 │ │ timeouts = request.extensions.get("timeout", {}) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:185 in handle_async_request │
│ │
│ 182 │ │ │ │ # must somehow have made a protocol violation. │
│ 183 │ │ │ │ raise LocalProtocolError(exc) # pragma: nocover │
│ 184 │ │ │ │
│ ❱ 185 │ │ │ raise exc │
│ 186 │ │
│ 187 │ async def _send_connection_init(self, request: Request) -> None: │
│ 188 │ │ """ │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:148 in handle_async_request │
│ │
│ 145 │ │ │ async with Trace( │
│ 146 │ │ │ │ "receive_response_headers", logger, request, kwargs │
│ 147 │ │ │ ) as trace: │
│ ❱ 148 │ │ │ │ status, headers = await self._receive_response( │
│ 149 │ │ │ │ │ request=request, stream_id=stream_id │
│ 150 │ │ │ │ ) │
│ 151 │ │ │ │ trace.return_value = (status, headers) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:292 in _receive_response │
│ │
│ 289 │ │ Return the response status code and headers for a given stream ID. │
│ 290 │ │ """ │
│ 291 │ │ while True: │
│ ❱ 292 │ │ │ event = await self._receive_stream_event(request, stream_id) │
│ 293 │ │ │ if isinstance(event, h2.events.ResponseReceived): │
│ 294 │ │ │ │ break │
│ 295 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:333 in _receive_stream_event │
│ │
│ 330 │ │ Will read more data from the network if required. │
│ 331 │ │ """ │
│ 332 │ │ while not self._events.get(stream_id): │
│ ❱ 333 │ │ │ await self._receive_events(request, stream_id) │
│ 334 │ │ event = self._events[stream_id].pop(0) │
│ 335 │ │ if isinstance(event, h2.events.StreamReset): │
│ 336 │ │ │ raise RemoteProtocolError(event) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:361 in _receive_events │
│ │
│ 358 │ │ │ # block until we've available flow control, event when we have events │
│ 359 │ │ │ # pending for the stream ID we're attempting to send on. │
│ 360 │ │ │ if stream_id is None or not self._events.get(stream_id): │
│ ❱ 361 │ │ │ │ events = await self._read_incoming_data(request) │
│ 362 │ │ │ │ for event in events: │
│ 363 │ │ │ │ │ if isinstance(event, h2.events.RemoteSettingsChanged): │
│ 364 │ │ │ │ │ │ async with Trace( │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:435 in _read_incoming_data │
│ │
│ 432 │ │ timeout = timeouts.get("read", None) │
│ 433 │ │ │
│ 434 │ │ if self._read_exception is not None: │
│ ❱ 435 │ │ │ raise self._read_exception # pragma: nocover │
│ 436 │ │ │
│ 437 │ │ try: │
│ 438 │ │ │ data = await self._network_stream.read(self.READ_NUM_BYTES, timeout) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\defau │
│ lt.py:67 in map_httpcore_exceptions │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\defau │
│ lt.py:371 in handle_async_request │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\connect │
│ ion_pool.py:216 in handle_async_request │
│ │
│ 213 │ │ │ │ closing = self._assign_requests_to_connections() │
│ 214 │ │ │ │
│ 215 │ │ │ await self._close_connections(closing) │
│ ❱ 216 │ │ │ raise exc from None │
│ 217 │ │ │
│ 218 │ │ # Return the response. Note that in this case we still have to manage │
│ 219 │ │ # the point at which the response is closed. │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\connect │
│ ion_pool.py:196 in handle_async_request │
│ │
│ 193 │ │ │ │ │
│ 194 │ │ │ │ try: │
│ 195 │ │ │ │ │ # Send the request on the assigned connection. │
│ ❱ 196 │ │ │ │ │ response = await connection.handle_async_request( │
│ 197 │ │ │ │ │ │ pool_request.request │
│ 198 │ │ │ │ │ ) │
│ 199 │ │ │ │ except ConnectionNotAvailable: │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\connect │
│ ion.py:101 in handle_async_request │
│ │
│ 98 │ │ │ self._connect_failed = True │
│ 99 │ │ │ raise exc │
│ 100 │ │ │
│ ❱ 101 │ │ return await self._connection.handle_async_request(request) │
│ 102 │ │
│ 103 │ async def _connect(self, request: Request) -> AsyncNetworkStream: │
│ 104 │ │ timeouts = request.extensions.get("timeout", {}) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:185 in handle_async_request │
│ │
│ 182 │ │ │ │ # must somehow have made a protocol violation. │
│ 183 │ │ │ │ raise LocalProtocolError(exc) # pragma: nocover │
│ 184 │ │ │ │
│ ❱ 185 │ │ │ raise exc │
│ 186 │ │
│ 187 │ async def _send_connection_init(self, request: Request) -> None: │
│ 188 │ │ """ │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:148 in handle_async_request │
│ │
│ 145 │ │ │ async with Trace( │
│ 146 │ │ │ │ "receive_response_headers", logger, request, kwargs │
│ 147 │ │ │ ) as trace: │
│ ❱ 148 │ │ │ │ status, headers = await self._receive_response( │
│ 149 │ │ │ │ │ request=request, stream_id=stream_id │
│ 150 │ │ │ │ ) │
│ 151 │ │ │ │ trace.return_value = (status, headers) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:292 in _receive_response │
│ │
│ 289 │ │ Return the response status code and headers for a given stream ID. │
│ 290 │ │ """ │
│ 291 │ │ while True: │
│ ❱ 292 │ │ │ event = await self._receive_stream_event(request, stream_id) │
│ 293 │ │ │ if isinstance(event, h2.events.ResponseReceived): │
│ 294 │ │ │ │ break │
│ 295 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:333 in _receive_stream_event │
│ │
│ 330 │ │ Will read more data from the network if required. │
│ 331 │ │ """ │
│ 332 │ │ while not self._events.get(stream_id): │
│ ❱ 333 │ │ │ await self._receive_events(request, stream_id) │
│ 334 │ │ event = self._events[stream_id].pop(0) │
│ 335 │ │ if isinstance(event, h2.events.StreamReset): │
│ 336 │ │ │ raise RemoteProtocolError(event) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:361 in _receive_events │
│ │
│ 358 │ │ │ # block until we've available flow control, event when we have events │
│ 359 │ │ │ # pending for the stream ID we're attempting to send on. │
│ 360 │ │ │ if stream_id is None or not self._events.get(stream_id): │
│ ❱ 361 │ │ │ │ events = await self._read_incoming_data(request) │
│ 362 │ │ │ │ for event in events: │
│ 363 │ │ │ │ │ if isinstance(event, h2.events.RemoteSettingsChanged): │
│ 364 │ │ │ │ │ │ async with Trace( │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:452 in _read_incoming_data │
│ │
│ 449 │ │ │ # incoming requests. │
│ 450 │ │ │ self._read_exception = exc │
│ 451 │ │ │ self._connection_error = True │
│ ❱ 452 │ │ │ raise exc │
│ 453 │ │ │
│ 454 │ │ events: typing.List[h2.events.Event] = self._h2_state.receive_data(data) │
│ 455 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_async\http2.p │
│ y:438 in _read_incoming_data │
│ │
│ 435 │ │ │ raise self._read_exception # pragma: nocover │
│ 436 │ │ │
│ 437 │ │ try: │
│ ❱ 438 │ │ │ data = await self._network_stream.read(self.READ_NUM_BYTES, timeout) │
│ 439 │ │ │ if data == b"": │
│ 440 │ │ │ │ raise RemoteProtocolError("Server disconnected") │
│ 441 │ │ except Exception as exc: │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_backends\anyi │
│ o.py:32 in read │
│ │
│ 29 │ │ │ anyio.ClosedResourceError: ReadError, │
│ 30 │ │ │ anyio.EndOfStream: ReadError, │
│ 31 │ │ } │
│ ❱ 32 │ │ with map_exceptions(exc_map): │
│ 33 │ │ │ with anyio.fail_after(timeout): │
│ 34 │ │ │ │ try: │
│ 35 │ │ │ │ │ return await self._stream.receive(max_bytes=max_bytes) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\contextlib.py:153 in __exit__ │
│ │
│ 150 │ │ │ │ # tell if we get the same exception back │
│ 151 │ │ │ │ value = typ() │
│ 152 │ │ │ try: │
│ ❱ 153 │ │ │ │ self.gen.throw(typ, value, traceback) │
│ 154 │ │ │ except StopIteration as exc: │
│ 155 │ │ │ │ # Suppress StopIteration *unless* it's the same exception that │
│ 156 │ │ │ │ # was passed to throw(). This prevents a StopIteration │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_exceptions.py │
│ :14 in map_exceptions │
│ │
│ 11 │ except Exception as exc: # noqa: PIE786 │
│ 12 │ │ for from_exc, to_exc in map.items(): │
│ 13 │ │ │ if isinstance(exc, from_exc): │
│ ❱ 14 │ │ │ │ raise to_exc(exc) from exc │
│ 15 │ │ raise # pragma: nocover │
│ 16 │
│ 17 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ReadTimeout
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \downloaders\image.py:53 in run │
│ │
│ 50 │ │ │ ImageDownloadError: If an error occurs during the download process. │
│ 51 │ │ """ │
│ 52 │ │ try: │
│ ❱ 53 │ │ │ return await self._download_image(self.client, url, target, storage) │
│ 54 │ │ except Exception as exc: │
│ 55 │ │ │ raise ImageDownloadError(url=url, cause=exc) from exc │
│ 56 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \downloaders\image.py:66 in _download_image │
│ │
│ 63 │ │ This also applies transformations to the stream, and saves it to the storage. │
│ 64 │ │ │
│ 65 │ │ """ │
│ ❱ 66 │ │ async with client.stream("GET", url) as response: │
│ 67 │ │ │ response.raise_for_status() │
│ 68 │ │ │ response_stream = response.aiter_bytes() │
│ 69 │ │ │ for transformer in self.transformers: │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\contextlib.py:199 in __aenter__ │
│ │
│ 196 │ │ # they are only needed for recreation, which is not possible anymore │
│ 197 │ │ del self.args, self.kwds, self.func │
│ 198 │ │ try: │
│ ❱ 199 │ │ │ return await anext(self.gen) │
│ 200 │ │ except StopAsyncIteration: │
│ 201 │ │ │ raise RuntimeError("generator didn't yield") from None │
│ 202 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py:1602 │
│ in stream │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py:1646 │
│ in send │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py:1674 │
│ in _send_handling_auth │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py:1711 │
│ in _send_handling_redirects │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py:1748 │
│ in _send_single_request │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\defau │
│ lt.py:370 in handle_async_request │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\contextlib.py:153 in __exit__ │
│ │
│ 150 │ │ │ │ # tell if we get the same exception back │
│ 151 │ │ │ │ value = typ() │
│ 152 │ │ │ try: │
│ ❱ 153 │ │ │ │ self.gen.throw(typ, value, traceback) │
│ 154 │ │ │ except StopIteration as exc: │
│ 155 │ │ │ │ # Suppress StopIteration *unless* it's the same exception that │
│ 156 │ │ │ │ # was passed to throw(). This prevents a StopIteration │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\defau │
│ lt.py:84 in map_httpcore_exceptions │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ReadTimeout
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \webtoon\downloaders\chapter.py:62 in run │
│ │
│ 59 │ │ │ ChapterDownloadError in case of error downloading the chapter. │
│ 60 │ │ """ │
│ 61 │ │ try: │
│ ❱ 62 │ │ │ return await self._run(chapter_info, directory, storage) │
│ 63 │ │ except Exception as exc: │
│ 64 │ │ │ raise ChapterDownloadError(chapter_info.viewer_url, exc, chapter_info=chapte │
│ 65 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \webtoon\downloaders\chapter.py:87 in _run │
│ │
│ 84 │ │ │ │ page = PageInfo(n, url, len(img_urls), chapter_info) │
│ 85 │ │ │ │ name = str(chapter_directory / self.file_name_generator.get_page_filenam │
│ 86 │ │ │ │ tasks.append(self._create_task(chapter_info, url, name, storage)) │
│ ❱ 87 │ │ │ res = await asyncio.gather(*tasks, return_exceptions=False) │
│ 88 │ │ │
│ 89 │ │ await self._report_progress(chapter_info, "Completed") │
│ 90 │ │ return res │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \webtoon\downloaders\chapter.py:107 in _task │
│ │
│ 104 │ │ """ │
│ 105 │ │ │
│ 106 │ │ async def _task() -> ImageDownloadResult: │
│ ❱ 107 │ │ │ res = await self.image_downloader.run(url, name, storage) │
│ 108 │ │ │ await self._report_progress(chapter_info, "PageCompleted") │
│ 109 │ │ │ return res │
│ 110 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \downloaders\image.py:55 in run │
│ │
│ 52 │ │ try: │
│ 53 │ │ │ return await self._download_image(self.client, url, target, storage) │
│ 54 │ │ except Exception as exc: │
│ ❱ 55 │ │ │ raise ImageDownloadError(url=url, cause=exc) from exc │
│ 56 │ │
│ 57 │ async def _download_image( │
│ 58 │ │ self, client: httpx.AsyncClient, url: str, target: str, storage: AioWriter │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ImageDownloadError: Failed to download from
"https://webtoon-phinf.pstatic.net/20220903_18/1662158595608dWbB0_PNG/1662158595595408940.png?type=opti" due to:
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\runpy.py:196 in _run_module_as_main │
│ │
│ 193 │ main_globals = sys.modules["__main__"].__dict__ │
│ 194 │ if alter_argv: │
│ 195 │ │ sys.argv[0] = mod_spec.origin │
│ ❱ 196 │ return _run_code(code, main_globals, None, │
│ 197 │ │ │ │ │ "__main__", mod_spec) │
│ 198 │
│ 199 def run_module(mod_name, init_globals=None, │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\runpy.py:86 in _run_code │
│ │
│ 83 │ │ │ │ │ __loader__ = loader, │
│ 84 │ │ │ │ │ __package__ = pkg_name, │
│ 85 │ │ │ │ │ __spec__ = mod_spec) │
│ ❱ 86 │ exec(code, run_globals) │
│ 87 │ return run_globals │
│ 88 │
│ 89 def _run_module_code(code, init_globals=None, │
│ │
│ in <module>:7 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\cmd\ │
│ cli.py:191 in run │
│ │
│ 188 │ """CLI entrypoint""" │
│ 189 │ if len(sys.argv) <= 1: │
│ 190 │ │ sys.argv.append("--help") │
│ ❱ 191 │ cli() # pylint: disable=no-value-for-parameter │
│ 192 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py:1130 in │
│ __call__ │
│ │
│ 1127 │ │
│ 1128 │ def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any: │
│ 1129 │ │ """Alias for :meth:`main`.""" │
│ ❱ 1130 │ │ return self.main(*args, **kwargs) │
│ 1131 │
│ 1132 │
│ 1133 class Command(BaseCommand): │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\rich_click\rich_command │
│ .py:152 in main │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py:1404 in │
│ invoke │
│ │
│ 1401 │ │ │ echo(style(message, fg="red"), err=True) │
│ 1402 │ │ │
│ 1403 │ │ if self.callback is not None: │
│ ❱ 1404 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ 1405 │ │
│ 1406 │ def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]: │
│ 1407 │ │ """Return a list of completions for the incomplete value. Looks │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py:760 in │
│ invoke │
│ │
│ 757 │ │ │
│ 758 │ │ with augment_usage_errors(__self): │
│ 759 │ │ │ with ctx: │
│ ❱ 760 │ │ │ │ return __callback(*args, **kwargs) │
│ 761 │ │
│ 762 │ def forward( │
│ 763 │ │ __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any # noqa: B902 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\click\decorators.py:26 │
│ in new_func │
│ │
│ 23 │ """ │
│ 24 │ │
│ 25 │ def new_func(*args, **kwargs): # type: ignore │
│ ❱ 26 │ │ return f(get_current_context(), *args, **kwargs) │
│ 27 │ │
│ 28 │ return update_wrapper(t.cast(F, new_func), f) │
│ 29 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\cmd\ │
│ cli.py:184 in cli │
│ │
│ 181 │ │ signal.signal(signal.SIGINT, _raise_graceful_exit) │
│ 182 │ │ signal.signal(signal.SIGTERM, _raise_graceful_exit) │
│ 183 │ │ with contextlib.suppress(GracefulExit): │
│ ❱ 184 │ │ │ loop.run_until_complete(main_task) │
│ 185 │
│ 186 │
│ 187 def run() -> None: │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py:649 in │
│ run_until_complete │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \webtoon\downloaders\comic.py:211 in download_webtoon │
│ │
│ 208 │ │ on_webtoon_fetched=opts.on_webtoon_fetched, │
│ 209 │ ) │
│ 210 │ │
│ ❱ 211 │ return await downloader.run() │
│ 212 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \webtoon\downloaders\comic.py:93 in run │
│ │
│ 90 │ │ │ │ task = self._create_task(chapter_info, semaphore) │
│ 91 │ │ │ │ tasks.append(task) │
│ 92 │ │ │ │
│ ❱ 93 │ │ │ results = await asyncio.gather(*tasks, return_exceptions=False) │
│ 94 │ │ │
│ 95 │ │ if self.exporter: │
│ 96 │ │ │ await self.exporter.write_data(self._directory) │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \webtoon\downloaders\comic.py:131 in task │
│ │
│ 128 │ │ async def task() -> list[DownloadResult]: │
│ 129 │ │ │ async with semaphore: │
│ 130 │ │ │ │ storage = await self._get_storage(chapter_info) │
│ ❱ 131 │ │ │ │ return await self.chapter_downloader.run(chapter_info, self._directory, │
│ 132 │ │ │
│ 133 │ │ return asyncio.create_task(task()) │
│ 134 │
│ │
│ C:\Users\1\AppData\Local\Programs\Python\Python310\lib\site-packages\webtoon_downloader\core │
│ \webtoon\downloaders\chapter.py:64 in run │
│ │
│ 61 │ │ try: │
│ 62 │ │ │ return await self._run(chapter_info, directory, storage) │
│ 63 │ │ except Exception as exc: │
│ ❱ 64 │ │ │ raise ChapterDownloadError(chapter_info.viewer_url, exc, chapter_info=chapte │
│ 65 │ │
│ 66 │ async def _run( │
│ 67 │ │ self, chapter_info: ChapterInfo, directory: str | PathLike[str], storage: AioWri │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ChapterDownloadError: Failed to download from
"https://www.webtoons.com/en/slice-of-life/daybreak/episode-4/viewer?title_no=4089&episode_no=4" due to: Failed to
download from "https://webtoon-phinf.pstatic.net/20220903_18/1662158595608dWbB0_PNG/1662158595595408940.png?type=opti"
due to:
Traceback (most recent call last): File "webtoon_downloader.py", line 169, in <module> def get_chapters_details(session: requests.session, viewer_url: str, start_chapter: int = 1, end_chapter: int = None) -> list[ChapterInfo]: TypeError: 'type' object is not subscriptable Traceback (most recent call last): File "webtoon_downloader.py", line 169, in <module> def get_chapters_details(session: requests.session, viewer_url: str, start_chapter: int = 1, end_chapter: int = None) -> list[ChapterInfo]: TypeError: 'type' object is not subscriptable
I've made sure the requirements are properly satisfied.
Webtoon Downloader doesn't understand that the filename rules for Windows are somewhat different than those for Linux. When I tried to download 'Batman: Wayne Family Adventures', it tried to create a directory name with a colon in it, which isn't valid in Windows. Attached is the resulting error text.
Like to request --chapter-name option.
Some webtoon like Infinite leveling murim or Dr Frost have custom name for each episode. Evidently, it's actuall for those, who save chapters as separate folders.
Would be great to save those names somehow!
D:\Program\mhxz\Webtoon-Downloader\src>python webtoon_downloader.py "https://www.webtoons.com/en/challenge/crawling-dreams/list?title_no=141539"
[21:02:06] WARNING Creading Directory: Crawling_Dreams webtoon_downloader.py:372
Downloading Crawling Dreams from https://www.webtoons.com/en/challenge/crawling-dreams/list?title_no=141539
Traceback (most recent call last):
File "D:\Program\mhxz\Webtoon-Downloader\src\webtoon_downloader.py", line 448, in
main()
File "D:\Program\mhxz\Webtoon-Downloader\src\webtoon_downloader.py", line 445, in main
download_webtoon(series_url, args.start, args.end, args.dest, args.images_format, args.latest, separate)
File "D:\Program\mhxz\Webtoon-Downloader\src\webtoon_downloader.py", line 383, in download_webtoon
chapters_to_download = get_chapters_details(session, viewer_url, start_chapter, end_chapter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Program\mhxz\Webtoon-Downloader\src\webtoon_downloader.py", line 206, in get_chapters_details
in enumerate(soup.find('div', class_='episode_cont').find_all('li'), start=1)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'find_all'
D:\Program\mhxz\Webtoon-Downloader\src>py "https://www.webtoons.com/en/challenge/crawling-dreams/list?title_no=141539"
C:\Python\python.exe: can't open file 'D:\Program\mhxz\Webtoon-Downloader\src\https:\www.webtoons.com\en\challenge\crawling-dreams\list?title_no=141539': [Errno 22] Invalid argument
For example, trying to download https://www.webtoons.com/en/slice-of-life/hyperfocus/list?title_no=3340 works fine, but trying to download https://www.webtoons.com/en/challenge/bittersweet-candy-bowl/list?title_no=552823 returns:
Traceback (most recent call last): File "webtoon_downloader.py", line 410, in <module> main() File "webtoon_downloader.py", line 407, in main download_webtoon(series_url, args.start, args.end, args.dest, args.images_format, args.latest, args.seperate) File "webtoon_downloader.py", line 318, in download_webtoon series_title = get_series_title(soup) File "webtoon_downloader.py", line 122, in get_series_title return html.find('h1', class_='subj').text AttributeError: 'NoneType' object has no attribute 'text'
.
I have escaped both URLs with quotes.
Greetings Zehina,
Just wanna say your program works like wonder.
Would like to know if there is a way to download the file as PDF instead of image?
Thanks in advance.
get_chapters_details
checks the url ending in episode_no=1
, if that fails it goes to episode_no=2
, however this comic: https://www.webtoons.com/en/challenge/sylvania/list?title_no=211010 starts at episode_no=3
. (Adding the extra check&get isn't particularly elegant but it works)
Is your feature request related to a problem? Please describe.
Connections get closed because the tool spams the API with requests
Describe the solution you would like
a simple delay to simulate human behavior
Would be nice to have an option to pad file/directory names with leading zeroes, so instead of a directory listing that goes:
1 10 100 101 102 103 104 105 106 107 108 109 11 110 (etc.)
instead it would produce a listing that goes
001 002 003 004 005 006 007 008 009 010 011 012 (etc.)
When Navigate to the webtoon then src using the cd command, then paste this "pip install -r requirements.txt " i get 'pip' is not recognized as an internal or external command,
operable program or batch file.
The previous steps worked fine
I was successfully able make it work once and downloaded a webtoon. Now I want to download it in separate folders, but I keep getting this: bash: webtoon_downloader: command not found
This is how I start off the program. Do I always have to run the pip everytime I open up git?
Also, how does python come into play with this? Everytime I open up python, I can never make it work. Ive been using just git and was able to download a webtoon successfully once. I do not know why I cannot produce the same results again
ok so I downloaded python since it seems like this is what this was using (im new to github things) and i copy and pasted this "git clone https://github.com/Zehina/Webtoon-Downloader.git" into the python bar >>> and it gave me the syntax error. I was wondering if there was a problem with this or if I'm doing something wrong.
py webtoon_downloader.py "https://www.webtoons.com/en/horror/ghost-teller/prologue/viewer?title_no=1307&episode_no=1"
Hello,
I wanted to ask whether it would be possible to make an option as described in the title. Some Webtoons have quite long pages, and look pretty bad when they get downloaded as separate images (e.g. text-bubbles getting cut of etc.), so I wanted to ask whether that would be possible.
Have a nice day!
Is there any major difference vs manga-py? I exclude the folder naming or organisation of files or the way app displays information at download. I think more like better quality of images, or additional content like soundtracks.
Could you please add a --latest tag option to download only the last available chapter?
Also would be great to add a "create .cbz" option.
I want to shedule the download process via task manager and the above options would help doing it effortlesly.
For example, here is available both of above mentioned options.
Hello,
I can't download this https://www.webtoons.com/en/challenge/serious-trans-vibes/list?title_no=206579 webtoon Haven't encountered that problem with any other webtoon. Here's the error message:
wtdl https://www.webtoons.com/en/challenge/serious-trans-vibes/list?title_no=206579
[08:20:04] WARNING Directory Exists: /home/username/Documents/Webtoon-Downloader/ webtoon_downloader.py:366
Downloading Serious Trans Vibes from https://www.webtoons.com/en/challenge/serious-trans-vibes/list?title_no=206579
Traceback (most recent call last):
File "/home/username/Documents/Webtoons/Webtoon-Downloader/src/webtoon_downloader.py", line 438, in
main()
File "/home/username/Documents/Webtoons/Webtoon-Downloader/src/webtoon_downloader.py", line 435, in main
download_webtoon(series_url, args.start, args.end, args.dest, args.images_format, args.latest, separate)
File "/home/username/Documents/Webtoons/Webtoon-Downloader/src/webtoon_downloader.py", line 374, in download_webtoon
chapters_to_download = get_chapters_details(session, viewer_url, start_chapter, end_chapter)
File "/home/username/Documents/Webtoons/Webtoon-Downloader/src/webtoon_downloader.py", line 203, in get_chapters_details
in enumerate(soup.find('div', class_='episode_cont').find_all('li'), start=1)]
AttributeError: 'NoneType' object has no attribute 'find_all'
. Sorry if it's something easy/obvious. Thanks in advance.
Is your feature request related to a problem? Please describe.
I live in russian, and sometimes "Roskomnadzor" blocks sites. Webtoon are blocked for no reason. How do i download chapters?
Describe the solution you would like
add "--proxy [optional proxy adress]", if there is no connection, WD tries to use some proxy adresses or using ur own adress.
Additional context
I hate living in russia cause' "Roskomnadzor" does more crazy.
This will create an issue where the chapter names may create an error if the --out
argument is used.
This is what happened in downloading dungeons and artifacts
The chapter name is What's a Goblin's Favorite Type of Fish?
Originally posted by @Code-Blender-7 in #3 (comment)
Hello,
Can you make it in the form of an app, so that ordinary people can use it?
Have a nice day!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.