mloader saves all images when using the -raw parameter as PNG when actually (as far as I've downloaded) they're JPG.

This I can verify when opening any image in irfanview and the program will say that it detected that the image is JPG, and it'll ask if I want to rename the extension.

I imagine that it detects this by looking at the file metadata.

Could you make mloader save the images as JPG?

mkdir doesn't work since the last update. I have to manually create the mloader_downloads folder to be able to download anything.

Traceback (most recent call last): File "C:\Users\XXXX\AppData\Local\Programs\Python\Python39\lib\pathlib.py", line 1312, in mkdir self._accessor.mkdir(self, mode)

Windows 10

Describe the bug
Doing any command in mloader including --help results in the error displayed on the stack trace.

Expected behavior

Expected to see help information

Stack trace

╰─ mloader --help
Traceback (most recent call last):
File "/home/subject/.local/bin/mloader", line 5, in
from mloader.main import main
File "/home/subject/.local/lib/python3.8/site-packages/mloader/main.py", line 10, in
from mloader.exporter import RawExporter, CBZExporter
File "/home/subject/.local/lib/python3.8/site-packages/mloader/exporter.py", line 8, in
from mloader.response_pb2 import Title, Chapter
File "/home/subject/.local/lib/python3.8/site-packages/mloader/response_pb2.py", line 21, in
create_key=_descriptor._internal_create_key,
AttributeError: module 'google.protobuf.descriptor' has no attribute '_internal_create_key'

Environment (please complete the following information):

• OS: Ubuntu 20.04.6 LTS on Windows 11 x86_64
• Python version: Python 3.8.10
• mloader version: 1.1.12
• Mangaplus URL: N/A

Any way to hide the display of the files you have already downloaded on the screen and only display the process of the new one being downloaded, so as not to always fill the entire screen with the entire downloaded list?

From #36 (comment):

As you probably know Shueisha locked some series behind a paywall of subscription. These series only have 3 first chapters available and after downloading them, the script throws an error. Assassination Clasroom (100001) is an example. Would be great if the error could throw some human readable message.

Describe the bug
As stated in the title, it fails to get chapter ids when trying to download an entire title.

Arguments:
-t 100235 -r

Output:

03.04.2023 01:49:40 |   INFO   |  __main__.py   206  | Started export
03.04.2023 01:49:42 |  ERROR   |  __main__.py   223  | Failed to download manga
Traceback (most recent call last):
  File "C:\Users\{redacted}\mloader\__main__.py", line 215, in main
    loader.download(
  File "C:\Python311\Lib\site-packages\mloader\loader.py", line 191, in download
    manga_list = self._normalize_ids(
                 ^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\mloader\loader.py", line 115, in _normalize_ids
    details.chapter_list_group.first_chapter_list,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'TitleDetailView' object has no attribute 'chapter_list_group'
03.04.2023 01:49:42 |   INFO   |  __main__.py   224  | SUCCESS

Environment:

• OS: Windows 11 x64
• Python version: python 3.11.0 x64
• mloader version: 1.1.9 dev branch (117689c)
• Mangaplus URL: any title

Hello!
I get this error while using this command: mloader 1002187

11.06.2020 15:30:17 | INFO | mloader.py 164 | Started export
11.06.2020 15:30:18 | ERROR | mloader.py 171 | Failed to save images
Traceback (most recent call last):
File "c:\python38\lib\site-packages\mloader\mloader.py", line 169, in main
loader.save(out_dir, RawExporter if raw else CBZExporter)
File "c:\python38\lib\site-packages\mloader\mloader.py", line 123, in save
exporter: ExporterBase = exporter_cls(dst, title, chapter_name)
File "c:\python38\lib\site-packages\mloader\mloader.py", line 69, in init
self.path.mkdir(parents=True, exist_ok=True)
File "c:\python38\lib\pathlib.py", line 1284, in mkdir
self._accessor.mkdir(self, mode)
NotADirectoryError: [WinError 267] Érvénytelen a könyvtárnév: 'mangaplus_downloads\Samurai 8: The Tale of Hachimaru'
11.06.2020 15:30:18 | INFO | mloader.py 172 | SUCCESS

And this error while using this command: mloader -r 1002187

11.06.2020 15:31:54 | INFO | mloader.py 164 | Started export
11.06.2020 15:31:56 | ERROR | mloader.py 171 | Failed to save images
Traceback (most recent call last):
File "c:\python38\lib\pathlib.py", line 1284, in mkdir
self._accessor.mkdir(self, mode)
OSError: [WinError 123] A fájlnév, a könyvtárnév vagy a kötetcímke szintaxisa nem megfelelő: 'mangaplus_downloads\Samurai 8: The Tale of Hachimaru\#1'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python38\lib\site-packages\mloader\mloader.py", line 169, in main
loader.save(out_dir, RawExporter if raw else CBZExporter)
File "c:\python38\lib\site-packages\mloader\mloader.py", line 123, in save
exporter: ExporterBase = exporter_cls(dst, title, chapter_name)
File "c:\python38\lib\site-packages\mloader\mloader.py", line 52, in init
self.path.mkdir(parents=True, exist_ok=True)
File "c:\python38\lib\pathlib.py", line 1293, in mkdir
if not exist_ok or not self.is_dir():
File "c:\python38\lib\pathlib.py", line 1403, in is_dir
return S_ISDIR(self.stat().st_mode)
File "c:\python38\lib\pathlib.py", line 1194, in stat
return self._accessor.stat(self)
OSError: [WinError 123] A fájlnév, a könyvtárnév vagy a kötetcímke szintaxisa nem megfelelő: 'mangaplus_downloads\Samurai 8: The Tale of Hachimaru\#1'
11.06.2020 15:31:56 | INFO | mloader.py 172 | SUCCESS

In the early version, I can download an unavailable chapter if I have the link. but now I can't do that. and the code appears like this:

Please bring me that feature to overcome if I missed downloading the weekly chapter.

This issue shall be laid out in three parts in order of ease of fixing, with the third part being more of an enhancement / suggestions. It can also serve as a good template for a prospective pull request to reference against.

Filenames are currently erroneously formatted

The format_image_name function currently erroneously formats files with two . characters, due to preceding {ext} with . while defaulting / expecting ext to include the . of the extension.

Fix: Remove the . in the format string or set ext to be ext.lstrip('.') to allow for ext = ".png" and ext = "png" to both be valid.

cbz archive is currently a flat archive

It would be highly preferable for the created cbz archive to be of a folder containing the individual pages rather than of the files themselves. I.e. rather than the layout of manga-chapter.cbz being manga-001.png, manga-002.png,... it should be manga-chapter/manga-001.png.

Fix: Change add_image to use str(Path(self.path.stem)/self.format_image_name(index))) for the first argument to writestr. Alternatively, str(self.name.relative_to(self.name.parent).with_suffix('') / self.format_image_name(index)) would work without having to recast stem to Path. Either way it looks ugly for a one liner.

Filenames do not conform with established manga archival conventions

This one is the biggest set of changes though mainly deals with the same two functions from above. Currently none of the filenames used by mloader conform to current best practices. Chief among them is that the individual pages do not include their chapter number in their names. Combined with the current issue that the archives are flat (above) this is just asking for files to get clobbered by accident. For best standards refer to Daiz.

This would mainly require changing format_image_name and the initial cbz creation line in CBZExporter to include info about chapter number and some extra formatting. The most stringent conforming implementation would have the following layout for chapter 29 of a manga named "Foo" consisting of png pages (for simplicity).

Foo - c029 (web) [Viz]/
Foo - c029 (web) [Viz]/Foo - c029 (web) - p001 [Viz].png
...
Foo - c029 (web) [Viz]/Foo - c029 (web) - p022 [Viz].png

(If the chapter number exceeds 999 the numbering becomes d1000 starting at the 1000th chapter)

Fix: It is, of course, personal choice if you wish to fully conform to existing best practices. However, at the very least it would be helpful to modify format_image_name to take the chapter number as well (or make a version specifically for CBZExporter which already has said info) and format pages as {self.title} - c{self.chapnum:0>3} - p{self.index:0>3}.{ext} (assuming the extension issue is fixed by specifying ext as "png" instead of ".png").

What do i do from here?

Is your feature request related to a problem? Please describe.

I would like to crontab mloader to download the last chapter of a list of titles.

Describe the solution you'd like

It would be great to not download a chapter if it has already been downloaded.

what should i do ?? any recommendations ?? please help.

Or maybe a new repository all together? Maybe call it kmloader?

Stack trace
03.03.2024 16:24:39 | ERROR | main.py 226 | Failed to download manga
Traceback (most recent call last):
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\site-packages\mloader_main_.py", line 218, in main
loader.download(
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\site-packages\mloader\loader.py", line 189, in download
self._download(manga_list)
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\site-packages\mloader\loader.py", line 154, in _download
exporter = self.exporter(
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\site-packages\mloader\exporter.py", line 140, in init
super().init(*args, **kwargs)
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\site-packages\mloader\exporter.py", line 47, in init
self._chapter_prefix = self._format_chapter_prefix(
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\site-packages\mloader\exporter.py", line 70, in _format_chapter_prefix
if Language(language) != Language.eng:
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\enum.py", line 385, in call
return cls.new(cls, value)
File "C:\Users*\AppData\Local\Programs\Python\Python310\lib\enum.py", line 710, in new
raise ve_exc
ValueError: 7 is not a valid Language

Environment:

• mloader version: 1.1.11
• Mangaplus URL: https://mangaplus.shueisha.co.jp/titles/800001

Hello,

Could you share how you created the response protobuf file? I want to create a similar application for a different site but I don't have a lot experience scrapping sites.

Also, how did you figure out the decryption method?

Thanks

Currently, mloader works by fetching the chapter data provided by mangaplus api.

When there's an extra chapter, it only denotes itself as extra, but whose extra(?), that data is not sent from server. Only the next chapter is given.

And if there are multiple simultaneous extras, the first ones' next chapter value is Always another extra.
So, naming (or more accurately, numbering) the (continuous) extras is hard this way.

Let the Manga Serial, Current implementation and Expected result partially be like this...

Following #38, and also my own findings, To mitigate the problem, I can suggest a way.

This process needs 2 globally available variables

(or whatever denotation is best for them)

1. PassedChNum (just downloaded chapter number, c[0-9]{3} or d[0-9]{4}),
2. OldExNum (number of already downloaded simultaneous extra chapters, [0-9]{1}, default = 0)

• When it's a full chapter, increase the main variable and reset the extra number,
• When it's an extra, only increase the extra number.

Main technique

• When a chapter is fetched (say, r3 from above table), save that chapter number (as-well-as an extra number) as the above mentioned variables
  • PassedChNum=c003 OldExNum=0
• When the next chapter (r4) is fetched, the previously saved chapter number can be re-used.
  • And if this 2nd chapter is an extra, increase the extra number (OldExNum: 0 > 1),
    • c003x1 (using the above 2 variables)
  • After this, the mentioned global variable can be modified/rewriten,
    • PassedChNum=c003 OldExNum=1
• When the 3rd chapter (r5) is again an extra, increase the number in the extra part of the variable,
  • c003x2
  • Rewrite > PassedChNum=c003 OldExNum=2
• Continue for 4th (r6) downloadable chapter
  • c003x2
  • Rewrite > PassedChNum=c003 OldExNum=3
• Continue for 4th (r7) downloadable chapter
  • c004
  • As it's not an extra, reset > OldExNum=0
• And so on...

Well, something like this might help someone implement it inside mloader🤔

Looking forward to see the upcoming changes... 😃

hello I have followed you're project and I'm trying to make a simple script to decode one image
unfortunately it doesn't work I get the image with the same size but it's not readable
I think I missed something if you can give me some help please, thank you

url = 'https://mangaplus.shueisha.co.jp/drm/title/100170/chapter/1009902/manga_page/super_high/7052269.jpg?key=e3749f6bba63f862625065a35551bf54&duration=86400'

session = requests.session()
session.headers.update(
            {
                "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; "
                "rv:72.0) Gecko/20100101 Firefox/72.0"
            }
        )

resp = session.get(url)
data = bytearray(resp.content)
key = bytes.fromhex('e3749f6bba63f862625065a35551bf54')
a = len(key)
for s in range(len(data)):
    data[s] ^= key[s % a]


path = Path('')
filename = Path('test.jpg')
path.joinpath(filename).write_bytes(data)

I don't know if they have updated the web or what exactly it is because until yesterday the downloads were working without problems.

Here is what the mloader tells me when downloading

• OS: Windows 10 x64
• Python version: python 3.11.6 x64
• mloader version: 1.1.11
• Mangaplus URL: https://mangaplus.shueisha.co.jp/viewer/2010475

I don't know what's really the problem here but maybe because it's a new manga so it has had to with maybe the code or something idk, here is the link btw The new manga about sasuke

Great program, but it always download the full chapters, even though you already put chapter id.

chapter id = number in chapter url

Did I get it wrong?

M+ added french releases recently to their site, is it possible to add support for french language please ?

The script had worked perfectly for years, but then entirely stopped on the Monday of last week. The next line after starting, after exactly two seconds, is "Failed to Download Manga".

I think the website might have changed how it hosts images.

Environment:

• OS: Windows 11 x64
• Python version: 3.12 x64
• mloader version: e.g. 1.1.11
• Mangaplus URL: https://mangaplus.shueisha.co.jp/viewer/1021261

EDIT: Nevermind, I see this has already been reported.

Change in viz url format leads to error/failure. No longer using the */viewer/# format format chapters for example. Now uses */title-chapter-#/chapter/#?action=read format.

From #36 (comment) :

I'm also curious - if I were to pay for subscription, would it be recognized by the script and let me download the completed series? Does secret key carry that information?

Any non-alphanumeric punctuation are being discarded as of now

When downloading a manga through the app, if there are any punctuation (like : or ' etc. ), in the Manga's Title or Chapter's Title, they are just replaced with a white-space.

It would be good if the formatting for naming images/cbz/folder should've retained some of the punctuation to make it look more readable.

Consider this for example: https://mangaplus.shueisha.co.jp/titles/100248

Manga Name: Nue's Exorcist, but it results as Nue S Exorcist
For Chapter's names: : are deleted and ' are replaced by a white-space

What should be done?

• Just delete any ' (So that the manga title should be Nues Exorcist at least)
• For chapters, replace : with - along with previous fix
  • Chapter 23: Nue's Candidate is saved with Chapter 23 Nue s Candidate, it should be Chapter 23 - Nues Candidate

If these can be implemented, please do so 🙏🏼

Like the title suggest there are free chapters and mangas which are App exclusive. Maybe you cant implement them.

Describe the bug
The PyPI tarball doesn't include the requirements.txt which is required by setup.py

$ pip install mloader
...
Traceback (most recent call last):
  File "/private/tmp/nix-build-mloader-1.1.8.drv-0/mloader-1.1.8/nix_run_setup", line 8, in <module>
    exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
  File "setup.py", line 19, in <module>
    with open("requirements.txt", "r", "utf-8") as f:
  File "/nix/store/76b5vjb1wkl7b38g8pazwv0p6pxmi53f-python3-3.10.5/lib/python3.10/codecs.py", line 905, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'

Expected behavior
Be able to install mloader from PyPI

Stack trace

Environment (please complete the following information):

• OS: MacOS
• Python version: Python 10
• mloader version: HEAD (1a64c2e)
• Mangaplus URL:

In some mangas, the quality of some images is too low compared with others of the same manga. Also the program doesn´t download all the pages of the selected manga. For example, in:
https://manga plus shueisha co jp/viewer/2001777
The program only download 55 pages and it has 58

Hi,

I was recently writing an arch PKGBUILD for mloader and ran into some issues with the requirements.txt. This is a minor enough change that rather than opening a PR I felt an issue made more sense.

Issue:

Currently mloader's requirements.txt restricts to exact package versions with ==. This is most likely not intentional, seeing as how it seems to run fine with newer versions of the required packages. In practice this means on systems with newer versions of the required packages any install from python setup.py install will fail at run time (I assume pip to just install multiple package versions).

Proposal:

Change requirements.txt to use >= or ~= instead of ==. I.e. sed -i.bak 's/==/>=/' requirements.txt (or ~= instead of >=, should be equivalent here)

P.S. Let me know if you are interested in the PKGBUILD / have any preferences about mloader's being in the Arch User Repository (AUR) or not.

Describe the bug
When i try to type this exact command mloader -l -o "D:\manga" -t 500001, the script gives me this error

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\dougl\scoop\apps\python\current\Scripts\mloader.exe\__main__.py", line 4, in <module>
  File "C:\Users\dougl\scoop\apps\python\current\Lib\site-packages\mloader\__main__.py", line 10, in <module>
    from mloader.exporter import RawExporter, CBZExporter
  File "C:\Users\dougl\scoop\apps\python\current\Lib\site-packages\mloader\exporter.py", line 8, in <module>
    from mloader.response_pb2 import Title, Chapter
  File "C:\Users\dougl\scoop\apps\python\current\Lib\site-packages\mloader\response_pb2.py", line 34, in <module>
    _descriptor.FieldDescriptor(
  File "C:\Users\dougl\scoop\apps\python\current\Lib\site-packages\google\protobuf\descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Expected behavior
I expected the script to download the manga normally, like it did before

Environment:

• OS: Windows 11 x64
• Python version: Python 3.11.3
• mloader version: 1.1.9
• Mangaplus URL: https://mangaplus.shueisha.co.jp/titles/500001

Steps to replicate

Install the develop branch, change AUTH_PARAMS, then run the following:

$ mloader --format raw --chapter 1017028

• Actual name: Sakamoto Days - 000 (web) [Oneshot] [Unknown]
• Expected name: Sakamoto Days - c123 (web) [Unknown]`

This chapter's name is "One shot". This is probably the cause.

Environment:

• mloader version: 1.1.10@develop
• Mangaplus URL: https://mangaplus.shueisha.co.jp/viewer/1017028

Title pretty much. Is the project dead?

Hello,

I would like to download this limited-time free manga, but I encountered an error. I am using pip3. Does this have anything to do with the pip version? Here is the error message.

mloader --chapter-subdir --chapter-title -o my_manga -c 10051 https://ynjn.jp/viewer/92

            _                 _
  _ __ ___ | | ___   __ _  __| | ___ _ __
 | '_ ` _ \| |/ _ \ / _` |/ _` |/ _ \ '__|
 | | | | | | | (_) | (_| | (_| |  __/ |
 |_| |_| |_|_|\___/ \__,_|\__,_|\___|_|

25.01.2024 17:12:05 |   INFO   |  __main__.py   206  | Started export
25.01.2024 17:12:06 |   INFO   |   loader.py    138  | 1/1) Manga: 
25.01.2024 17:12:06 |   INFO   |   loader.py    139  |     Author: 
25.01.2024 17:12:06 |  ERROR   |  __main__.py   226  | Failed to download manga
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/mloader/__main__.py", line 218, in main
    loader.download(
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/mloader/loader.py", line 189, in download
    self._download(manga_list)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/mloader/loader.py", line 144, in _download
    chapter = viewer.pages[-1].last_page.current_chapter
              ~~~~~~~~~~~~^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/protobuf/internal/containers.py", line 93, in __getitem__
    return self._values[key]
           ~~~~~~~~~~~~^^^^^
IndexError: list index out of range
25.01.2024 17:12:06 |   INFO   |  __main__.py   227  | SUCCESS

manga url is : https://ynjn.jp/viewer/92/10051

Works fine for normal chapters, but extra (ex) chapter titles are appended to the file name by default. This shouldn't be the expected behavior as --chapter-title option was not used.

Used command:

$ mloader https://mangaplus.shueisha.co.jp/titles/100041

Part of the filenames:

Summer Time Rendering - c007 (web) [Viz].cbz
Summer Time Rendering - c007x1 (web) [Record 001 Shiori Kobayakawa s Diary Excerpt] [Viz].cbz
Summer Time Rendering - c011 (web) [Viz].cbz
Summer Time Rendering - c011x1 (web) [Record 002 Reviews of Ryuunosuke Nagumo s Swampman] [Viz].cbz

please help

I'm trying to download some currently hidden chapters for title:200020 at once, is there a way to do this?

mloader saves images with a trailing dot which causes some issues with image viewer programs I'm using.

Hello, just wondering was there a particular reason you wouldn't do this? I know the sub-title comes in various prefix like [Number #n:..., Chapter #n:..., #n:]. But its still possible to filter and omit all of those with regex and just grab the following sub-title, right?

Would be good as a reference for jumping between chapters.

There are a bunch of non-VIZ translations on M+, so please remove the Viz from filenames.

It seems that now when they put the ID of the chapter it detects it but does not start the download

As part of my workflow I take the downloaded chapters and insert a ComicInfo.xml into the cbz files. However, right now this step has to be manually generated and requires a bunch of work for things like dates a chapter was released. It would be awesome if some or all of that information could be generated since it should all be available on the site.

Something like adding a "--metadata" flag, to create a file with author, and date for each chapter.

Describe the bug

The LICENSE file specifies GPL3, but the license listed in setup.py is MIT

Expected behavior
Both licenses should match to help packaging mloader.

• PyPI shows MIT License (GPLv3)
• aur uses GPL 3
• nixpkgs uses GPL 3 (I'm maintainer)

Stack trace

Environment (please complete the following information):

• OS: MacOS
• Python version: Python 10
• mloader version: HEAD (1a64c2e)
• Mangaplus URL: