Duplicate Mangas/chapters ID handling about haruneko HOT 15 OPEN

The only way to not get the internal svelte to crash is to dedupe the list.
This is something that MUST be dore by the engine.

from haruneko.

I see no reasons why a chapter id would be duplicated. Different language but same ID? Website must be able to differentiate what to load therefore Haru plugin should compose an identifier using id and language to make it unique anyway.

1. Would it be doable to filter dupes by default?
2. Or since it an edge case UI should at least handle it properly when we switch to another manga.

from haruneko.

Oh interresting unit test that i should add in SAM
Is that an engine issue or an UI issue ? (going to test)

from haruneko.

Error is throwed in node_modules/svelte/internal/index.mjs.

Is that an engine issue or an UI issue ? (going to test)

Both i guess. Error occurs because engine dont filter duplicate chapters, but UI should have be cleaned after we switched manga.

from haruneko.

That's because i'm using a unique key identifier:
https://github.com/manga-download/haruneko/blob/master/web/src/frontend/classic/components/MediaItemSelect.svelte#L244
{#each filteredItems as item (item.Identifier)}
looks like that having the same item.Identifier is a big issue for svelte (seems logicical tho)

from haruneko.

You're not wrong, its the website that is wrong ofc. Nonetheless, maybe something can be done to clean MedialistItem list UI (or whatever you call it) when changing manga (like, force remove spinner??)

Or remove spinner in case of error when fetching chapters? not sure if this one is easily catchable.

from haruneko.

This is an easy fix, but there are some open questions:

• If the website shows the same chapter twice, should HaruNeko do the same?
• If duplicate chapters are to be filtered out, which chapter shall be kept in case the title is different?

from haruneko.

If the website shows the same chapter twice, should HaruNeko do the same?

I think the answer is obvious. If the chapter id is the same, it means links leads to the same resource. Even if the link set a cookie for language and link to the same chapter (url) but displayed differently according to the cookie then it should be treated as 2 different chapters, Haruneko should generate 2 different ID. If its the same ID, then those are identical, there is no reasons to display 2.

If duplicate chapters are to be filtered out, which chapter shall be kept in case the title is different?

We are talking about identical ID, referencing the same resource. In other words we are creating a workaround for a website error. I see no harm in arbitrarily choosing any of them, being the first or the second, as long as we stick to our choice.
"Keep the longest title" is not a really good criteria after all.

from haruneko.

This is an easy fix, but there are some open questions:

• If the website shows the same chapter twice, should HaruNeko do the same?
• If duplicate chapters are to be filtered out, which chapter shall be kept in case the title is different?
  We have 2 issues, Titles and Identifiers (i'm using the identifier as key, but perhaps that's wrong)

T = Title, L = Link

T1/L1 + T1/L1 = Same

Ignore duplicate (show only 1)

T1/L1 + T1/L2 = different target but cannot have same name (would create the same folder).

Append "(x)" (x being the current count of duplicates of the same title)

T1/L1 + T2/L1 = same target but different name.

Leave it as it is (yes it would duplicate the content) but that's up to the website.
But that means that I shouldn't be using the identifier as key but the name ?

from haruneko.

Case : https://w31.holymanga.net/manga-list/page-331/

It ends the Manga list loop 🤷

A whole page of dupe manga

from haruneko.

T1/L1 + T1/L1 = Same
Ignore duplicate (show only 1) 👍

T1/L1 + T1/L2 = different target but cannot have same name (would create the same folder).
Append "(x)" (x being the current count of duplicates of the same title)

Those will get unique id anyway, so its not a filtering problem but a download sanitizing problem
Therefore its not a plugin issue but something to be settled in download manager.

And ofc there is the issue of recognizing duplicate folders like that in UI, since we display downloaded folders i think, idk?
Also users can different name convention when downloading :D

T1/L1 + T2/L1 = same target but different name.
Leave it as it is (yes it would duplicate the content) but that's up to the website.
But that means that I shouldn't be using the identifier as key but the name ?

Ultimately if target is the same, its the same content. I see no use case where the content behind would be different with the same url. If its different content (like, url is the same but there is a dataset parameter on the link for different language idk, and script put a cookie idk before redirecting) then the plugin have to generate an unique identifier no matter what
aka JSON.stringify(id, title, language, whatever) as unique ID.

from haruneko.

I suppose we can code a default filtering after FetchMangas & FetchChapters in the engine?

Something that could be overwritten for particular websites.

As decorator functions maybe? idk

My point is :

• same id = same ressource : take either
• if for some reason the same id point to different ressources (ie, langage set by session/cookie) the Fetch function MUST create 2 separate id

from haruneko.

That's because i'm using a unique key identifier: https://github.com/manga-download/haruneko/blob/master/web/src/frontend/classic/components/MediaItemSelect.svelte#L244 {#each filteredItems as item (item.Identifier)} looks like that having the same item.Identifier is a big issue for svelte (seems logicical tho)

Is it possible to use the item (object reference) itself as identifier instead of its ID?

{#each filteredItems as item (item)}

from haruneko.

There are other places as well, e.g. in Bookmarks.

• https://github.com/manga-download/haruneko/blob/master/web/src/frontend/classic/components/Suggestions.svelte#L63
• https://github.com/manga-download/haruneko/blob/master/web/src/frontend/classic/components/content-pages/Bookmarks.svelte#L24

It is a valid use case to have duplicate Identifiers in bookmarks (a user may have the same manga bookmarked from different websites which are using the same path style, such as seen in various WordPressMangaStream websites).

from haruneko.

Is it possible to use the item (object reference) itself as identifier instead of its ID?

Looks like it's valid (tested it quickly on the tutorial)
https://learn.svelte.dev/tutorial/keyed-each-blocks

You can use any object as the key, as Svelte uses a Map internally — in other words you could do (thing) instead of (thing.id). Using a string or number is generally safer, however, since it means identity persists without referential equality, for example when updating with fresh data from an API server.

from haruneko.