Many skipped files about telegram_media_downloader HOT 8 CLOSED

Update: this approach is working. I will know tomorrow if it catches all missing files. It is not downloading duplicates, and I see more files in the folders.. so this si a sensible workaround in any case. Many thanks!

from telegram_media_downloader.

@awb99 please check if the files are present inside another folder by mistake? like photo, video, voice.

make sure that your config is exactly what you want and you are not filtering out some media types or file formats

media_types:
- audio
- document
- photo
- video
- voice
file_formats:
  audio: ## all audio formats will be downloaded
  - all
  document: ## only pdf and epub will be downloaded every other format will be skipped in this case
  - pdf
  - epub

Yes change the last_id to 0 and re-runt the script it will download all files again but will check if the file is already present then the duplicate download will be removed. Please let me know how it goes also if the Telegram channel you are using is a Public channel then share it here so that I can try downloading the same and see what is happening.

from telegram_media_downloader.

The files are all in the correct folder.
Media types and file formats I kept the default config, so 'all' will be downloaded.
If you give me a telegram user then I can add you to the Channel.
The good news is that at least now with the fix a few days ago it is not aborting completely when there is a problem with one file. But would be great to have a sort of reliable backup tool.

from telegram_media_downloader.

I use "telegram desktop app" to check how many files there are .. and it still seems to download only 40% of them. In any case this is already better than exporting a channel manually; because telegram Desktop breaks after there is one problem; so big channels will never be exported at all.

from telegram_media_downloader.

The files are all in the correct folder.
Media types and file formats I kept the default config, so 'all' will be downloaded.
If you give me a telegram user then I can add you to the Channel.
The good news is that at least now with the fix a few days ago it is not aborting completely when there is a problem with one file. But would be great to have a sort of reliable backup tool.

Can you add deekayrav user to the channel so that I can have a look and fix the issue?

from telegram_media_downloader.

I tried to add you; but it says your permission does not allow it.

from telegram_media_downloader.

The new code works much better and is getting many more files.
What I noticed is that when internet connection drops, then the script exits. Perhaps instead it should go to an endless loop,
or have configurable retries:

WARNING:pyrogram.connection.connection:Connection failed! Trying again...
INFO:pyrogram.connection.connection:Disconnected
INFO:pyrogram.session.session:Session stopped
INFO:pyrogram.connection.connection:Connecting...
WARNING:pyrogram.connection.connection:timed out
Traceback (most recent call last):
  File "media_downloader.py", line 357, in <module>
    main()
  File "media_downloader.py", line 340, in main
    begin_import(config, pagination_limit=100)
  File "/usr/lib64/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "media_downloader.py", line 305, in begin_import
    async for message in messages_iter:
  File "/usr/local/lib/python3.7/site-packages/pyrogram/methods/messages/iter_history.py", line 85, in iter_history
    reverse=reverse
  File "/usr/local/lib/python3.7/site-packages/pyrogram/methods/messages/get_history.py", line 99, in get_history
    sleep_threshold=60
  File "/usr/local/lib/python3.7/site-packages/pyrogram/methods/advanced/send.py", line 81, in send
    else self.sleep_threshold)
  File "/usr/local/lib/python3.7/site-packages/pyrogram/session/session.py", line 460, in send
    return await self.send(data, retries - 1, timeout)
  File "/usr/local/lib/python3.7/site-packages/pyrogram/session/session.py", line 460, in send
    return await self.send(data, retries - 1, timeout)
  File "/usr/local/lib/python3.7/site-packages/pyrogram/session/session.py", line 460, in send
    return await self.send(data, retries - 1, timeout)
  [Previous line repeated 2 more times]
  File "/usr/local/lib/python3.7/site-packages/pyrogram/session/session.py", line 453, in send
    raise e from None
  File "/usr/local/lib/python3.7/site-packages/pyrogram/session/session.py", line 441, in send
    return await self._send(data, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/pyrogram/session/session.py", line 395, in _send
    raise e
  File "/usr/local/lib/python3.7/site-packages/pyrogram/session/session.py", line 392, in _send
    await self.connection.send(payload)
  File "/usr/local/lib/python3.7/site-packages/pyrogram/connection/connection.py", line 80, in send
    raise OSError
OSError

from telegram_media_downloader.

In the background, the script tries to re-connect. But running in an endless loop is not a solution.
Configurable reties I will add it to the backlog features.

Regarding the less number of files, those are duplicate files that are being skipped.
Closing the issue, create a new one if you face any other issues.

from telegram_media_downloader.