This project aims to develop a command line interface program to download all the relevant data from joined Telegram's groups and chats.

In order to do that, we heavily rely on Telethon library to interface with Telegram's official APIs. You can read more about its usage in their docs.

• Installation
• Running the code
  • List all chats
  • Download everything
  • Download only text
  • Download only users
  • Download only past media
  • Download only grous/channels
• Searching for invite links
  • Twitter's method
  • Telegram's method
• Export collected data
• Related work

Before installing, you must first edit docker-compose.yml and config/config.yaml to provide the necessary credentials for the application. In order for everything to work, you should supply

• Pgadmin credentials (optional)
• Postgres database credentials
• Telegram API credentials
• Twitter API credentials (optional)

Then, you must initialize the necessary infrastructure and virtual environment.

docker-compose up -d
poetry install
poetry run alembic upgrade head

This program implements several functionalities. You can check the most up to date ones by typing poetry run python main.py --help.

By default, it runs on every joined group and chat. If you want to change that behavior, specify in the config/config.yaml file a whitelist or a blacklist.

To list all joined groups and channels, along with their ID, type

poetry run python main.py --list-dialogs

To download channels, users, messages and media from joined groups and channels, type

poetry run python main.py

To download only messages from joined groups and channels, in order to speed up the process, type

poetry run python main.py --without-media

To download only users from joined groups and channels, type

poetry run python main.py --get-participants

To download only media from already seen messages, type

poetry run python main.py --download-past-media

TO BE IMPLEMENTED

In order to enumerate channels and groups to join, there are three strategies available:

• Manual search: manually search in the web for available invite links. Any channel/group joined through the official app will be shown next time you run this code.
• Twitter search: use twarc2 to automate the search for invite links in old tweets
• Telegram/Messages search: use the acquired Telegram messages to search for invite links

For now, every valid invite link found with the automated methods will be joined, without the option of manual approval by the user.

You can change the queries used to search twitter by modifying the search_queries.txt file.

poetry run python main.py --search-twitter

poetry run python main.py --search-messages

You can export the collected data stored in the database as a pg_dump file.

poetry run python scripts.py export [--dest-file DEST_FILE] [--compress]

These are two related repositories that heavily inspired the developing of telegram-bot.

• pushshift/telegram
• expectocode/telegram-export (archived)