tulir / telethon-session-sqlalchemy Goto Github PK

View Code? Open in Web Editor NEW

39.0 6.0 23.0 33 KB

SQLAlchemy backend for Telethon session storage

License: MIT License

Python 100.00%

telethon session sqlalchemy

telethon-session-sqlalchemy's Introduction

Telethon SQLAlchemy session

A Telethon session storage implementation backed by SQLAlchemy.

Installation

telethon-session-sqlalchemy @ PyPI

pip install telethon-session-sqlalchemy

Usage

This session implementation can store multiple Sessions in the same database, but to do this, each session instance needs to have access to the same models and database session.

To get started, you need to create an AlchemySessionContainer which will contain that shared data. The simplest way to use AlchemySessionContainer is to simply pass it the database URL:

from alchemysession import AlchemySessionContainer
container = AlchemySessionContainer('postgres://user:pass@localhost/telethon')

If you already have SQLAlchemy set up for your own project, you can also pass the engine separately:

my_sqlalchemy_engine = sqlalchemy.create_engine('...')
container = AlchemySessionContainer(engine=my_sqlalchemy_engine)

By default, the session container will manage table creation/schema updates/etc automatically. If you want to manage everything yourself, you can pass your SQLAlchemy Session and declarative_base instances and set manage_tables to False:

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import orm
import sqlalchemy
...
session_factory = orm.sessionmaker(bind=my_sqlalchemy_engine)
session = session_factory()
my_base = declarative_base()
...
container = AlchemySessionContainer(
    session=session, table_base=my_base, manage_tables=False
)

You always need to provide either engine or session to the container. If you set manage_tables=False and provide a session, engine is not needed. In any other case, engine is always required.

After you have your AlchemySessionContainer instance created, you can create new sessions by calling new_session:

session = container.new_session('some session id')
client = TelegramClient(session)

where some session id is an unique identifier for the session.

telethon-session-sqlalchemy's People

Contributors

Stargazers

Watchers

telethon-session-sqlalchemy's Issues

MySQL not working with SQLAlchemy > v1.4.0

no problem with SQLAlchemy v1.3.23

Code that causes the issue

import alchemysession
from sqlalchemy import orm
from sqlalchemy.engine import create_engine

engine = create_engine("mysql+mysqldb://user:pass@localhost:3306/telethon?charset=utf8mb4")
session_container = alchemysession.AlchemySessionContainer(engine)

Traceback

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/alchemysession/sqlalchemy.py", line 52, in __init__
    self.db.commit()
  File "<string>", line 2, in commit
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1415, in commit
    self._transaction.commit(_to_root=self.future)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 829, in commit
    self._prepare_impl()
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
    self.session.flush()
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3233, in flush
    self._flush(objects)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3373, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 198, in raise_
    raise exception
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3333, in _flush
    flush_context.execute()
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 453, in execute
    rec.execute(self)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 627, in execute
    util.preloaded.orm_persistence.save_obj(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 242, in save_obj
    _emit_insert_statements(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1094, in _emit_insert_statements
    c = connection._execute_20(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1520, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 313, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1389, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1748, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1929, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 198, in raise_
    raise exception
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 681, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/usr/local/lib/python3.9/site-packages/MySQLdb/connections.py", line 259, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.IntegrityError: (MySQLdb._exceptions.IntegrityError) (1062, "Duplicate entry '2' for key 'version.PRIMARY'")
[SQL: INSERT INTO version (version) VALUES (%s)]
[parameters: (2,)]
(Background on this error at: http://sqlalche.me/e/14/gkpj)

MultipleResultsFound: Multiple rows were found for one_or_none

MultipleResultsFound: Multiple rows were found for one_or_none()
  File "telethon/client/updates.py", line 392, in _dispatch_update
    await self._get_difference(update, channel_id, pts_date)
  File "telethon/client/updates.py", line 515, in _get_difference
    where = await self.get_input_entity(channel_id)
  File "telethon/client/users.py", line 406, in get_input_entity
    return self.session.get_input_entity(peer)
  File "telethon/sessions/memory.py", line 216, in get_input_entity
    result = self.get_entity_rows_by_id(key, exact)
  File "alchemysession/orm.py", line 135, in get_entity_rows_by_id
    row = query.one_or_none()
  File "sqlalchemy/orm/query.py", line 3441, in one_or_none
    "Multiple rows were found for one_or_none()"

Can I use MySQL instead of SQLite?

Hello，I use MySQL instead of SQLite，I got an error

sqlalchemy.exc.CompileError: (in table 'update_state', column 'session_id'): VARCHAR requires a length on dialect mysql

Is there any solution?

column "date" is of type timestamp without time zone but expression is of type numeric

I get the error every time I disconnect the client. Below you have the full error log. I have it since the last update but I thought is because of my database and I didn't opened an issue, but after switching to a new database this keeps coming. Looks like is similar to one of my old issue: #8

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/alexandrumarcel/TelegramCompanion/tg_companion/__main__.py", line 80, in <module>
    client.loop_until_disconnected()
  File "/home/alexandrumarcel/TelegramCompanion/tg_companion/tgclient.py", line 41, in loop_until_disconnected
    self.disconnect()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/telethon/client/telegrambaseclient.py", line 379, in disconnect
    self.session.set_update_state(0, self._state)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/alchemysession/sqlalchemy.py", line 203, in set_update_state
    self.save()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/alchemysession/sqlalchemy.py", line 222, in save
    self.container.save()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/alchemysession/sqlalchemy.py", line 152, in save
    self.db.commit()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1023, in commit
    self.transaction.commit()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 487, in commit
    self._prepare_impl()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 466, in _prepare_impl
    self.session.flush()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2436, in flush
    self._flush(objects)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2574, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 277, in reraise
    raise value
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2534, in _flush
    flush_context.execute()
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 416, in execute
    rec.execute(self)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 583, in execute
    uow,
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
    insert,
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 1063, in _emit_insert_statements
    c = cached_connections[connection].execute(statement, multiparams)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
    return meth(self, multiparams, params)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
    distilled_params,
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
    e, statement, parameters, cursor, context
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1458, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 276, in reraise
    raise value.with_traceback(tb)
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
    cursor, statement, parameters, context
  File "/home/alexandrumarcel/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) column "date" is of type timestamp without time zone but expression is of type numeric
LINE 1: ...read_count) VALUES ('tg_companion', 0, 144853, 0, 1551024351...
                                                             ^
HINT:  You will need to rewrite or cast the expression.
 [SQL: 'INSERT INTO update_state (session_id, entity_id, pts, qts, date, seq, unread_count) VALUES (%(session_id)s, %(entity_id)s, %(pts)s, %(qts)s, %(date)s, %(seq)s, %(unread_count)s)'] [parameters: {'session_id': 'tg_companion', 'entity_id': 0, 'pts': 144853, 'qts': 0, 'date': 1551024351.0, 'seq': 254, 'unread_count': 1}] (Background on this error at: http://sqlalche.me/e/f405)

column version.version does not exist LINE 1: SELECT version.version AS version_version

I'm trying to create a sqlalchemy session using a pre-definied engine but I keep getting the error from the issue's title. What am I doing wrong here?

db_engine = sqlalchemy.create_engine(CONFIG.general.sqlalchemy_db_uri)
db_factory = orm.sessionmaker(bind=db_engine)
db_session = orm.scoped_session(db_factory)
db_base = declarative_base()
db_base.metadata.bind = db_engine

container = AlchemySessionContainer(engine=db_engine)

INFO:telethon.crypto.aes:libssl detected, it will be used for encryption
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column version.version does not exist
LINE 1: SELECT version.version AS version_version
               ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/mnt/c/Users/Marcel/PycharmProjects/pythonProject1/tl_radio/__main__.py", line 2, in <module>
    from .sql import sql_telethon as sql
  File "/mnt/c/Users/Marcel/PycharmProjects/pythonProject1/tl_radio/sql/sql_telethon.py", line 9, in <module>
    container = AlchemySessionContainer(engine=db_engine)
  File "/home/nitanmarcel/.local/lib/python3.8/site-packages/alchemysession/sqlalchemy.py", line 53, in __init__
    self.check_and_upgrade_database()
  File "/home/nitanmarcel/.local/lib/python3.8/site-packages/alchemysession/sqlalchemy.py", line 156, in check_and_upgrade_database
    row = self.Version.query.all()
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2726, in all
    return list(self)
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2878, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2901, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1055, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1195, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1411, in _handle_dbapi_exception
    util.raise_from_cause(
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column version.version does not exist
LINE 1: SELECT version.version AS version_version
               ^
 [SQL: 'SELECT version.version AS version_version \nFROM version'] (Background on this error at: http://sqlalche.me/e/f405)

Include LICENSE in the release

The LICENSE file is currently not included in the PyPI release. Adding the file would make the project compliant with the MIT license which requires its text to be shipped along with the source code. Additionally its inclusion would make it easier to package this project.

SQLite not working with latest SQLAlchemy

Traceback:

Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.9.1_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/[email protected]/3.9.1_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/evgeny/Documents/Development/tele/client.py", line 96, in <module>
    main()
  File "/Users/evgeny/Documents/Development/tele/client.py", line 86, in main
    helper = Client(_config)
  File "/Users/evgeny/Documents/Development/tele/client.py", line 25, in __init__
    container = AlchemySessionContainer('sqlite:///test_sessions.db')
  File "/Users/evgeny/Documents/Development/tele/venv/lib/python3.9/site-packages/alchemysession/sqlalchemy.py", line 47, in __init__
    if not self.db_engine.dialect.has_table(self.db_engine,
  File "/Users/evgeny/Documents/Development/tele/venv/lib/python3.9/site-packages/sqlalchemy/dialects/sqlite/base.py", line 2016, in has_table
    self._ensure_has_table_connection(connection)
  File "/Users/evgeny/Documents/Development/tele/venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 341, in _ensure_has_table_connection
    raise exc.ArgumentError(
sqlalchemy.exc.ArgumentError: The argument passed to Dialect.has_table() should be a <class 'sqlalchemy.engine.base.Connection'>, got <class 'sqlalchemy.engine.base.Engine'>. Additionally, the Dialect.has_table() method is for internal dialect use only; please use ``inspect(some_engine).has_table(<tablename>>)`` for public API use.```

Phone field should be BigInteger instead of Integer

Entity.phone attribute should refer to a Column(BigInteger) instead of Integer because some phone numbers will exceed the size of a normal integer in some database engines.

Message Serialization

Hi Tulir,
I know you made a Message serialization feature, is it available somewhere?

BLOB/TEXT column 'md5_digest' used in key specification without a key length

OperationalError rasies when try to a create a container with AlchemySessionContainer() with MySQL database:

sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1170, "BLOB/TEXT column 'md5_digest' used in key specification without a key length")

[SQL: CREATE TABLE sent_files ( session_id VARCHAR(255) NOT NULL, md5_digest BLOB NOT NULL, file_size INTEGER NOT NULL, type INTEGER NOT NULL, id BIGINT, hash BIGINT, PRIMARY KEY (session_id, md5_digest, file_size, type)

Telethon v1.10.6
SQLAlchemy v1.3.10 (Dialect = MySQL v8.0.18, Driver = mysqlclient v1.4.4)
telethon-session-sqlalchemy v0.2.15

Delete the session file and generate a new one?

What is the right way to delete a session. And generate a new one?

I tried with session.delete() and it seems it worked the first time but after I tried again I couldn't make it work.

The way I'm connecting is:

container = AlchemySessionContainer(DB_URI)
session = container.new_session("userbot")

Column "date" is of type bigint but expression is of type timestamp without time zone

I already made a database so I'm connecting to it with my DB_URL and everything works fine but when I start my client again the error from the title comes out.

Support saving updates.State

See LonamiWebs/Telethon@2a00bca for changes.