Giter VIP home page Giter VIP logo

ydb-sqlalchemy's Issues

order_by is failing

attempt to get sorted data

from sqlalchemy import create_engine, Column, Integer
from sqlalchemy.dialects import registry
from sqlalchemy.orm import declarative_base
from sqlalchemy.orm import sessionmaker

registry.register("yql.ydb", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")
registry.register("ydb", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")
registry.register("yql", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")

engine = create_engine("yql+ydb://localhost:2136/local")
Base = declarative_base()
Session = sessionmaker(bind=engine)
session = Session()

class Table(Base):
    __tablename__ = "mytable"
    col = Column(Integer, primary_key=True)

Base.metadata.create_all(engine)

session.add(Table(col=1))
session.commit()

q = session.query(Table).order_by(Table.col).first()
print(q.col)

fails with following error message:

sqlalchemy.exc.DataError: (ydb_sqlalchemy.dbapi.errors.DataError) position { row: 2 column: 23 } message: "Column col is not in source column set" end_position { row: 2 column: 23 } severity: 1 (server_code: 400080)
[SQL: SELECT mytable.col AS mytable_col 
FROM mytable ORDER BY mytable.col
 LIMIT CAST(%(param_1)s AS UInt64)]
[parameters: {'param_1': 1}]
(Background on this error at: https://sqlalche.me/e/20/9h9h)

although this works

q = session.query(Table.col.label("labelled_col")).order_by("labelled_col").first()
print(q.labelled_col)

but its inconvenient.

it may be related to label style:
https://docs.sqlalchemy.org/en/14/core/selectable.html#label-style-constants

if we change LABEL_STYLE_TABLENAME_PLUS_COL to LABEL_STYLE_DISAMBIGUATE_ONLY in this row:
https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/orm/context.py#L1073
order_by works without having to use label.

tried to use set_label_style or _label_style in MySession as in:
https://stackoverflow.com/questions/75322094/sqlalchemy-relationship-fields-name-constructor
but it doesnt help (need label)

using sqlalchemy 2+.

postgresql dialect works fine without relabelling.

ilike is not supported

code

from sqlalchemy import Column, Integer, String, create_engine
from sqlalchemy.dialects import registry
from sqlalchemy.orm import declarative_base
from sqlalchemy.orm import sessionmaker

registry.register("yql.ydb", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")
registry.register("ydb", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")
registry.register("yql", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")

engine = create_engine("yql+ydb://localhost:myport/mydb")
Base = declarative_base()
Session = sessionmaker(bind=engine)

class Table(Base):
    __tablename__ = "myexampletable"
    cstr = Column(String(), primary_key=True)
    cint = Column(Integer(), primary_key=True)

Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)

session = Session()

values_list = [
    Table(cstr="aaa", cint=111),
    Table(cstr="ccc", cint=111),
    Table(cstr="bbb", cint=222),
]

session.add_all(values_list)
session.commit()

q = session.query(Table).filter(Table.cstr.ilike("a%")).all()
for c in q:
    print(c.cstr, c.cint)

results in following error:

sqlalchemy.exc.DataError: (ydb_sqlalchemy.dbapi.errors.DataError) position { row: 3 column: 39 } message: "Unknown builtin: lower" end_position { row: 3 column: 39 } severity: 1 ,position { row: 3 column: 7 } message: "Unknown builtin: lower" end_position { row: 3 column: 7 } severity: 1 (server_code: 400080)
[SQL: SELECT myexampletable.cstr AS myexampletable_cstr, myexampletable.cint AS myexampletable_cint 
FROM myexampletable 
WHERE lower(myexampletable.cstr) LIKE lower(%(cstr_1)s)]
[parameters: {'cstr_1': 'a%'}]

i suggest it happens somewhere here:
https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/sql/compiler.py#L3451

maybe should be rewritten like its done for postgresql:
https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/dialects/postgresql/base.py#L1791

there is lower function support in ydb:
https://ydb.tech/docs/ru/yql/reference/udf/list/unicode

Support Binary/String datatype

Run Tests:
$ pytest -vv test/test_suite.py::BinaryTest
Failed with:
sqlalchemy.exc.StatementError: (builtins.AttributeError) module 'ydb_sqlalchemy.dbapi' has no attribute 'Binary'

authorization is not supported

if in connection.py in driver we add credentials=ydb.credentials_from_env_variables() like this:

155     def _create_driver(self):
156         driver_config = ydb.DriverConfig(
157             endpoint=self.endpoint,
158             database=self.database,
159             table_client_settings=self._get_table_client_settings(),
160             credentials=ydb.credentials_from_env_variables(),
161         )

we can use: export YDB_ACCESS_TOKEN_CREDENTIALS=my_token

update query is failing

cannot update table:

from sqlalchemy import create_engine, Column, Boolean, Integer
from sqlalchemy.dialects import registry
from sqlalchemy.orm import declarative_base
from sqlalchemy.orm import sessionmaker

registry.register("yql.ydb", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")
registry.register("ydb", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")
registry.register("yql", "ydb_sqlalchemy.sqlalchemy", "YqlDialect")

engine = create_engine("yql+ydb://localhost:2136/local")
Base = declarative_base()
Session = sessionmaker(bind=engine)
session = Session()

class Table(Base):
    __tablename__ = "dir/table"
    pk = Column(Integer, primary_key=True)
    col = Column(Boolean)

Base.metadata.create_all(engine)

session.add(Table(pk=1, col=False))
session.commit()

row = session.query(Table).first()
row.col = True
session.commit()

fails with following message:

sqlalchemy.exc.DataError: (ydb_sqlalchemy.dbapi.errors.DataError) position { row: 1 column: 57 } message: "Unknown name: $dir" end_position { row: 1 column: 57 } severity: 1 (server_code: 400080)
[SQL: UPDATE `dir/table` SET col=%(col)s WHERE `dir/table`.col = %(dir/table_col)s]
[parameters: {'col': True, 'dir/table_col': False}]
(Background on this error at: https://sqlalche.me/e/20/9h9h)

SQLAlchemy for YDB

Roadmap:

  • #5
  • dbapi:
    • OLTP (use execute instead of scan_query)
    • support yql statements with parameters (declare generation)
    • tests
    • #15
    • support transactions: commit/rollback/tx_mode
  • support ddl statements (create/drop table)
  • support dml:
    • select:
      • where/filter
      • #6
      • distinct
      • view (support index)
      • join
      • group by
      • limit/offset
      • support scan query
    • insert
      • simple insert (one row insert)
      • inserts multiple rows - generate unify query (as an option use List<Struct<..>> with as_table)
    • update
    • delete
    • upsert
    • replace
  • types:
    • Bool
    • Integer/Int64
    • Int8, Int32
    • Uint8, Uint32, Uint64
    • Double
    • #12
    • #13
    • Utf8
    • Uuid as Utf8
    • Json, JsonDocument
    • Yson
    • Timestamp
    • #7
    • Date
    • List
    • Set
    • Tuple
    • Dict
    • Struct
    • Enum
    • #9
  • support transactions commit/rollback/mode
  • support sqlalchemy v2 (https://docs.sqlalchemy.org/en/20/)
  • asyncio support (https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html)
  • benchmarks (native driver vs sqlalchemy)
  • demo application
  • slo
  • #11
  • #14

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.