• taos version: 3.0.0.0
• taospy version: 2.6.2
  DB's time precision is ns which create db by:create database xx precision 'ns'. And use pd.read_sql to read data from this db's table.But the time column is int64 type not datetime type.

add higher level api for ops data, such as

1. insert batch with pd.DataFrame or numpy or tuple
2. query numpy or pd

Environment
Linux dev-desktop 5.11.0-37-generic #41~20.04.2-Ubuntu SMP Fri Sep 24 09:06:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Run
python3 insert-lines.py
Got following error prompt

['st,t1=3i64,t2=4f64,t3="t3" c1=3i64,c3=L"pass",c2=false,c4=4f64 1626006833639000000'] 1 0
Traceback (most recent call last):
  File "insert-lines.py", line 13, in <module>
    conn.schemaless_insert(lines, taos.SmlProtocol.LINE_PROTOCOL, taos.SmlPrecision.NOT_CONFIGURED)
  File "/home/dev/.local/lib/python3.8/site-packages/taos/connection.py", line 174, in schemaless_insert
    return taos_schemaless_insert(self._conn, lines, protocol, precision)
  File "/home/dev/.local/lib/python3.8/site-packages/taos/cinterface.py", line 842, in taos_schemaless_insert
    _check_if_supported()
  File "/home/dev/.local/lib/python3.8/site-packages/taos/cinterface.py", line 862, in _check_if_supported
    raise InterfaceError("C function %s is not supported in v%s: %s" % (func, taos_get_client_info(), _UNSUPPORTED[func]))
taos.error.InterfaceError: [0xffff]: C function taos_schemaless_insert is not supported in v2.2.2.0: /lib/libtaos.so: undefined symbol: taos_schemaless_insert

Issues

taos版本

root@8777afda5dc9:~# taos
Welcome to the TDengine Command Line Interface, Client Version:3.2.1.0
Copyright (c) 2023 by TDengine, all rights reserved.

  ********************************  Tab Completion  ************************************
  *   The TDengine CLI supports tab completion for a variety of items,                 *
  *   including database names, table names, function names and keywords.              *
  *   The full list of shortcut keys is as follows:                                    *
  *    [ TAB ]        ......  complete the current word                                *
  *                   ......  if used on a blank line, display all supported commands  *
  *    [ Ctrl + A ]   ......  move cursor to the st[A]rt of the line                   *
  *    [ Ctrl + E ]   ......  move cursor to the [E]nd of the line                     *
  *    [ Ctrl + W ]   ......  move cursor to the middle of the line                    *
  *    [ Ctrl + L ]   ......  clear the entire screen                                  *
  *    [ Ctrl + K ]   ......  clear the screen after the cursor                        *
  *    [ Ctrl + U ]   ......  clear the screen before the cursor                       *
  **************************************************************************************

Server is Community Edition.

taos> 

taos CLI：

SELECT * FROM test2 where  ts > '2023-06-01 00:00:00' and ts < '2023-06-30 23:59:59';

ts            |         test_1          |       
================================================================================
 2023-06-30 23:59:30.000 |            3.3378906 | 

简化后，原表50列数据

taos linux(ubuntu 20.04)：

pandas                   1.5.3   
SQLAlchemy               2.0.23 

import pandas
from sqlalchemy import create_engine, text

engine = create_engine("taos://root:[email protected]:6030/db")
conn = engine.connect()
df = pandas.read_sql(text("SELECT * FROM test2 where  ts > '2023-06-01 00:00:00' and ts < '2023-06-30 23:59:59'"), conn)
conn.close()


# print index
print(df.index)
# print data type  of element in ts column
print(type(df.ts[0]))
print(df.head(3))

RangeIndex(start=0, stop=1, step=1)
<class 'pandas._libs.tslibs.timestamps.Timestamp'>
                   ts       test_1      ...
0 2023-06-30 23:59:30  3.337891  	...
[1 rows x 53 columns]

taos windows

TDengine-client-3.2.1.0-Windows-x64.exe (9.3 M)
sqlalchemy pip 下载，应该是最新版本

import pandas
from sqlalchemy import create_engine, text

engine = create_engine("taos://root:[email protected]:6030/db")
conn = engine.connect()
df = pandas.read_sql(text("SELECT * FROM test2 where  ts > '2023-06-01 00:00:00' and ts < '2023-06-30 23:59:59'"), conn)
conn.close()


# print index
print(df.index)
# print data type  of element in ts column
print(type(df.ts[0]))
print(df.head(3))


# 两种方式都已尝试
 #self.conndf = create_engine("taos://root:[email protected]:6030/db").connect()
        # self.conn = create_engine("taos://root:taosdata@localhost:6030/db").connect()
        # self.conn = taos.connect(host="127.0.0.1", user="root",
        #                                      password="taosdata",
        #                                      database="db",
        #                                      port=6030,
        #                                      config="/etc/taos",  # for windows the default value is C:\TDengine\cfg
        #                                      timezone="Asia/Shanghai")  # default your host's timezone

Connected to pydev debugger (build 201.6668.113)
SELECT * FROM test2 where ts > '2023-06-30 23:55:00' and ts < '2023-06-30 23:59:59'
Empty DataFrame
Columns: [ts, test_1,...]
Index: []

Process finished with exit code 0

Hi there,

I was wondering if there is a way to import pandas dataframes.
I saw how to export data into a dataframe but couldn't find the reverse.

Thanks.

sub is typeof <class 'taos.subscription.TaosSubscription'>

python3.7.8
taospy 2.2.0
taosserver 2.4.0

When using a dnode, create a connect like this: taos.connect(host=self.host, port=self.port, user=self.user, password=self.pwd, database=self.db, timezone=self.tz).When I use this connect object to query sql twice,it throws an exception:[0x000b]: Unable to establish connection

But, When I create a connect like this: taos.connect(user=self.user, password=self.pwd, database=self.db, timezone=self.tz), it's all ok.In addition, the passing host and port are displayed with multiple dnodes without error too.

system info:

系统：

1. 客户端所在系统：Ubuntu 20.04.5 LTS
2. 服务端所在系统：Ubuntu 20.04.5 LTS

软件版本：

1. taospy：2.7.10
2. tdengine：3.1.0.0
3. taos客户端：3.1.0.0

配置：

1. vgroups为8
2. 去读的表算上stable的tags，一共有9个字段

使用的代码：

import taos

td_conn_local = taos.connect(host="127.0.0.1",
                     user="root",
                     password="taosdata",
                     database="test",
                     port=6030,
                     config="/etc/taos",  # for windows the default value is C:\TDengine\cfg
                     timezone="Asia/Shanghai") 
import time
now = time.time()
query = td_conn_local.query("select * from k1 limit 1000000").fetch_all_into_dict()
print(time.time()-now)

运算时间：20.743690967559814 s

切换golang的连接器：
运行时间约为3s

问题：

1. taospy是否没有多线程数据拉取，有没有计划做？
2. taospy正常拉取数据的基准速度是多少？

reappear and error msg:

import taos
import pyarrow as pa

munmap_chunk(): invalid pointer
[1]    8417 IOT instruction  ipython

python version: 3.11.4
pyarrow version: 12.0.1
libarrow version: 12.0.1

taospy 会根据Cpu核心数创建线程。

是否有方式控制线程数量

您好应用场景为网络数据包监听收集，网络原始数据包为bytes类型，在taos数据库中采用binary类型字段存储。
进行数据查询时可能会出现报错：
time_str = '2021-12-02 14:40:39.828'
result = db_connection.query("select * from nd_002 where ts = '%s'" % time_str)
for row in result:
print("result data length:", len(row[4]))
print(row)

不管结果集中是否有数据，TaosResult.row_count 一直是 0。TaosResult.field_count 正常。
版本 2.2.1

使用的unix时间戳插入数据，然后用python库查询时，时间差了8个小时，且datetime对象没有时区信息。

使用python代码查询

import taos

conn = taos.connect(host="127.0.0.1", user="root", password="taosdata")
conn.select_db("db")

r = conn.query("select time,value from  local_pac_test.test_ts_table2 limit 10")



for v in r:
    print(v)
    print(f"time:{v[0]}  value:{v[1]}")

结果

(datetime.datetime(2022, 3, 24, 16, 50, 18, 393000), 1.0)
time:2022-03-24 16:50:18.393000  value:1.0

使用taos命令行查询

python时间库处理时区问题是有点蠢……应该是直接使用datetime初始化时间，又没有添加时区信息导致的

有没有办法设定返回datetime对象的默认时区，或者直接返回时间戳的float或int？

TD版本：
3.1.0.0

连接器版本：
taos-ws-py : 0.2.5

报错代码：
host = 'XXXXXX'
port = 6041
user = 'XXXXX'
password = 'XXXXXX'
database = 'XXXXX'

self.conn: taosws.Connection = taosws.connect(f"taosws://{user}:{password}@{host}:{port}/{database}")

query = f"select _wstart, max(out_power), dev_id from td_manabox.super_pv_pcs_data where dev_id in ('2016010235100099') and ts >= '2023-08-29T17:02:20+08:00' and ts <= '2023-09-05T17:02:20+08:00' partition by dev_id interval(5m);"
result: taosws.TaosResult = self.conn.query(query)
cnt = 1
for res in result:
print(cnt, res)

报错内容：

hello there.
I encounter a problem using taospy, the version is 2.7.12.
As I execute the sql statement below, taospy raises a exception, with error message:
taos.error.ProgrammingError: [0x0216]: syntax error near "range('2022-10-26T14:45:10','2023-10-26T14:4
the sql is something like: select interp(value) from some_super_table where tag_name = 'tag1' range('2022-10-26T14:45:10','2023-10-26T14:45:10') every(10m) fill(linear) group by tbname

But this sql statement executing succefully when using taos cli.
Does it mean current version 2.7.12 can not recognize those sql statements.

use conn.statement("insert info cache_record values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")

Traceback (most recent call last):
  File "/home/ehigh/work/PythonWorker/Worker/HGSocialSecurity/main.py", line 8, in <module>
    curd.bind_insert([
  File "/home/ehigh/work/PythonWorker/Worker/HGSocialSecurity/curd/cache_table.py", line 15, in bind_insert
    stmt = conn.statement("insert info cache_record values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")
  File "/home/ehigh/.local/lib/python3.9/site-packages/taos/connection.py", line 103, in statement
    taos_stmt_prepare(stmt, sql)
  File "/home/ehigh/.local/lib/python3.9/site-packages/taos/cinterface.py", line 626, in taos_stmt_prepare
    raise StatementError(msg=taos_stmt_errstr(stmt), errno=res)
  File "/home/ehigh/.local/lib/python3.9/site-packages/taos/cinterface.py", line 656, in taos_stmt_errstr
    err = c_char_p(_libtaos.taos_stmt_errstr(stmt))
  File "/usr/local/python3/lib/python3.9/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/local/python3/lib/python3.9/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /lib/libtaos.so: undefined symbol: taos_stmt_errstr

python read_example.py

Traceback (most recent call last):
File "d:/harchern/unite/hub/test_code/tdengine/read_example.py", line 74, in
raise (err)
File "d:/harchern/unite/hub/test_code/tdengine/read_example.py", line 66, in
value = c1.execute(
File "D:\harchern\unite\hub\venv\lib\site-packages\taos\cursor.py", line 123, in execute
self._result = taos_query(self._connection._conn, stmt)
File "D:\harchern\unite\hub\venv\lib\site-packages\taos\cinterface.py", line 266, in taos_query
raise ProgrammingError(errstr, errno)
taos.error.ProgrammingError: [0x0015]:

python==3.8.2
taospy==2.2.1
win10

部分系统数据流转用的是时间戳，需要增加api控制是否进行时间戳转换。

目前是有入侵式地使用，2.2.1版本升级到2.3.5后有不兼容问题。

def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, precision=FieldType.C_TIMESTAMP_UNKNOWN):
    """Function to convert C bool row to python row"""
    
    if precision == FieldType.C_TIMESTAMP_MILLI:
        _divide = 1e+3
    elif precision == FieldType.C_TIMESTAMP_MICRO:
        _divide = 1e+6
    elif precision == FieldType.C_TIMESTAMP_NANO:
        _divide = 1e+9
    else:
        raise DatabaseError("Unknown precision returned from database")

    return [
        None if ele == FieldType.C_BIGINT_NULL else ele / _divide
        for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[: abs(num_of_rows)]
    ]

你好，注意到tdengine的python库没有支持例如创建流和定义流逻辑等，翻看历史之前好像有这个示例但已经被删除并且写法也不是很pythonic，请问后续是否有对这块的支持，或者是基于其他的考量不支持， 谢谢

all connecting way has such errors.

engine= sqlalchemy.create_engine('taosws://root:taosdata@localhost:6041')
df.read_sql('show stables',engine)

AttributeError: 'AlchemyTaosConnection' object has no attribute 'Error'

can not change database, or connect with database directly

engine= sqlalchemy.create_engine('taosws://root:taosdata@localhost:6041/log')

conn = taos.connect(database = "demo")
cursor = conn.cursor()
cursor.execute("select * from t >> t_py.csv")
Traceback (most recent call last):
File "", line 1, in
File "/home/udev/.local/lib/python3.6/site-packages/taos/cursor.py", line 127, in execute
self._result = taos_query(self._connection._conn, stmt)
File "/home/udev/.local/lib/python3.6/site-packages/taos/cinterface.py", line 294, in taos_query
raise ProgrammingError(errstr, errno)
taos.error.ProgrammingError: [0x0216]: syntax error near ">> t_py.csv"

在python中使用cursor执行sql导出数据到csv文件中时报了上述错误
python版本为3.6
tdengine版本为2.4.0.18
taospy版本为2.3.1
系统版本为ubuntu 18.04

请问我该如何做才能使用python连接器导出数据到文件中呢？谢谢。

使用的3.0 TDengine
conn = taos.connect()
sub = conn.subscribe(True, "task_data", "select * from task;", 1000)

报错AttributeError: /lib/libtaos.so: undefined symbol: taos_subscribe
看了一下，cinterface.py第346行_libtaos.taos_subscribe(从库中找不到taos_subscribe方法

Traceback (most recent call last):
python3.9/site-packages/taos/connection.py", line 198, in __del__
python3.9/site-packages/taos/connection.py", line 59, in close
TypeError: 'NoneType' object is not callable

出错代码在 connection.py 第 59 行 taos_close(self._conn)

用sqlalchemy insert 会出现问题，近期会补充这部分代码逻辑嘛？

I just runned some test and found that taos_fetch_block_v3 is pretty slow here, especaily :

• _crow_binary_to_python_block_v3
• _crow_nchar_to_python_block_v3
• taos_is_null

it seems too much cast, too much python loop.it cost at least 80% time to parse the data from c to python. is it any plan to improve this?