Giter VIP home page Giter VIP logo

clickhouse-cli's Introduction

clickhouse-cli

An unofficial command-line client for the ClickHouse DBMS. It implements some common and awesome things, such as:

  • Autocompletion (work in progress)
  • Syntax highlighting for the queries & data output (Pretty* formats)
  • Multiquery & multiline modes by default - paste anything as much as you want!
  • Pager support (less) for the data output
  • Custom, PostgreSQL-like commands like \d+ table_name or \ps. See \?
  • User-defined functions

But it works over the HTTP port, so there are some limitations for now:

  • Doesn't fully support sessions. SET options are stored locally and are sent with every request.

Install

Python 3.7+ is required.

$ pip3 install clickhouse-cli

Options

$ clickhouse-cli --help
Usage: clickhouse-cli [OPTIONS] [SQLFILE]

  A third-party client for the ClickHouse DBMS.

Options:
  -h, --host TEXT          Server host (hostname, or URL)
  -p, --port INTEGER       Server HTTP port
  -u, --user TEXT          User
  -P, --password           Password
  -d, --database TEXT      Database
  -s, --settings TEXT      Query string to be appended to every query
  -c, --cookie TEXT        Cookie header to be sent with every query
  -f, --format TEXT        Data format for the interactive mode
  -F, --format-stdin TEXT  Data format for stdin/file queries
  -m, --multiline          Enable multiline shell
  -k, --insecure           Allow insecure server connections when using SSL
  --stacktrace             Print stacktraces received from the server.
  --version                Show the version and exit.
  --help                   Show this message and exit.

Configuration file

~/.clickhouse-cli.rc is here for your service!

[defaults]
# Default connection options that will be used if the relevant argument was omitted.

host = 127.0.0.1
port = 8123
db = default
user = default
password =
cookie =

# It's not secure to store the password here in plain text.


[main]
# Allow insecure server connections when using SSL
insecure = False
# Disable multiline mode by default
multiline = False

# Show SQL statements timing
timing = True

# Preferred data format for the interactive mode
format = PrettyCompact

# Preferred data format for the non-interactive mode (file/stdin)
format_stdin = TabSeparated

# Show the reformatted query after its execution
show_formatted_query = True

# Syntax highlighting
highlight = True

# Syntax highlight certain output in the interactive mode:
highlight_output = True

# Syntax highlighting in TrueColor (if supported, see https://gist.github.com/XVilka/8346728)
highlight_truecolor = True

# Pygments Highlight theme (check out https://help.farbox.com/pygments.html for available themes)
highlight_theme = default

# Show the output via pager (if defined)
pager = False


[settings]
# You can place the server-side settings here!

# max_memory_usage = 20000000000

Quickstart

$ clickhouse-cli
clickhouse-cli version: 0.1.6
Connecting to localhost:8123
Connected to ClickHouse server.

 :) help

clickhouse-cli's custom commands:
---------------------------------
USE     Change the current database.
SET     Set an option for the current CLI session.
QUIT    Exit clickhouse-cli.
HELP    Show this help message.

PostgreSQL-like custom commands:
--------------------------------
\l      Show databases.
\c      Change the current database.
\d, \dt Show tables in the current database.
\d+     Show table's schema.
\ps     Show current queries.
\kill   Kill query by its ID.

Query suffixes:
---------------
\g, \G  Use the Vertical format.
\p      Enable the pager.

 :) \l

┌─name───────┐
│ default    │
│ quickstart │
│ system     │
└────────────┘

Ok. 3 rows in set. Elapsed: 0.022 sec.

 :) USE quickstart

Changed the current database to quickstart.

Ok.

 :) \dt

┌─name───┐
│ ontime │
└────────┘

Ok. 1 row in set. Elapsed: 0.012 sec.

 :) SELECT OriginCityName, count(*) AS flights
    FROM ontime GROUP BY OriginCityName ORDER BY flights DESC LIMIT 5

┌─OriginCityName────────┬──flights─┐
│ Chicago, IL           │ 10536203 │
│ Atlanta, GA           │  8867847 │
│ Dallas/Fort Worth, TX │  7601863 │
│ Houston, TX           │  5714988 │
│ Los Angeles, CA       │  5575119 │
└───────────────────────┴──────────┘

Ok. 5 rows in set. Elapsed: 1.317 sec.

Advanced usage

Environment variables

The available environment variables are:

  • CLICKHOUSE_HOST
  • CLICKHOUSE_PORT
  • CLICKHOUSE_USER
  • CLICKHOUSE_PASSWORD
  • CLICKHOUSE_DATABASE
  • CLICKHOUSE_COOKIE

The order of precedence is:

  • command argument
  • environment variable
  • default value in the ~/.clickhouse-cli.rc

Reading from file / stdin

$ echo 'SELECT 1, 2, 3; SELECT 4, 5, 6;' | clickhouse-cli
1	2	3

4	5	6

$ cat test.sql
SELECT 1, 2, 3;
SELECT 4, 5, 6;

$ clickhouse-cli test.sql
1 2 3

4 5 6

$ clickhouse-cli -F CSV <<< 'SELECT 1, 2, 3 UNION ALL SELECT 4, 5, 6'
1,2,3
4,5,6

Inserting the data from file

$ clickhouse-cli -q 'CREATE TABLE test (date Date, s String, i UInt64) ENGINE = TinyLog'

$ cat data.csv
2017-01-01,hello,1
2017-02-02,world,2

$ clickhouse-cli -q 'INSERT INTO test (date, s, i)' -F CSV data.csv

Ok. Elapsed: 0.037 sec.

$ clickhouse-cli -q 'SELECT * FROM test'
2017-01-01	hello	1
2017-02-02	world	2

Custom settings

$ clickhouse-cli -h 10.1.1.14 -s 'max_memory_usage=20000000000&enable_http_compression=1'

User-defined functions

Oh boy. It's a very dirty (and very untested) hack that lets you define your own functions or, actually, whatever you want, by running a find & replace operation over the query before sending the query to the server.

Say, you often run queries that parse some JSON, so you use visitParamExtractString all the time:

 :) SELECT date, ip, visitParamExtractString(headers, 'User-Agent') AS ua FROM visits LIMIT 1;

Even with autocompletion, this makes it harder to work with such queries. With this feature, you'll be able to create custom find & replace pairs to make things a little bit easier (or harder; it depends). Put this in your .clickhouse-cli.rc:

udf = {
        r'header\((.*?)\)': r'visitParamExtractString(headers, \1)',
  }

And rejoice!

 :) SELECT date, ip, header('User-Agent') AS ua FROM visits LIMIT 1;

The client will replace the matching expressions with another ones, and the query will execute correctly. See .clickhouse-cli.rc for a full example.

clickhouse-cli's People

Contributors

damnwidget avatar deric avatar dqminh avatar egzvor avatar git-hulk avatar hatarist avatar martenlindblad avatar mw866 avatar nvartolomei avatar shlyakpavel avatar vavrusa avatar windreamer avatar worenga avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

clickhouse-cli's Issues

Descriptives are broken

The standard as well as this client try to show some statistics on querying the table (see image below). While this feature still works in the official client it seems to be broken in this one:

image 2018-01-20 15 14 54

screen shot 2018-01-20 at 15 13 03

Error: Invalid value for '[FILES]...'

Trying to connect to a clickhouse instance but unable to, not sure if the password format/ parsing is tripping it up, or I am I misusing the CLI

+ ~/bin ➜ clickhouse-cli -h localhost -p 8123 -u default -P 9k:/1f:==wZZ
Usage: clickhouse-cli [OPTIONS] [FILES]...
Try 'clickhouse-cli --help' for help.

Error: Invalid value for '[FILES]...': '9k:/1f:==wZZ': No such file or directory

+ ~/bin ➜ clickhouse-cli -h localhost -p 8123 -u default -P '9k:/1f:==wZZ'
Usage: clickhouse-cli [OPTIONS] [FILES]...
Try 'clickhouse-cli --help' for help.

Error: Invalid value for '[FILES]...': '9k:/1f:==wZZ': No such file or directory

import fails on exception

I tried to import a bigger file (google big query csv dump) and it seems like a unicode character is the problem. Is a utf-8 support planned?

[root@imsm spotify]# zcat 000000000000 | clickhouse-cli  --query="INSERT INTO spotify FORMAT CSV"
Password: 
Traceback (most recent call last):
  File "/usr/bin/clickhouse-cli", line 11, in <module>
    load_entry_point('clickhouse-cli==0.1.9.6', 'console_scripts', 'clickhouse-cli')()
  File "/usr/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/clickhouse_cli/cli.py", line 335, in run_cli
    cli.run(query=query, data=sql_input)
  File "/usr/lib/python3.6/site-packages/clickhouse_cli/cli.py", line 137, in run
    return self.handle_query(query, data=data, stream=True)
  File "/usr/lib/python3.6/site-packages/clickhouse_cli/cli.py", line 244, in handle_query
    query_id=query_id
  File "/usr/lib/python3.6/site-packages/clickhouse_cli/clickhouse/client.py", line 231, in query
    response = self._query(query, params, fmt=fmt, stream=stream, data=data, **kwargs)
  File "/usr/lib/python3.6/site-packages/clickhouse_cli/clickhouse/client.py", line 106, in _query
    self.url, data=data, params=params, auth=(self.user, self.password), stream=stream, headers={'Accept-Encoding': 'identity'}, **kwargs
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 110, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 356, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1284, in _send_request
    body = _encode(body, 'body')
  File "/usr/lib64/python3.6/http/client.py", line 161, in _encode
    (name.title(), data[err.start:err.end], name)) from None
UnicodeEncodeError: 'latin-1' codec can't encode character '\u015f' in position 91844245: Body ('ş') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.

HTTPS support?

Hi, is there any plan to support HTTPS endpoint with this or is there anything blocking that? I can try to create a PR for that.

Autocompletion bugs & improvements

(in no particular order)

  • datatype completion is broken for the second column (and later)
    1

  • only one completion is displayed in case of SELECT * FROM sys<tab>
    2

  • table autocompletion should have the current database in mind
    3

  • aliased column in subqueries don't work
    src 2017-05-09 at 07 16 51

  • autocompletion prepends the table name if called inside the function's parentheses

scr 2017-05-26 at 13 46 12

clickhouse-cli panics with newer versions of clickhouse

I have a server running a custom build with a version string like "18.14.17-stable-28-g368986d02".
Connection to it, clickhouse-cli fails hard with

clickhouse-cli version: 0.2.6
Connecting to some-server:443
Traceback (most recent call last):
  File "/usr/local/bin/clickhouse-cli", line 11, in <module>
    load_entry_point('clickhouse-cli==0.2.6', 'console_scripts', 'clickhouse-cli')()
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/clickhouse_cli/cli.py", line 546, in run_cli
    cli.run(query, data_input)
  File "/usr/local/lib/python3.6/dist-packages/clickhouse_cli/cli.py", line 185, in run
    if not self.connect():
  File "/usr/local/lib/python3.6/dist-packages/clickhouse_cli/cli.py", line 124, in connect
    self.server_version = (int(version[0]), int(version[1]), int(version[2]))
ValueError: invalid literal for int() with base 10: '17-stable-28-g368986d02'

Python 3 standard library seems to be broken with latest clickhouse_server version

Hi Igor.

Looks like requests is broken with the latest version of clickhouse_server when querying for anything that takes a while. As you can see in the provided copy of the traceback looks like the standard http.client library is crashing.

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 321, in begin
    self.headers = self.msg = parse_headers(self.fp)
  File "/usr/lib/python3.6/http/client.py", line 211, in parse_headers
    raise HTTPException("got more than %d headers" % _MAXHEADERS)
http.client.HTTPException: got more than 100 headers

Probably something to report directly on ClickHouse repo?

[Feature request] Pass config as argument

Currently, you can specify default settings in the ~/.clickhouse-cli.rc which is great. However, it would be helpful to be able to simply pass a path to the config file so you can easily save and toggle between different connections. Thanks for your consideration!

Migrate to prompt-toolkit 2.x

iPython 7.0 introduced a prompt-toolkit>2.0 dependency and now it's a little bit of pain in the ass for those who have both iPython and clickhouse-cli in the same environment. (high five for the fellow Mac users who have it installed system-wide in the /usr/local/)

As a dirty workaround, I suggest putting either clickhouse-cli or ipython (or whatever else you do have with the conflicting prompt-toolkit version) in a separate virtualenv to isolate it and either modifying the PATH or making a symlink.

Sadly, I currently don't have much time to fix this, so I'll be grateful if anyone could help with that.

My silly WIP branch is here: https://github.com/hatarist/clickhouse-cli/tree/prompt_toolkit_upgrade.
It has some things broken (i.e. tab autocompletion).

Make metadata refresh optional

This code refreshes metadata after every query:

        if refresh_metadata and input_data:
            self.cli.application.buffer.completer.refresh_metadata()

On a large remote cluster this takes a long time:

SHOW DATABASES
time: 0.037s bytes: 159 rows: 21

SELECT database, table, name, type FROM system.columns;
time: 0.585s bytes: 971449 rows: 15396

clickhouse-cli

Let's have a cli option to disable this behavior.

[Bugs] clickhouse-cli= 0.3.5 does not work with prompt_toolkit = 3.0.0

$ clickhouse-cli
Error:

Traceback (most recent call last):
  File "/opt/conda/envs/sci-tools/bin/clickhouse-cli", line 6, in <module>
    from clickhouse_cli.cli import run_cli
  File "/opt/conda/envs/sci-tools/lib/python3.7/site-packages/clickhouse_cli/cli.py", line 21, in <module>
    from prompt_toolkit.eventloop.defaults import use_asyncio_event_loop
ModuleNotFoundError: No module named 'prompt_toolkit.eventloop.defaults

unexpected escape sequences on CSV export

Seems like the client adds some strange escape characters on top when exporting a table as CSV.

Example command: clickhouse-cli -q "SELECT * FROM table1 FORMAT CSV" > table1.csv

Output:
screen shot 2017-11-08 at 16 15 03

windows support

Is a windows support planned? I tried to install it on win10 today. Although, I was able to install and open the program, a flood of escape sequences was returned - I am not sure but I think win10 does not support a escape-sequence based coloured output.

fix highlighting

Highlighting seems to be broken on some systems (I use a custom theme and it looks quite weird). How can I deactivate or fix this?
screen shot 2017-05-02 at 23 58 07

sqlparse==0.4.4 causes a ValueError

The version of sqlparse released a few days ago seems to cause clickhouse-cli (0.3.8) to not start, giving a ValueErrror.

Full traceback:

  File "/home/jrolfe/.pyenv/versions/3.11.3/bin/clickhouse-cli", line 33, in <module>
    sys.exit(load_entry_point('clickhouse-cli==0.3.8', 'console_scripts', 'clickhouse-cli')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/clickhouse_cli/cli.py", line 580, in run_cli
    cli.run(query, data_input)
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/clickhouse_cli/cli.py", line 201, in run
    if not self.connect():
           ^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/clickhouse_cli/cli.py", line 114, in connect
    response = self.client.query('SELECT version();', fmt='TabSeparated')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/clickhouse_cli/clickhouse/client.py", line 156, in query
    query = sqlparse.format(query, strip_comments=True).rstrip(';')
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/sqlparse/__init__.py", line 59, in format
    return ''.join(stack.run(sql, encoding))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/sqlparse/engine/filter_stack.py", line 26, in run
    stream = lexer.tokenize(sql, encoding)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/sqlparse/lexer.py", line 155, in tokenize
    return Lexer.get_default_instance().get_tokens(sql, encoding)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/sqlparse/lexer.py", line 52, in get_default_instance
    cls._default_intance.default_initialization()
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/sqlparse/lexer.py", line 59, in default_initialization
    self.set_SQL_REGEX(keywords.SQL_REGEX)
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/sqlparse/lexer.py", line 78, in set_SQL_REGEX
    self._SQL_REGEX = [
                      ^
  File "/home/jrolfe/.pyenv/versions/3.11.3/lib/python3.11/site-packages/sqlparse/lexer.py", line 80, in <listcomp>
    for rx, tt in SQL_REGEX
        ^^^^^^
ValueError: too many values to unpack (expected 2)

Easily solved with a pip install sqlparse==0.4.3, but probably deserves a proper fix.

Retry functionality

Is it possible to set a retry counter (incl. sleep)? I have to import a few hundred gigs, but I receive a timeout exception from time to time, which makes it really difficult to do it.

[Feature Request] parallel bulk file import

I often have to import "huge" files and it is suggested to import such files in parallel. This can be "easily" done using tools like GNU parallel, but in most cases it is not so easy to do it right (for example GNUs parallel's --pipe option should not be used, because it is slow; someone should use pigz instead of gzip, which does not use multiple cores).

So I would really really like to see an new option to simply import large files and the program automatically uses multiple connections or even uncompresses files on the fly. This would be also a benefit for all platforms that support python, but not GNU parallel.

cli interface chars

now cli interface adds ' ] ' to begin of wrapped line, which cause problems when you try to copy, part of query, is it possible to make it like ordinary *nix command line, without additional chars ?
image

multiline not work

$ python --version
Python 3.6.5 :: Anaconda, Inc.

reproduce:
$ clickhouse-cli --multiline -P
clickhouse-cli version: 0.3.2
Connecting to 127.0.0.1:8123

:) create table test (

create table test (

Query:
create table test (

Received exception from server:
Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 21 (end of query): . Expected one of: columns or indices declaration list, column declaration, INDEX, identifier, list of elements, column or index declaration (version 19.13.2.19 (official build))

Elapsed: 0.003 sec.

[Feature request] External Data for Query Processing

Hi,
When using the HTTP interface, we can use external data for the processing eg.

$ cat /etc/passwd | sed 's/:/\t/g' > passwd.tsv

$ curl -F '[email protected];' 'http://localhost:8123/?query=SELECT+shell,+count()+AS+c+FROM+passwd+GROUP+BY+shell+ORDER+BY+c+DESC&passwd_structure=login+String,+unused+String,+uid+UInt16,+gid+UInt16,+comment+String,+home+String,+shell+String'
/bin/sh 20
/bin/false      5
/bin/bash       4
/usr/sbin/nologin       1
/bin/sync       1

But i don't know how to do it inclickhouse-cli, could you please help to support this feature?

output buffering

Awesome work!
I am trying to export some data (10sGB) from clickhouse on a node with limited DRAM, clickhouse-cli buffers the data before output and uses too much DRAM (and OOMed), is it possible to allow efficient data export? Thank you!

[Feature Request]: chdb support

Hello @hatarist 👋 I'm one of the maintainers at chdb (in-process OLAP engine based on ClickHouse) and found your amazing unofficial CLI while looking for inspiration to extend our prompt_toolkit basic tool. Would you perhaps be open and/or interested in working together on a "chdb" mode for your CLI to execute ClickHouse functions locally, or perhaps cooperating to find a way to extend the scope of your great work for the chdb community? Thanks in advance and great project!

[Bugs] can not use settings arguments in v 0.3.4

clickhouse-cli --user=$USENAME--arg-password=$PASSWORD  --host=https://ch.company.com --port 443
 -s 'max_memory_usage=1000&enable_http_compression=1'
clickhouse-cli version: 0.3.4
Connecting to ch.company.com:443

Error:
Code: 113, e.displayText() = DB::Exception: There is no session (version 19.13.4.32)

[Feature request] Set decimal precision for outputs

Firstly, my thanks for the best clickhouse command line client out there. Its syntax highlighting, auto-completes and multi-line editing make things so much easier than the clickhouse-client.

One of the main pain points in current output formats is arbitrary long floats, which makes the column width way larger than they should be and much less readable.
An option to set precision to fix number of decimals will be really very big improvement in my opinion. Something similar to numpy's set_printoptions(precision=4) .

Is it something that can be implemented easily in the current flow ?
Will be happy to help out if its something that won't need a lot of refactoring.

RuntimeError: generator raised StopIteration

 :) create table twocol (call String, id String, value Int64) ENGINE = MergeTree() ORDER by (call, id)

create table twocol (call String, id String, value Int64) ENGINE = MergeTree()
 ORDER by (call, id)


Ok. 0 rows in set. Elapsed: 0.015 sec. Processed: 0 rows, 0.0B (0 rows/s, 0.0B/s)

 :) insert into twocol (call, id, value) values ('c1', 'a', 4), ('c1', 'b', 3)
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/parseutils/tables.py", line 38, in extract_from_part
    raise StopIteration
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/.local/bin/clickhouse-cli", line 8, in <module>
    sys.exit(run_cli())
  File "/home/user/.local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/user/.local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/.local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/cli.py", line 580, in run_cli
    cli.run(query, data_input)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/cli.py", line 291, in run
    cli_input = self.session.prompt()
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1034, in prompt
    return self.app.run(
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/application/application.py", line 937, in run
    return loop.run_until_complete(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/application/application.py", line 856, in run_async
    return await _run_async2()
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/application/application.py", line 832, in _run_async2
    await self.cancel_and_wait_for_background_tasks()
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/application/application.py", line 1051, in cancel_and_wait_for_background_tasks
    await task
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/buffer.py", line 1912, in new_coroutine
    await coroutine(*a, **kw)
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/buffer.py", line 1739, in async_completer
    async for completion in self.completer.get_completions_async(
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/completion/base.py", line 271, in get_completions_async
    async for completion in completer.get_completions_async(
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/completion/base.py", line 227, in get_completions_async
    async for completion in generator_to_async_generator(
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/eventloop/async_generator.py", line 74, in generator_to_async_generator
    await runner_f
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/eventloop/async_generator.py", line 43, in runner
    for item in get_iterable():
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/completion/base.py", line 228, in <lambda>
    lambda: self.completer.get_completions(document, complete_event)
  File "/home/user/.local/lib/python3.10/site-packages/prompt_toolkit/completion/base.py", line 264, in get_completions
    return completer.get_completions(document, complete_event)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/completer.py", line 424, in get_completions
    suggestions = suggest_type(document.text, document.text_before_cursor)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/parseutils/helpers.py", line 195, in suggest_type
    return suggest_based_on_last_token(stmt.last_token, stmt)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/parseutils/helpers.py", line 345, in suggest_based_on_last_token
    return (Column(table_refs=stmt.get_tables('insert')),)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/parseutils/helpers.py", line 145, in get_tables
    tables = extract_tables(self.full_text if scope == 'full' else self.text_before_cursor)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/parseutils/tables.py", line 145, in extract_tables
    return tuple(i for i in identifiers if i.name)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/parseutils/tables.py", line 145, in <genexpr>
    return tuple(i for i in identifiers if i.name)
  File "/home/user/.local/lib/python3.10/site-packages/clickhouse_cli/ui/parseutils/tables.py", line 95, in extract_table_identifiers
    for item in token_stream:
RuntimeError: generator raised StopIteration

1 → clickhouse-cli -B user -d table
clickhouse-cli version: 0.3.8
Connecting to 127.0.0.1:8123
Connected to ClickHouse server v23.1.2.

 :)

Feature Request: optionally use POST for select queries

In some cases, we want to send huge queries over to Clickhouse, using GET method would return "414 request-uri too large". It would be great if the method can be configurable somehow (via CLI flag or the configuration file), so we can use POST for these queries.

setting highlight to false does not disable colors

Hello,

I have set highlight=False in ~/.clickhouse-cli.rc and still have colored output. I have also tried highlight=false and highlight_output=False. No effect.

I'd like to disable the colors because they are hard to read on a light background. Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.