jsanz / carto_cli Goto Github PK

Need to add a CONTRIBUTORS file to the repo with everyone helping here.

Current query to get indexes is not getting those that are based on expressions.

Related with #8, it'd be interesting to upgrade to click 7. The changelo is really long but hopefully the number of breaking changes should be short.

When running carto_dataset describe with a non-existent table name, the output is still provided (although the tables are emtpy).

Example

carto_dataset describe this_table_does_not_exist

Output

# Report for table: this_table_does_not_exist

## Postgres Metadata

+----------+-------+
| Property | Value |
+----------+-------+
+----------+-------+

## CARTO Metadata

Table is not CartoDBfied

## Schema

+-----------+------+
| attribute | type |
+-----------+------+
+-----------+------+

## Indexes

+------------+-------------+------------+
| index_name | column_name | index_type |
+------------+-------------+------------+
+------------+-------------+------------+

## Triggers

+--------+
| tgname |
+--------+
+--------+

This is due to the fact that no Exception is ever raised because no iteration is performed over the empty result['rows'] array.

Allow CRUD operations for API keys. AFAIK this is not supported at Python SDK level so it will have to interact directly with the Auth API

commands so catch quick if authorization is set up and report back properly.

Review this with my account

 ± carto_dataset list -f pretty
Usage: carto_dataset list [OPTIONS]

Error: Error retrieving the list of datasets: non-empty format string passed to object.__format__

Datasets should be ordered when listing for easy finding, related to #1

Following this work, support a carto_map command to download one or all maps, maybe implementing some kind of wildcards/search

• https://carto.com/developers/python-sdk/examples/#example-export-map
• https://gist.github.com/jsanz/01fadd668926916de8be5680a10a5fef

Many commands here (especially those that take a single argument) could benefit from a more UNIX-like approach, where arguments can be declared explicitly or passed in a pipe as stdin. For example, one could locate and kill queries in a single line:

carto_sql queries | jq '.[0].pid' | carto_sql kill

I tend to read a Batch query's status just after running it, in case it's failed. Instead of copying and pasting the job_id, one could just do:

carto_batch create "SELECT * FROM rivers LIMIT 10;" | carto_batch read

The approach is simple enough: check if arguments are passed and, if not, read from stdin. A nice way of avoiding having to wait for user input is detailed in this blogpost, which depends on checking whether it's an interactive session or not.

I gave this a small test on the later example and it works pretty well:

@click.command(help="Returns details about a job id (JSON)")
@click.argument('job_id', required=False)
@click.help_option('-h', '--help')
@click.pass_context
def read(ctx,job_id):
    '''
    Just returns the details of a job using as a JSON
    '''
    carto_obj = ctx.obj['carto']
    if not job_id:
        if not sys.stdin.isatty(): # Don't wait for input
            job_id = sys.stdin.read()
        else:
            ctx.fail(
                "A job ID is required. "
                "Either pass it explicitly or pipe into the command"
            )
    try:
        job_details = carto_obj.batch_check(job_id)
        click.echo(json.dumps(job_details))
    except Exception as e:
        ctx.fail(e)

@jsanz let me know if you think this can be an interesting feature and I'd be happy to provide a PR. I think this could be useful for single-argument commands, such as:

• carto_env load
• carto_env list
• carto_sql run
• carto_sql kill
• carto_batch create
• carto_batch read
• most carto_dataset commands

On-Premises installations use domainless url schema, review the way the URLs are used to be sure commands work in these cases.

Click has built-in support for simple Bash/ZSH completion of subcommands and arguments.

Activation is as simple as adding a couple of lines to .bashrc or .zshrc. In this case, one for each program:

eval "$(_CARTO_BASH_COMPLETE=source carto_bash)"
eval "$(_CARTO_DATASET_COMPLETE=source carto_dataset)"
eval "$(_CARTO_ENV_COMPLETE=source carto_env)"
eval "$(_CARTO_SQL_COMPLETE=source carto_sql)"

Some notes:

• .bashrc vs .bash_profile: Click needs those lines to be placed explicitly in .bashrc (used for interactive login shells), not .bash_profile. Some MAC users may only have .bash_profile (because MacOS always runs .bash_profile for new terminal windows) so .bashrc should be created if it does not exist.

• zsh completion: Click 6.x.x does not include ZSH completion in its docs (the latest stable release, 7.x, does, but carto-cli is currently running version 6.7). However, it works just fine, with a small caveat: the same line must be placed in .zshrc as in .bashrc, which uses source instead of source_zsh (at least on MacOS).

• Shell activation time: As stated in the Click docs, in order for this to work each shell has to invoke each application on startup, which can cause slowdowns (not that I've seen since all these applications are pretty lightweight). Anyhow, the option of including an Activation Script and then calling that from .bashrc/.zshrc is also feasible.

Activating this simple feature seems like pretty low-hanging fruit and I think that current and future users would appreciate it considerably, given the high amount of sub-commands and arguments for each application. I'd be happy to help with this if needed!