Giter VIP home page Giter VIP logo

jtbl's Introduction

Tests Pypi

jtbl

A simple cli tool to print JSON data as a table in the terminal.

jtbl accepts piped JSON data from stdin and outputs a text table representation to stdout. e.g:

$ cat cities.json | jtbl
  LatD    LatM    LatS  NS      LonD    LonM    LonS  EW    City               State
------  ------  ------  ----  ------  ------  ------  ----  -----------------  -------
    41       5      59  N         80      39       0  W     Youngstown         OH
    42      52      48  N         97      23      23  W     Yankton            SD
    46      35      59  N        120      30      36  W     Yakima             WA
    42      16      12  N         71      48       0  W     Worcester          MA
    43      37      48  N         89      46      11  W     Wisconsin Dells    WI
    36       5      59  N         80      15       0  W     Winston-Salem      NC
    49      52      48  N         97       9       0  W     Winnipeg           MB

jtbl expects a JSON array of JSON objects or JSON Lines.

It can be useful to JSONify command line output with jc, filter through a tool like jq, and present in jtbl:

$ jc ifconfig | jq -c '.[] | {name, type, ipv4_addr, ipv4_mask}'| jtbl
name     type            ipv4_addr       ipv4_mask
-------  --------------  --------------  -------------
docker0  Ethernet        172.17.0.1      255.255.0.0
ens33    Ethernet        192.168.71.146  255.255.255.0
lo       Local Loopback  127.0.0.1       255.0.0.0

Installation

You can install jtbl via pip, via OS Package Repositories, MSI installer for Windows, or by downloading the correct binary for your architecture and running it anywhere on your filesystem.

Pip (macOS, linux, unix, Windows)

For the most up-to-date version and the most cross-platform option, use pip or pip3 to download and install jtbl directly from PyPi:

Pypi

pip3 install jtbl

OS Packages

Packaging status

See Releases on Github for MSI packages and binaries.

Usage

Just pipe JSON data to jtbl. (e.g. cat a JSON file, jc, jq, aws cli, kubectl, etc.)

$ <JSON Data> | jtbl [OPTIONS]

Options

  • --cols=n manually configure the terminal width
  • -c, --csv CSV table output
  • -d, --dokuwiki Dokuwiki table output
  • -f, --fancy fancy table output
  • -h, --help prints help information
  • -H, --html HTML table output
  • -m, --markdown markdown table output
  • -n, --no-wrap no data wrapping if too long for the terminal width (overrides --cols and -t)
  • -q, --quiet don't print error messages to STDERR
  • -r, --rotate rotate the data (each row turns into a table of key/value pairs)
  • -t, --truncate truncate data instead of wrapping if too long for the terminal width
  • -v, --version prints version information

Compatible JSON Formats

jtbl works best with a shallow array of JSON objects. Each object should have a few elements that will be turned into table columns. Fortunately, this is how many APIs present their data.

JSON Array Example

[
  {
    "unit": "proc-sys-fs-binfmt_misc.automount",
    "load": "loaded",
    "active": "active",
    "sub": "waiting",
    "description": "Arbitrary Executable File Formats File System Automount Point"
  },
  {
    "unit": "sys-devices-pci0000:00-0000:00:07.1-ata2-host2-target2:0:0-2:0:0:0-block-sr0.device",
    "load": "loaded",
    "active": "active",
    "sub": "plugged",
    "description": "VMware_Virtual_IDE_CDROM_Drive"
  },
  ...
]

jtbl can also work with JSON Lines format with similar features.

JSON Lines Example

{"name": "docker0", type": "Ethernet", "ipv4_addr": "172.17.0.1", "ipv4_mask": "255.255.0.0"}
{"name": "ens33", "type": "Ethernet", "ipv4_addr": "192.168.71.146", "ipv4_mask": "255.255.255.0"}
{"name": "lo", "type": "Local Loopback", "ipv4_addr": "127.0.0.1", "ipv4_mask": "255.0.0.0"}
...

Filtering the JSON Input

If there are too many elements, or the data in the elements are too large, the table may not fit in the terminal screen. In this case you can use a JSON filter like jq or jello to send jtbl only the elements you are interested in:

jq Array Method

The following example uses jq to filter and format the filtered elements into a proper JSON array.

$ cat /etc/passwd | jc --passwd | jq '[.[] | {username, shell}]'
[
  {
    "username": "root",
    "shell": "/bin/bash"
  },
  {
    "username": "bin",
    "shell": "/sbin/nologin"
  },
  {
    "username": "daemon",
    "shell": "/sbin/nologin"
  },
  ...
]

(Notice the square brackets around the filter)

jq Slurp Method

The following example uses jq to filter and 'slurp' the filtered elements into a proper JSON array.

$ cat /etc/passwd | jc --passwd | jq '.[] | {username, shell}' | jq -s
[
  {
    "username": "root",
    "shell": "/bin/bash"
  },
  {
    "username": "bin",
    "shell": "/sbin/nologin"
  },
  {
    "username": "daemon",
    "shell": "/sbin/nologin"
  },
  ...
]

(Notice the jq -s at the end)

jq JSON Lines Method

The following example will send the data in JSON Lines format, which jtbl can understand:

$ cat /etc/passwd | jc --passwd | jq -c '.[] | {username, shell}'
{"username":"root","shell":"/bin/bash"}
{"username":"bin","shell":"/sbin/nologin"}
{"username":"daemon","shell":"/sbin/nologin"}
...

(Notice the -c option being used)

jello List Comprehension Method

If you prefer python list and dictionary syntax to filter JSON data, you can use jello:

$ cat /etc/passwd | jc --passwd | jello '[{"username": x.username, "shell": x.shell} for x in _]'
[
  {
    "username": "root",
    "shell": "/bin/bash"
  },
  {
    "username": "bin",
    "shell": "/sbin/nologin"
  },
  {
    "username": "daemon",
    "shell": "/sbin/nologin"
  },
  ...
]

When piping any of these to jtbl you get the following result:

$ cat /etc/passwd | jc --passwd | jello '[{"username": x.username, "shell": x.shell} for x in _]' | jtbl
username         shell
---------------  --------------
root             /bin/bash
bin              /sbin/nologin
daemon           /sbin/nologin
...

Working with Deeper JSON Structures

jtbl will happily dump deeply nested JSON structures into a table, but usually this is not what you are looking for.

$ jc dig www.cnn.com | jtbl
╒══════════╤══════════╤═══════╤════════════╤═════════════╤══════════════╤══════════════╤══════════════╤══════════════╤════════════╤════════════╤══════════════╤════════════╤════════════╤════════╤══════════════╤══════════════╕
│ opcode   │ status   │    id │ flags      │   query_num │   answer_num │   authority_ │   additional │ opt_pseudo   │ question   │ answer     │   query_time │ server     │ when       │   rcvd │   when_epoch │ when_epoch   │
│          │          │       │            │             │              │          num │         _num │ section      │            │            │              │            │            │        │              │ _utc         │
╞══════════╪══════════╪═══════╪════════════╪═════════════╪══════════════╪══════════════╪══════════════╪══════════════╪════════════╪════════════╪══════════════╪════════════╪════════════╪════════╪══════════════╪══════════════╡
│ QUERY    │ NOERROR  │ 36494 │ ['qr', 'rd │           1 │            4 │            0 │            1 │ {'edns': {   │ {'name': ' │ [{'name':  │           47 │ 2600:1700: │ Wed Dec 22 │    100 │   1640200072 │              │
│          │          │       │ ', 'ra']   │             │              │              │              │ 'version':   │ cnn.com.', │ 'cnn.com.' │              │ bab0:d40:: │  11:07:52  │        │              │              │
│          │          │       │            │             │              │              │              │  0, 'flags   │  'class':  │ , 'class': │              │ 1#53(2600: │ PST 2021   │        │              │              │
│          │          │       │            │             │              │              │              │ ': [], 'ud   │ 'IN', 'typ │  'IN', 'ty │              │ 1700:bab0: │            │        │              │              │
│          │          │       │            │             │              │              │              │ p': 4096}}   │ e': 'A'}   │ pe': 'A',  │              │ d40::1)    │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │ 'ttl': 60, │              │            │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │  'data': ' │              │            │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │ 151.101.12 │              │            │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │ 9.67'}, {' │              │            │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │ name': 'cn │              │            │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │ n.com.', ' │              │            │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │ class': 'I │              │            │            │        │              │              │
│          │          │       │            │             │              │              │              │              │            │ ...        │              │            │            │        │              │              │
╘══════════╧══════════╧═══════╧════════════╧═════════════╧══════════════╧══════════════╧══════════════╧══════════════╧════════════╧════════════╧══════════════╧════════════╧════════════╧════════╧══════════════╧══════════════╛

Diving Deeper into the JSON with jq or jello:

To get to the data you are interested in you can use a JSON filter like jq or jello to dive deeper.

Using jq:

$ jc dig www.cnn.com | jq '.[0].answer'

or with jello:

$ jc dig www.cnn.com | jello '_[0].answer'

Both will produce the following output:

[
  {
    "name": "www.cnn.com.",
    "class": "IN",
    "type": "CNAME",
    "ttl": 90,
    "data": "turner-tls.map.fastly.net."
  },
  {
    "name": "turner-tls.map.fastly.net.",
    "class": "IN",
    "type": "A",
    "ttl": 20,
    "data": "151.101.1.67"
  }
  ...
]

This will produce the following table in jtbl

$ jc dig www.cnn.com | jello '_[0].answer' | jtbl
name                        class    type      ttl  data
--------------------------  -------  ------  -----  --------------------------
www.cnn.com.                IN       CNAME      11  turner-tls.map.fastly.net.
turner-tls.map.fastly.net.  IN       A          23  151.101.129.67
turner-tls.map.fastly.net.  IN       A          23  151.101.1.67
turner-tls.map.fastly.net.  IN       A          23  151.101.65.67
turner-tls.map.fastly.net.  IN       A          23  151.101.193.67

Column Width

jtbl will attempt to shrink columns to a sane size if it detects the output is wider than the terminal width. The --cols option will override the automatic terminal width detection.

You can use the -t option to truncate the rows instead of wrapping when the terminal width is too small for all of the data.

The -n option disables wrapping and overrides the --cols and -t options.

This can be useful to present a nicely non-wrapped table of infinite width in combination with less -S:

$ jc ps aux | jtbl -n | less -S
user                  pid        vsz     rss  tt    stat    started    time       command
------------------  -----  ---------  ------  ----  ------  ---------  ---------  ---------------------------------------------------
joeuser             34029    4277364   24800  s000  S+      9:28AM     0:00.27    /usr/local/Cellar/python/3.7.6_1/Frameworks/Python....
joeuser             34030    4283136   17104  s000  S+      9:28AM     0:00.20    /usr/local/Cellar/python/3.7.6_1/Frameworks/Python....
joeuser               481    5728568  189328        S       17Apr20    21:46.52   /Applications/Utilities/Terminal.app/Contents/MacOS...
joeuser             45827    6089084  693768        S       Wed01PM    84:54.87   /Applications/Microsoft Teams.app/Contents/Framewor...
joeuser              1493    9338824  911600        S       17Apr20    143:27.08  /Applications/Microsoft Outlook.app/Contents/MacOS/...
joeuser             45822    5851524  163840        S       Wed01PM    38:48.83   /Applications/Microsoft Teams.app/Contents/MacOS/Te...

jtbl's People

Contributors

kellyjonbrazil avatar samsu-f avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

jtbl's Issues

Preserve field order when resizing or truncating

In general, if jtbl is not resizing or truncating fields it will preserve the order of fields in the column output. Field order has no significance in JSON, so this is best effort. That is, there may be records that don't include all of the fields or the fields may be in a different order, so there is not an obvious way to do this.

Today, if jtbl encounters a new field that didn't exist in previous records, it will place that column at the end of the table, even when not resizing or truncating fields.

When resizing or truncating, jtbl checks all of the rows (including the header) and finds the largest cell for each column and basically truncates/resizes them and prints them out in the order of which column has the least-longest cell to the column with the largest cell.

It could be possible to scan all fields that exist in all of the records of the JSON document and try to preserve the column order in some way. It might also be possible to have built-in options to sort the columns alphabetically, by size, user-defined ordering, etc.

Multiline for multiple values within square brackets

Thanks for this tool! Would there be a way to make jtbl recognize that values separated by commas within square brackets should be on new lines, with json files of the following structure?

{
  "Date": [
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00",
    "2021-08-24 21:00:00"
  ],
  "Location": [
    "Indonesia - TERNATE/BABULLAH",
    "Bolivia - TRINIDAD",
    "United States - PENSACOLA, FL",
    "Russian Federation - IVDEL'",
    "India - GULBARGA",
    "Honduras - CHOLUTECA",
    "United States - JACKSONVILLE/INTNL.,  FL.",
    "Honduras - LA MESA (SAN PEDRO SULA)",
    "Russian Federation - JARCEVO",
    "Russian Federation - VEL' MO",
    "Azerbaijan - GUBA",
    "Niger - NIAMEY-AERO",
    "Vietnam - PHU LIEN"
  ]
}

Currently, when I pipe that into jtbl, it just shows a table with two columns and a single row containing all values separated by commas.

Supress scientific notation in output

Is there a way to suppress the scientific notation that sometimes appears in the output?

I'd like jtbl to use the same notation as the source.

Here's a simple example of the surprising behavior:

$ jq -n '{"a": 1000000, "b": 1000000.1}' | jtbl -m
|       a |     b |
|---------|-------|
| 1000000 | 1e+06 |

Is there something I'm missing?

Add option --sort/--order

Hi. Can you add an option to sort columns in a table?

Now:

curl --silent http://<my-repo-in-intranet>/apt/debian/dists/stable/main/binary-amd64/Packages | /usr/local/bin/jc --pkg-index-deb -p | /usr/local/bin/jp "sort_by([], &package)[].{package: package, version: version, homepage: homepage}"  | /usr/local/bin/jtbl --markdown
| homepage                                       | package      | version    |
|------------------------------------------------|--------------|------------|
| https://schollz.com/software/croc/             | croc         | 9.6.6      |
| https://github.com/jgraph/drawio               | draw.io      | 22.1.2     |
|                                                | ifcplugin    | 3.1.1.0    |
| https://github.com/kellyjonbrazil/jc           | jc           | 1.24.0-1   |
| https://github.com/kellyjonbrazil/jtbl         | jtbl         | 1.6.0-1    |
|                                                | naps2        | 7.2.1      |
|                                                | pandoc       | 3.1.11-1   |
| https://rclone.org                             | rclone       | 1.65.0     |
|                                                | scenebuilder | 21.0.0     |
| https://github.com/ansible-semaphore/semaphore | semaphore    | 2.9.37     |
| www.hamrick.com                                | vuescan      | 9.8.22.0-0 |

How i want:

jtbl --markdown --order=package,version,homepage
| package       | version       | homepage                                       |
|---------------|---------------|------------------------------------------------|
| croc          | 9.6.6         | https://schollz.com/software/croc/             |
| draw.io       | 22.1.2        | https://github.com/jgraph/drawio               |
| ifcplugin     | 3.1.1.0       |                                                |
| jc            | 1.24.0-1      | https://github.com/kellyjonbrazil/jc           |
| jtbl          | 1.6.0-1       | https://github.com/kellyjonbrazil/jtbl         |
| naps2         | 7.2.1         |                                                |
| pandoc        | 3.1.11-1      |                                                |
| rclone        | 1.65.0        | https://rclone.org                             |
| scenebuilder  | 21.0.0        |                                                |
| semaphore     | 2.9.37        | https://github.com/ansible-semaphore/semaphore |
| vuescan       | 9.8.22.0-0    | www.hamrick.com                                |

Crash if wrap_width == 0

  File "/home/kbrazil/.local/lib/python3.6/site-packages/jtbl/cli.py", line 68, in main
    entry[k] = '\n'.join([str(v)[i:i + wrap_width] for i in range(0, len(str(v)), wrap_width)])
ValueError: range() arg 3 must not be zero

This happened on a serial console - maybe shutil wasn't able to determine terminal size. Can probably fix by setting a default value to wrap_width in case shutil fails.

Add changelog to PyPI package

It would be great if the changelog was included in the PyPI package. It's useful for when including documentation in distribution packages.

Be able to accept an empty array as input

Description

I notice that jtbl throws an error when it receives an empty list as input ([]).

This feels like a bug since en empty list is considered valid json.

Version

jtbl:   version 1.5.1

Steps to recreate

echo "[]" | jtbl

Expected Output

An empty response (similar to what is returned when you send an empty object {}.

Observed Output

Traceback (most recent call last):
  File "/opt/homebrew/bin/jtbl", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/jtbl/cli.py", line 339, in main
    succeeded, json_data = check_data(json_data, columns=columns)
    ^^^^^^^^^^^^^^^^^^^^

Catch CTRL-C

Gracefully exit on CTRL-C instead of generating a stack trace.

Improve auto-scaling to fit terminal width

Right now auto-scaling the column width to fit inside the terminal is pretty rudimentary.

I'm adding up all of the longest items, whether the header or the data values, seeing if they are larger than the terminal width and then setting a maximum column width that should allow the table to fit if all columns are the same size.

Since some columns are smaller than the evenly divided column length this doesn't always fully fill the terminal width.

There is also no logic to do something different if the table is clearly too wide to ever fit, no matter how small the columns get. Maybe need to remove columns that would fall off the screen.

Wondering if this problem has already been solved somehow. I'll continue to make incremental improvements, but if someone else knows how to solve this I'm open to ideas.

Add long options format

Hi!

Can you add a long key format to use in scripts?

Now:

jtbl:   Converts JSON and JSON Lines to a table

Usage:  <JSON Data> | jtbl [OPTIONS]

        --cols=n   manually configure the terminal width
        -c         CSV table output
        -f         fancy table output
        -h         help
        -H         HTML table output
        -m         markdown table output
        -n         no-wrap - do not try to wrap if too wide for the terminal
        -q         quiet - don't print error messages
        -r         rotate table output
        -t         truncate data if too wide for the terminal
        -v         version info

How i want:

jtbl:   Converts JSON and JSON Lines to a table

Usage:  <JSON Data> | jtbl [OPTIONS]

        --cols=n               manually configure the terminal width
        -c, --csv              CSV table output
        -f, --fancy            fancy table output
        -h, --help             help
        -H, --html             HTML table output
        -m, --markdown         markdown table output
        -n, --no-wrap          no-wrap - do not try to wrap if too wide for the terminal
        -q, --quiet            quiet - don't print error messages
        -r, --rotate           rotate table output
        -t, --truncate         truncate data if too wide for the terminal
        -v, --version          version info

Have a `--quiet-already` when no data is passed

jtbl:   Missing piped data

Would be nice if we can ignore this with a --quiet or -q So it doesn't output this. Sometimes our elaborate JQ pipelines result into 0 results and then just don't make a table.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.