ajkerrigan / visidata-plugins Goto Github PK

View Code? Open in Web Editor NEW

47.0 47.0 3.0 227 KB

Custom plugins for https://github.com/saulpw/visidata/

License: MIT License

Python 100.00%

visidata-plugins's Introduction

Hi there 👋

Other places I might be:

visidata-plugins's People

Contributors

Stargazers

Watchers

Forkers

poeblu anjakefala robber-m

visidata-plugins's Issues

Add some demo animations

It would be helpful to include some terminal animations (with keystrokes displayed). The README is probably not super useful as is.

Support object versions

s3fs has support for handling versioned buckets, but that's not currently exposed in vds3.

The first step here would probably be adding a command/keybinding to display versions of selected objects in a Dir or Glob sheet.

It would also be possible to toggle including versions in a Dir/Glob sheet by default, but that seems riskier and less clearly desirable.

[feature request] Split navigation should be smart about frequency tables.

One window could display the frequency table. As you navigate the SheetFreqTable in one window, the other window shows the rows from the original table matching the cursorRow.

Doubled rows and columns for vds3 bucket listings with VisiData 2.3

With VisiData 2.2 and earlier, vd 's3://ajtest-vd' produces:

 name     | type | size#| modtime            @║
 test.txt | file |    3 | 2021-04-07 15:23:17 ║

But with VisiData 2.3 I get:

 name      | type | size#| modtime           @| name      | type | size#| modtime            @║
 test.txt  | file |    3 | 2021-04-07 15:23:17| test.txt  | file |    3 | 2021-04-07 15:23:17 ║

With larger buckets I get doubled rows also. Need to investigate more, I suspect I was making assumptions about vd internals that I shouldn't!

Clean up split_navigation for inclusion in upstream plugins.jsonl

At a minimum...

Go through the contributor plugin guide

But also maybe...

Rename it? parent_navigation might be more fitting, given that the use case motivating upstream inclusion has nothing to do with split views
Remove keybindings - define commands and document options, but let users pick their own keyboard shortcuts

Feature Request: kvpairs should look for more key/value names

Feature request add a way to add additional names as key or value names.

Currently the values are 'Key' and 'Value'. I have data that uses 'name' and 'value'.
I think this was in a har file.

This seems to be a common pattern with how data is encoded, it seems like it would be
useful if to look for multiple possible names, and ignore case.

Also, I don't know how your code handles a collision where the same key is listed more than once.

Open parquet file with `vds3` fails

When I try to open parquet files from AWS S3, I always get following error message:

ArrowInvalid: Expected a local filesystem path, got a URI: 's3:/<bucket-name>/<filename>.parquet'

Tried with VisiData v2.10.1 and 2.11 along with different pyarrow versions (10, 11) and vds3 v0.8 on macOS Ventura 13.1 with Python 3.9.16.

All other common file types (CSV, JSON, etc.) are working fine.

AttributeError: type object 'Column' has no attribute 'api'

Hi, thanks for your plugins!
I was trying to install vpager, but I'm receiving following error:

# ~/.visidata/plugins/vpager.py #8 
AttributeError: visidata "type object Column has no attribute api"

I am using official version from PyPI for Ubuntu:

$ vd  --version
saul.pw/VisiData v1.5.2

For which version of visidata is your plugin adjusted?
Do you know if there any docs about visidata API for writing plugins?

Support progressive loading for glob matches?

When opening a glob-matching path in a bucket with many objects, there can be a noticeable lag with no visual feedback. Look into progressively loading matches, or at least providing more of a visual indicator.

FEATURE: Add header to slave record in master/slave view

Often the data won't make sense without a header prefix. You could use the 'dataframe.T' function of Pandas to quickly produce a nicely formatted output if you don't want to write the formatting yourself.

InvalidRange or EOFError opening previous versions of gzipped files

With version support enabled, I've found an issue that seems to only affect some previous versions of gzipped objects. I can open current versions fine, and previous versions of uncompressed files fine. But for some previous versions of gzipped files, I consistently get either an InvalidRange error from botocore or an EOFError from gzip (full tracebacks below).

These issues are reproducible outside VisiData, so it seems like something related to either s3fs or fsspec. Possibly related to fsspec/s3fs#225? I can work around some of the errors by setting cache_type to 'none' and/or messing with block_size during the s3fs open() call, but it's not 100%. Putting a flag here so I can dig more, and so I have a reference in case anyone else bumps into the same trouble.

Version info:

Python: 3.8.5
s3fs: 0.5.1
fsspec: 0.8.3

InvalidRange traceback:

Traceback (most recent call last):
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 207, in _call_s3
    return await method(**additional_kwargs)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/aiobotocore/client.py", line 151, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidRange) when calling the GetObject operation: The requested range is not satisfiable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/threads.py", line 207, in _toplevelTryFunc
    t.status = func(*args, **kwargs)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/sheets.py", line 267, in reload
    for r in self.iterload():
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/loaders/json.py", line 28, in iterload
    ret = json.load(fp, object_pairs_hook=OrderedDict)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 292, in read
    return self._buffer.read(size)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 485, in read
    buf = self._fp.read(io.DEFAULT_BUFFER_SIZE)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 87, in read
    return self.file.read(size)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/spec.py", line 1388, in read
    out = self.cache._fetch(self.loc, self.loc + length)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/caching.py", line 350, in _fetch
    new = self.fetcher(self.end, bend)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 1425, in _fetch_range
    return _fetch_range(self.fs, self.bucket, self.key, self.version_id,
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 1540, in _fetch_range
    resp = fs.call_s3(fs.s3.get_object, Bucket=bucket, Key=key,
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 118, in wrapper
    return maybe_sync(func, self, *args, **kwargs)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 97, in maybe_sync
    return sync(loop, func, *args, **kwargs)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 68, in sync
    raise exc.with_traceback(tb)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 52, in f
    result[0] = await future
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 225, in _call_s3
    raise translate_boto_error(err) from err
OSError: [Errno 22] The requested range is not satisfiable

EOFError traceback:

Traceback (most recent call last):
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/threads.py", line 207, in _toplevelTryFunc
    t.status = func(*args, **kwargs)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/sheets.py", line 267, in reload
    for r in self.iterload():
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/loaders/json.py", line 28, in iterload
    ret = json.load(fp, object_pairs_hook=OrderedDict)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 292, in read
    return self._buffer.read(size)
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 470, in read
    self._read_eof()
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 514, in _read_eof
    crc32, isize = struct.unpack("<II", self._read_exact(8))
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 416, in _read_exact
    raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached

vdrepl explodes if invoked while a stdin-sourced sheet is open

If we pipe data into VisiData and then try to invoke vdrepl's open-repl command, the terminal has a seizure involving lots of these errors:

Unhandled exception in event loop:
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 669, in read_from_input
    f.set_exception(EOFError)

Exception invalid state
^[[56;1RPress ENTER to continue...

Steps to reproduce

Pipe some text into VisiData:

echo "test" | vd

Hit ` or run the open-repl command.

vds3 broken in VisiData 2.10.1

See saulpw/visidata#1534 - this looks like a vds3 problem not a vd problem!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.