Other places I might be:
ajkerrigan / visidata-plugins Goto Github PK
View Code? Open in Web Editor NEWCustom plugins for https://github.com/saulpw/visidata/
License: MIT License
Custom plugins for https://github.com/saulpw/visidata/
License: MIT License
Other places I might be:
When opening a glob-matching path in a bucket with many objects, there can be a noticeable lag with no visual feedback. Look into progressively loading matches, or at least providing more of a visual indicator.
With version support enabled, I've found an issue that seems to only affect some previous versions of gzipped objects. I can open current versions fine, and previous versions of uncompressed files fine. But for some previous versions of gzipped files, I consistently get either an InvalidRange
error from botocore or an EOFError
from gzip (full tracebacks below).
These issues are reproducible outside VisiData, so it seems like something related to either s3fs or fsspec. Possibly related to fsspec/s3fs#225? I can work around some of the errors by setting cache_type
to 'none'
and/or messing with block_size
during the s3fs open()
call, but it's not 100%. Putting a flag here so I can dig more, and so I have a reference in case anyone else bumps into the same trouble.
Version info:
InvalidRange traceback:
Traceback (most recent call last):
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 207, in _call_s3
return await method(**additional_kwargs)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/aiobotocore/client.py", line 151, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidRange) when calling the GetObject operation: The requested range is not satisfiable
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/threads.py", line 207, in _toplevelTryFunc
t.status = func(*args, **kwargs)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/sheets.py", line 267, in reload
for r in self.iterload():
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/loaders/json.py", line 28, in iterload
ret = json.load(fp, object_pairs_hook=OrderedDict)
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 292, in read
return self._buffer.read(size)
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 485, in read
buf = self._fp.read(io.DEFAULT_BUFFER_SIZE)
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 87, in read
return self.file.read(size)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/spec.py", line 1388, in read
out = self.cache._fetch(self.loc, self.loc + length)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/caching.py", line 350, in _fetch
new = self.fetcher(self.end, bend)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 1425, in _fetch_range
return _fetch_range(self.fs, self.bucket, self.key, self.version_id,
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 1540, in _fetch_range
resp = fs.call_s3(fs.s3.get_object, Bucket=bucket, Key=key,
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 118, in wrapper
return maybe_sync(func, self, *args, **kwargs)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 97, in maybe_sync
return sync(loop, func, *args, **kwargs)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 68, in sync
raise exc.with_traceback(tb)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/fsspec/asyn.py", line 52, in f
result[0] = await future
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/s3fs/core.py", line 225, in _call_s3
raise translate_boto_error(err) from err
OSError: [Errno 22] The requested range is not satisfiable
EOFError traceback:
Traceback (most recent call last):
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/threads.py", line 207, in _toplevelTryFunc
t.status = func(*args, **kwargs)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/sheets.py", line 267, in reload
for r in self.iterload():
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/visidata/loaders/json.py", line 28, in iterload
ret = json.load(fp, object_pairs_hook=OrderedDict)
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 292, in read
return self._buffer.read(size)
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 470, in read
self._read_eof()
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 514, in _read_eof
crc32, isize = struct.unpack("<II", self._read_exact(8))
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/gzip.py", line 416, in _read_exact
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached
Hi, thanks for your plugins!
I was trying to install vpager, but I'm receiving following error:
# ~/.visidata/plugins/vpager.py #8
AttributeError: visidata "type object Column has no attribute api"
I am using official version from PyPI for Ubuntu:
$ vd --version
saul.pw/VisiData v1.5.2
For which version of visidata is your plugin adjusted?
Do you know if there any docs about visidata API for writing plugins?
With VisiData 2.2 and earlier, vd 's3://ajtest-vd'
produces:
name | type | size#| modtime @โ
test.txt | file | 3 | 2021-04-07 15:23:17 โ
But with VisiData 2.3 I get:
name | type | size#| modtime @| name | type | size#| modtime @โ
test.txt | file | 3 | 2021-04-07 15:23:17| test.txt | file | 3 | 2021-04-07 15:23:17 โ
With larger buckets I get doubled rows also. Need to investigate more, I suspect I was making assumptions about vd internals that I shouldn't!
Feature request add a way to add additional names as key or value names.
Currently the values are 'Key' and 'Value'. I have data that uses 'name' and 'value'.
I think this was in a har file.
This seems to be a common pattern with how data is encoded, it seems like it would be
useful if to look for multiple possible names, and ignore case.
Also, I don't know how your code handles a collision where the same key is listed more than once.
One window could display the frequency table. As you navigate the SheetFreqTable in one window, the other window shows the rows from the original table matching the cursorRow.
If we pipe data into VisiData and then try to invoke vdrepl's open-repl
command, the terminal has a seizure involving lots of these errors:
Unhandled exception in event loop:
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "/Users/akerrigan/.local/pipx/venvs/visidata/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 669, in read_from_input
f.set_exception(EOFError)
Exception invalid state
^[[56;1RPress ENTER to continue...
Steps to reproduce
echo "test" | vd
`
or run the open-repl
command.It would be helpful to include some terminal animations (with keystrokes displayed). The README is probably not super useful as is.
Often the data won't make sense without a header prefix. You could use the 'dataframe.T' function of Pandas to quickly produce a nicely formatted output if you don't want to write the formatting yourself.
When I try to open parquet files from AWS S3, I always get following error message:
ArrowInvalid: Expected a local filesystem path, got a URI: 's3:/<bucket-name>/<filename>.parquet'
Tried with VisiData v2.10.1 and 2.11 along with different pyarrow
versions (10, 11) and vds3
v0.8 on macOS Ventura 13.1 with Python 3.9.16.
All other common file types (CSV, JSON, etc.) are working fine.
s3fs has support for handling versioned buckets, but that's not currently exposed in vds3.
The first step here would probably be adding a command/keybinding to display versions of selected objects in a Dir or Glob sheet.
It would also be possible to toggle including versions in a Dir/Glob sheet by default, but that seems riskier and less clearly desirable.
See saulpw/visidata#1534 - this looks like a vds3 problem not a vd problem!
At a minimum...
But also maybe...
parent_navigation
might be more fitting, given that the use case motivating upstream inclusion has nothing to do with split viewsA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.