ibm / gwhisper Goto Github PK
View Code? Open in Web Editor NEWA gRPC CLI featuring reflection and tab-completion
License: Apache License 2.0
A gRPC CLI featuring reflection and tab-completion
License: Apache License 2.0
Describe the bug
build takes very long
To Reproduce
create a pull request to trigger CI build
Expected behavior
expect build to finish in less than 5 minutes
Additional context
This seems to be caused, as travis has to download, build and install grpc and protobuf for each build.
I think we should migrate to docker builds, in this case we could start with a docker-image with pre-installed gRPC.
Describe the bug
googletest installed on system
To Reproduce
make install
Expected behavior
googletest not installed, as it is only needed for unit tests and not for productive use
Describe the solution you'd like
It would be nice to have a way to persistently configure default behavior of the tool:
This configuration obviously needs to be stored on the file-system.
If no configuration is found, default values should be used.
Describe alternatives you've considered
specify all config options on command-line. But this is really annoying, as it forces the user to implement wrappers.
Describe why you think this feature is useful for gWhisper users
Especially for colors this is important, as the used color-set depends on the selected background color of the users terminal. Black on black or white on white does not make sense.
Describe the bug
I got the following compile error, as I try to compile gwisper with build.sh
:
[ 35%] Building CXX object third_party/gRPC_utils/CMakeFiles/reflection.dir/cli_call.cc.o
cd /home/chw2lr/tools/gWhisper/build/third_party/gRPC_utils && /usr/bin/c++ -I/home/chw2lr/tools/gWhisper/build -I/home/chw2lr/tools/gWhisper/build/src -I/home/chw2lr/tools/gWhisper -I/home/chw2lr/tools/gWhisper/src -I/home/chw2lr/tools/gWhisper/build/third_party/gRPC_utils -std=gnu++11 -o CMakeFiles/reflection.dir/cli_call.cc.o -c /home/chw2lr/tools/gWhisper/third_party/gRPC_utils/cli_call.cc
In file included from /home/chw2lr/tools/gWhisper/third_party/gRPC_utils/cli_call.cc:21:0:
/home/chw2lr/tools/gWhisper/third_party/gRPC_utils/cli_call.h:32:7: error: using typedef-name ‘grpc::ClientContext’ after ‘class’
class ClientContext;
^~~~~~~~~~~~~
In file included from /usr/local/include/grpcpp/impl/codegen/call_op_set.h:34:0,
from /usr/local/include/grpcpp/impl/codegen/server_context_impl.h:28,
from /usr/local/include/grpcpp/impl/codegen/server_context.h:22,
from /usr/local/include/grpcpp/impl/codegen/async_stream.h:25,
from /usr/local/include/grpcpp/support/async_stream.h:22,
from /usr/local/include/grpcpp/generic/generic_stub_impl.h:24,
from /usr/local/include/grpcpp/generic/generic_stub.h:22,
from /home/chw2lr/tools/gWhisper/third_party/gRPC_utils/cli_call.h:26,
from /home/chw2lr/tools/gWhisper/third_party/gRPC_utils/cli_call.cc:21:
/usr/local/include/grpcpp/impl/codegen/client_context.h:26:36: note: ‘grpc::ClientContext’ has a previous declaration here
typedef ::grpc_impl::ClientContext ClientContext;
^~~~~~~~~~~~~
third_party/gRPC_utils/CMakeFiles/reflection.dir/build.make:100: recipe for target 'third_party/gRPC_utils/CMakeFiles/reflection.dir/cli_call.cc.o' failed
make[2]: *** [third_party/gRPC_utils/CMakeFiles/reflection.dir/cli_call.cc.o] Error 1
To Reproduce
./build.sh
with
$ g++ --version
g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Expected behavior
It compiled.
Provide details about your environment
gwhisper --version
output: - cmake --version
cmake version 3.10.2
CMake suite maintained and supported by Kitware (kitware.com/cmake).
Describe the solution you'd like
It would be nice to be able to document RPCs and message fields in the .proto files.
Something like this:
message test{
int32 count = 1 [doc:Number of test runs to perform];
int32 registerValue = 2 [doc:Chip register value, with which to execute the test];
}
gWhisper should then display this documentation during tab-completion or when requested to (e.g. via CLI flags)
Example 1:
$ gwhisper 127.0.0.1 test.<TAB><TAB>
test.helloWorld returns a string containing "Hello World"
test.echoInt returns the integer which was passed into the RPC
Example 2:
$ gwhisper 127.0.0.1 test.echoInt <TAB><TAB>
number the number to be returned
In shells which support documentation in completions (zsh, fish, etc.) documentation should be correctly displayed as documentation.
Describe alternatives you've considered
Useful naming of RPC method names and field names is already very helpful, but often not sufficient.
Describe why you think this feature is useful for gWhisper users
According to the scope of the project the top priority is usability without the need for external documentation.
Additional context
We could use protobuf extensions for this (also available in proto3), as this allows documentation to be transmitted via reflection API.
Is your feature request related to a problem? Please describe.
I'm always frustrated when I type the wrong IP/hostname and gWhisper does wait forever trying to connect.
Describe the solution you'd like
Three solutions I can think of right now:
--connect_timeout_ms=xxx
which enables a timeout, if not given timeout is infinite--no_connect_timeout
which disables the timeout and use a timeout by defaultDescribe why you think this feature is useful for gWhisper users
For debug use, the default use-case of a user is to debug/test a server's service implementation. If IP is typed wrong or server is not running, we should get a response fast. The user should not have to think about why call is not returning.
Describe the solution you'd like
Currently completion only works for bash
Describe alternatives you've considered
stick with bash but more and more users are migrating to modern shells (fish is one of the most popular one)
Describe why you think this feature is useful for gWhisper users
users do not have to change shell to use gWhisper
Additional context
From my initial research fish might not provide a low-level API to "inject" a list of completions. As far as I know, fish supports two kinds of completion implementations:
Is your feature request related to a problem? Please describe.
Currently all gwhisper options and flags need to be placed before the IP address, It would be nice to have the possibility to add options ad other places.
Describe the solution you'd like
Add options before IP after IP and in the end.
maybe also in between message? this might give problems...
Describe alternatives you've considered
keep current implementation.
Describe why you think this feature is useful for gWhisper users
Users might forget to add an option and then need to scroll back to the beginning to ad an option
Additional context
this is not high priority right now, as not many options exist except gwhisper debug options.
Describe the bug
--version
just gives build date
To Reproduce
gwhisper --version
Expected behavior
version printed
Provide details about your environment
n/a
Additional context
version can be provided via environment variable, but this is easily forgotten by users. This will lead to bug-reports which have un-traceable code version.
Is your feature request related to a problem? Please describe.
server might not support reflection
Describe the solution you'd like
give optional proto file path as CLI argument, if given, reflection is not used
Describe alternatives you've considered
none
Describe why you think this feature is useful for gWhisper users
Describe the bug
map output not sorted
To Reproduce
do not know. sometimes maps appear randomized
Expected behavior
maps consistently transmitted
Following candidates for a move:
this could clean up the code structure and does not make the project look so messy.
Is your feature request related to a problem? Please describe.
Currently build only builds gwhisper
Describe the solution you'd like
make install
should work
Is your feature request related to a problem? Please describe.
I'm always frustrated when I tab, only :
is completed in fish shell
I'm always frustrated when I tab at end of a message, : :
is completed in bash shell
I'm always frustrated when I tab nothing is completed because a number is expected
I'm always frustrated when I tab enter and the app does not terminate
Describe the solution you'd like
I'm always frustrated when I tab, only :
is completed in fish shell
=> possible to complete space? or some hints
I'm always frustrated when I tab at end of a message, : :
is completed in bash shell
=> only one :
is compeletet
I'm always frustrated when I tab nothing is completed because a number is expected
=> I need some hints, that an integer is expected
I'm always frustrated when I tab enter and the app does not terminate
=> terminate gWhisper also after streaming
Describe why you think this feature is useful for gWhisper users
to be interactiver. improve user experence
Additional context
Maybe two many request to the server? My server always receive empty message after using gWhisper
To Reproduce
when no test server is running execute:
gwhisper 127.0.0.1 examples.ScalarTypeRpcs incrementNumbers
the cmd will fail with error message:
Parse failed. Parsed until: '127.0.0.1 '
Expected behavior
Expected more clear error indicating the problem that grammar could not be fetched from server.
Functional tests against completion sub-systems are already done.
We should do something similar for actually executing RPCs.
Describe the bug
RPCs with empty request message cannot be called.
To Reproduce
against test server execute:
./gwhisper 127.0.0.1 examples.NestedTypeRpcs getTime
Expected behavior
Expected rpc to succeed, however I observed
Parse failed. Parsed until: '127.0.0.1 examples.NestedTypeRpcs getTime'
Possible Candidates:
'127.0.0.1 examples.NestedTypeRpcs getTime '
2: #################################################################
2: Executing test 'oneof input choices again' at line 120
2: execute cmd '/home/travis/build/IBM/gWhisper/build/gwhisper --complete 127.0.0.1 examples.ComplexTypeRpcs sendNumberOrStringOneOf both=:number=5 str=5:'
2: Received:
2: number= (Both, a number and a string)
2: str= (Only a string)
2: both= (Both, a number and a string)
2: Expected:
2: number= (Only a number)
2: str= (Only a string)
2: both= (Both, a number and a string)
2: FAIL: line 1 received and expected text does not match.
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
./gwhisper 127.0.0.1 some.service ReadF + TAB still shows
ReadFile ReadFileProperties ReadFanBut after adding an "i" I get
ReadFile`
as completion.
subsequent tabs complete as if "ReadFile" was selected.
Describe the bug
I got the following error message with fish shell:
⋊> ~/t/gWhisper on 3af9796 ⨯ gwhisper /usr/share/fish/vendor_completions.d/gwhisper.fish (line 1): Variables may not be used as commands. In fish, please define a function or use 'eval $commandName'.
$commandName --complete=fish $args
^
in command substitution
called on line 0 of file /usr/share/fish/vendor_completions.d/gwhisper.fish
in function “getCompletions”
called on standard input
in command substitution
called on standard input
gwhisper gwhisper 17:17:04 17:17:04
To Reproduce
in shell tip gwhisper and then tab
Expected behavior
A clear and concise description of what you expected to happen.
Provide details about your environment
gwhisper --version
output:fish, version 2.7.1
gRPC/protobuf version: no idea
Compiler version (if related to build):
CMake version (if related to build):
⋊> ~ cmake --version 17:20:49
cmake version 3.10.2
CMake suite maintained and supported by Kitware (kitware.com/cmake)
Additional context
I use actually oh my fish
Describe the bug
The build is broken with the current snap package of cppcheck in the revision 228.
The configuration files like std.cfg were existent in previous packages (rev 213).
To Reproduce
Expected behavior
Provide details about your environment
Additional context
Checking out a specific snap revision of cppcheck seems to be disallowed.
sudo snap install cppcheck --revision=213
error: cannot install "cppcheck": Access by specifying a revision is not allowed for this Snap.
So the current plan is to disable cppcheck for the moment to get the build running again and then see if the snap package will be fixed or if the required config files will be added to this repo.
Is your feature request related to a problem? Please describe.
currently only unsecure servers are supported.
Describe why you think this feature is useful for gWhisper users
Users want to test secure servers as well
Is your feature request related to a problem? Please describe.
Describe the solution you'd like
It would be nice to have a reference gRPC server implementation exposing all gRPC and protobuf features (e.g. unary, streaming, empty messages, nested messages, repeated fields, oneof fields, ...)
Describe alternatives you've considered
none
Describe why you think this feature is useful for gWhisper users
I already implemented something similar for repeated fields, which works quite well.
Just need to generalize the solution (e.g. privide an abstract "composite" 'ListWithSeparator(", ")' grammar element constructor, to quickly create a grammar like this)
Describe the solution you'd like
It would be nice to automatically format fields the way they are supposed to.
Currently for example integers are by default formatted in both decimal and hex. One of those representations is in most cases unnecessary. When you write a .proto file you will know which representation of a field makes sense, so you could write something like this:
message test{
int32 count = 1 [formatHint:dec];
int32 registerValue = 2 [formatHint:hex];
}
which then influences the gWhisper output formatter.
Describe alternatives you've considered
none
Describe why you think this feature is useful for gWhisper users
makes output easier to read by only printing necessary information
Describe the bug
- map fields:
As a repeated field with nested key, value pairs (protobuf representation of maps).
E.g. my_map=::key=5 value=testval :, :key=40 value=anotherValue::
Map fields are tab-completed
Expected behavior
no space before ":"
nested Output programs or nested field references.
something like this:
@.listOfDevices:Found device /device_name/ @.device_features:allows music = /allows_music/:$'\n':
or this
@.listOfDevices:Found device /device_name/ with allows music = /device_name.allows_music/$'\n':
is not supported
ideally we should find a mechanism to generate manpage, online docu and --help output from the same source
Describe the bug
$ ./gwhisper 127.0.0.1 helloworld.Greeter SayHello name=asd fsd
Parse failed. Parsed until: '127.0.0.1 helloworld.Greeter SayHello name=asd'
also
$ ./gwhisper 127.0.0.1 helloworld.Greeter SayHello name="asd fsd"
Parse failed. Parsed until: '127.0.0.1 helloworld.Greeter SayHello name=asd'
To Reproduce
see above
Expected behavior
$ ./gwhisper 127.0.0.1 helloworld.Greeter SayHello name="asd fsd"
2019-03-06 16:08:55: Received message:
| message = "Hello asd fsd"
RPC succeeded :D
caused by infinite depth grammar being created
Typical scenario:
./gwhisper localhost testService callWithRecursiveMessage
fish: “./gwhisper localhost teatSer…” terminated by signal SIGSEGV (Address boundary error)
TODO:
we should have some checks, that only semi-stable code will find its way into the master branch.
semi-stable means:
Describe the solution you'd like
Currently, multiple short-lived TCP connections / gRPC channels are estabished for each call:
We could establish the channel only once and maybe also retrieve Service and Methods in one call to the reflection API.
This will have significant performance impact, as we will not have to go through TCP handshake and slow-start each time.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Describe why you think this feature is useful for gWhisper users
Especially in slow simulation environments this will improve performance significantly and also will make tab-completion much faster
Describe the bug
If gwhisper source is packaged as part of another git repository, the version string contains information about this repository.
To Reproduce
gwhisper --version
Expected behavior
we should see something else or nothing in the --version
output but not info from the other repository
Error parsing bytes field 'value': Given value is not a multiple of 8 bits long
Error parsing sub-message for field 'fields'
Segmentation fault (core dumped)
Describe the solution you'd like
Some method to transmit / receive binary data other than typing everything in hex
Describe alternatives you've considered
Describe why you think this feature is useful for gWhisper users
Some RPCs allow read/write access to rather large binary blobs (e.g. firmware update). It would be nice to also debug those RPCs.
Additional context
Binary output could be implemented as some kind of extension of --customOutput
.
This will be not so easy however, if other values in the reply also want to be seen, as --customOutput
completely replaces standard output formatting.
We more likely want something like --bytesToFileWithPrefix="myPrefix"
and then files are generated
Or some option --matchingFieldsToFile="regex"
Input can easily be done via some magic identifier instead of 0x in field value.
E.g.
data=0xab45 -> reads data as hex
data=file://~/file.bin -> reads data from file
Still not sure how to present the check results in pull requests
Describe the bug
...
[/home/travis/build/IBM/gWhisper/build/tests/testServer/examples.pb.h:3245]: (style) Condition 'GetArenaNoVirtual()==NULL' is always true
[/home/travis/build/IBM/gWhisper/build/tests/testServer/examples.pb.h:3277]: (style) Condition 'message_arena==NULL' is always true
[/home/travis/build/IBM/gWhisper/build/tests/testServer/examples.pb.h:3299]: (style) Condition 'GetArenaNoVirtual()==NULL' is always true
...
To Reproduce
trigger a travis build
Expected behavior
Generated code not checked (at least not generated code from protoc or other third-party code generators).
Describe the bug
Build fails because googletest directory does not contain a CMakeLists.txt
file
To Reproduce
./build.sh
Expected behavior
build successful
Provide details about your environment
gwhisper --version
output: v0.1.5 downloaded from githubAdditional context
Maybe it is possible to configure the build to not require googletest by default?
./gwhisper 127.0.0.1 examples.NestedTypeRpcs echoNestedMaps simple_map_string=::key=8 value=ad::
terminate called after throwing an instance of 'std::invalid_argument'
what(): stoul
Aborted (core dumped)
Currently gWhisper does not know anything about maps, it reads and writes them as repeated messages with key and value members.
It would be nice to detect maps and format them in a shorter format. e.g.
key -> value
Is your feature request related to a problem? Please describe.
currently all message have to be specified at start
Describe the solution you'd like
read from stdin line by line, parse messages and send to stream
Describe alternatives you've considered
none yet
Describe why you think this feature is useful for gWhisper users
full streaming support
Describe the bug
always 8 cores are used in build script
To Reproduce
build iwth build.sh script
Expected behavior
sense and use num of available cpu cores
Describe the bug
gwhisper 127.0<TAB>
causes the shell to freeze up, only <Ctrl>+C
allows recovery
To Reproduce
see above
Expected behavior
shell should not freeze up.
Additional context
Only happens when ip/hostname is incomplete/wrong
Is your feature request related to a problem? Please describe.
On machines with high-latency network connections (over-sea) or in simulation environments, tab-completion is sometimes slow (>300ms).
Describe the solution you'd like
Currently for every completion attempt one or more reflection API channels is established and destroyed to retrieve the descriptor database. However the database stays more or less the same every time.
It would be nice to have some mechanism to cache the descriptor database, and avoid connecting to the server for future requests.
Describe alternatives you've considered
I am still open for suggestions.
Describe why you think this feature is useful for gWhisper users
Increased usability by acceptable tab-completion performance.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.