google / go-jsonnet Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Annotate individual end to end tests with things like:
This is needed to properly test some features and it would be very helpful in unifying the Go and C++ test suites.
We're affected by this: golang/go#14932
BTW it seems it is non-trivial to have exactly the same values of numeric functions on all platforms - golang/go#20319.
It would be nice to use go version as a commandline drop-in replacement for C++ version.
We can avoid implementing it in Go for now if we implement libjsonnet #46 and use that from C++ commandline. The advantage is full compatibility, more tests of go-jsonnet (currently libjsonnet tests are very minimal, interesting tests happen through invoking jsonnet command) and we get it almost for free once we have #46.
If I were writing a jsonnet formatter, I'd like to have access to comments as AST nodes so I can include them when I'm rewriting the source. It would also be helpful if using the AST to generate jsonnet as a conversion from another source.
By avoiding parsing the std every single time.
Hi jsonnet team,
As golang is now the preferred language for ops, will go-jsonnet become the de facto implementation for jsonnet?
IMHO, the go version is more likely to be adopted by the community as a library. Also from code readability perspective, golang version is more likely to get merge-able PRs.
They are currently ignored
We want to be able to use go version as a drop-in replacement for C++ version (for evaluation only, no formatting). This will probably require some changes to happen first on C++ side, e.g. separation of evaluation and formatting.
It's a part of the larger API discussion.
Just asking because i am using this with kubernetes and trying to avoid any c++ dependencies.
I've set myself an arbitrary objective today to allow passing through JSON, i.e. add support for desugaring, checking, executing, manifesting basic objects and arrays containing other objects / arrays / primitives.
EvaluateSnippetMulti returns an "object" map[string]string
, not an array.
Currently we don't detect missing arguments etc.
It's going to be useful for development of tools.
For now it's possible to have a generic dump using go-spew (https://github.com/davecgh/go-spew).
Super can be a perfectly good value. It's not an object, though. It would be just a selfBinding - it's indexable, but + doesn't make sense on it.
We could desugar in super etc. and avoid treating it specially during evaluation.
I found this in code:
case *valueFunction:
return nil, makeRuntimeError("Couldn't manifest function in JSON output.", i.getCurrentStackTrace(trace))
and my input is something like this:
local a = import "../../../applib/a.libsonnet";
local b = import "../../../applib/b.libsonnet";
function(config={})
local overall_config = default_config + config;
(if a.match(overall_config) then
{
...
}
else
{
...
}) + {
...
}
And i get this:
RUNTIME ERROR: Couldn't manifest function in JSON output.
During manifestation
How can i resolve this?
Thanks!
First we need to have some benchmarking in place, and some reasonable "corpus" of jsonnet to test on. We can start by using current test suite, but dedicated benchmarks make more sense.
Once we have that, we can try to find bottlenecks and potentially optimize. Instead of having them scattered throughout the code, I'll dump some of these ideas here:
Before we implement any of these we should have a benchmark that proves that it actually helps.
Some constructs are desugared to functions from std. The problem is that if the user wraps them in their own definition of std they can change the meaning of core language constructs (probably just break them) which will be extremely confusing.
We can have special alias for std, unavailable for the user, for example $td
.
At present, it appears that go-jsonnet's public interface starts with jsonnet.MakeVM
, against which one can call (*VM).ExtVar
, (*VM).ExtCode
, and (*VM).Importer
to prepare an environment, and eventually call (*VM).EvaluateSnippet
against a sequence of Jsonnet expressions.
That call to (*VM).EvaluateSnippet
eventually calls on the internal snippetToAST
function, which yields a parsed ast.Node
ready for later use in the evaluate
function. In there, we find some more code-to-AST translation for environment entries that are Jsonnet code.
Is it viable to cleave a new break into this interface, so that it's possible to parse some Jsonnet—together with its import callbacks, and perhaps code entries from the environment—and yield a prepared object that could then be used for repeated evaluation against different TLAs?
The use case I have in mind is a server that would take a Jsonnet function that expects to receive some JSON as a TLA and evaluate the same Jsonnet function repeatedly and frequently, just with its input changing. It's unfortunate to have to restart this procedure each time from raw text when the code involved won't change between evaluations, throwing away the parsed form of the input text.
I concede that I have no measurements to critique the cost of the current interface compared to the separate prepare-once/evaluate-repeatedly approach sketched above. Has this project considered and rejected this approach before? If not, I'm willing to help pursue it.
A few ideas for an interface:
(*vm).PrepareSnippet(filename string, snippet string) (*Program, error)
(*Program).Eval(vs map[string]string) (string, error)
(*Program).EvalTo(w io.Writer, vs map[string]string) error
Hi,
Thanks for open sourcing go-jsonnet library. Do you have any plan implementing the 'import' function in jsonnet language any time soon?
Thanks,
Jim
Hi, it's me again, I found another thing which behaviours not the same as the c++'s version.
I think this is because the go join function doesn't behaviours the same as the c++ version's.
In the c++'s version, if the expression result is null, it will be ignored.
@sbarzowski
@sparkprime
Still there is a bunch of builtins that are available in C++ version and not here.
Current status:
builtins["makeArray"] = &Interpreter::builtinMakeArray; DONE
builtins["pow"] = &Interpreter::builtinPow; DONE
builtins["floor"] = &Interpreter::builtinFloor; DONE
builtins["ceil"] = &Interpreter::builtinCeil; DONE
builtins["sqrt"] = &Interpreter::builtinSqrt; DONE
builtins["sin"] = &Interpreter::builtinSin; DONE
builtins["cos"] = &Interpreter::builtinCos; DONE
builtins["tan"] = &Interpreter::builtinTan; DONE
builtins["asin"] = &Interpreter::builtinAsin; DONE
builtins["acos"] = &Interpreter::builtinAcos; DONE
builtins["atan"] = &Interpreter::builtinAtan; DONE
builtins["type"] = &Interpreter::builtinType; DONE
builtins["filter"] = &Interpreter::builtinFilter; DONE
builtins["objectHasEx"] = &Interpreter::builtinObjectHasEx; DONE
builtins["length"] = &Interpreter::builtinLength; DONE
builtins["objectFieldsEx"] = &Interpreter::builtinObjectFieldsEx; DONE
builtins["codepoint"] = &Interpreter::builtinCodepoint; DONE
builtins["char"] = &Interpreter::builtinChar; DONE
builtins["log"] = &Interpreter::builtinLog; DONE
builtins["exp"] = &Interpreter::builtinExp; DONE
builtins["mantissa"] = &Interpreter::builtinMantissa; DONE
builtins["exponent"] = &Interpreter::builtinExponent; DONE
builtins["modulo"] = &Interpreter::builtinModulo; DONE
builtins["extVar"] = &Interpreter::builtinExtVar; DONE
builtins["primitiveEquals"] = &Interpreter::builtinPrimitiveEquals; DONE
builtins["native"] = &Interpreter::builtinNative; TODO: see #44
builtins["md5"] = &Interpreter::builtinMd5; DONE
Expected and actual output seem to be randomly mixed together. Maybe line-based diff would be better after all.
Some constructs using super don't work yet properly.
$ go-jsonnet --max-stack 7 -e 'local f(n, c=0) = if n == 0 then c else f(n - 1, c + n) tailstrict; f(100)'
RUNTIME ERROR: Max stack frames exceeded.
<std>:979:29-30 thunk from <thunk <ta> from <function <anonymous>>>
<builtin> builtin function <type>
<std>:981:33-35 thunk from <function <anonymous>>
<builtin> builtin function <primitiveEquals>
<std>:981:12-40 function <anonymous>
<cmdline>:1:69-75 $
During evaluation
When fixed, remove special go golden for test_cmd case max_trace4
go vendor folder is missing.
Where does it come from?
https://imgur.com/a/oJMNy
Accessing a field multiple times results in multiple evaluations. This can be very easily avoided and innocent looking code may be in fact exponential.
I know there are some concerns about memory usage.
github.com/fatih/color is a small dep, but holds a larger dep tree. Coloring output is great for a terminal program, but not so great for a package.
I can work something up if you approve of the idea. I have a google CLA for the golang/go project already.
While the RuntimeError type is exported, there does not seem to be an exported way to construct a RuntimeError or an exported way to populate the StackTrace ([]TraceFrame) from.
It would be useful to be able to have an exported variant of the get* functions in evaluator.go that return Go types (e.g. string instead of valueString) and a RuntimeError. However, just being able to get the stacktrace and exporting makeRuntimeError would be sufficient.
Returning other error types from a native function results in an "INTERNAL ERROR" error message, requesting that the user file a bug. The lack of a stack trace in the error message can make it difficult to know what caused the error.
Current version completely ignores tailstrict
The parameter names are visible to the user - they can use named arguments instead of positional for every function, including those in stdlib and builtins. Generally the names of parameters should be considered a part of the interface.
So we need to make sure they are right (consistent with http://jsonnet.org/docs/stdlib.html) and it should be verified by tests.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.