tategakibunko / jingoo Goto Github PK

When {{ ... }} occurs inside a function body, only the last piece of the function body is returned, whether it is a literal part or an substituted value:

utop # #require "jingoo";;
utop # open Jingoo
utop # Jg_template.from_string "{% function test() %}a{{x}}{% endfunction %}{% set x = 'b' %}{{test()}}";;
- : string = "b"
utop # Jg_template.from_string "{% function test() %}a{{x}}c{% endfunction %}{% set x = 'b' %}{{test()}}";;
- : string = "c"

I think that original jinja engine would render an object using its __str__ method. I would like to have the same behavior in jingoo. If string_of_tvalue is called on an object, I would expect it to look for a __str__ (or another name, but I would prefer to stick with the original conventions).

Would you accept a PR to make string_of_tvalue looks for this property and uses "<obj>" (or corresponding strings) if the property is not found.

In Jingoo.Jg_template, each function take an association list for defining the values. Is it possible to have this on : ?⁠models:(string -> Jg_types.tvalue)

This allow to use different sources for the variables.

It could be useful to have the urlencode filter

Here is a proposal:
Have a special tests collection in context or environment. In the case of a is test, we could look at this collection first, and look for a regular filter if you found nothing.

In this case, we could have something like this

  | TestOpExpr(target, IdentExpr test) -> jg_test env ctx test [value_of_expr env ctx target]
  | TestOpExpr(target, test) -> jg_apply (value_of_expr env ctx test) [value_of_expr env ctx target]

and jg_test env ctx test v =
  try (Hashtbl.find env.tests test) v
  with Not_found -> jg_apply  (jg_get_value ctx test) v

with some std_tests defined in Jg_runtime, but nothing hardcoded in Jg_interp. i.e. user could use its own implementation / add any tests he wants.

Pattern matching would be really useful in order to avoid long series of {% if foo == ... %} with foo being the same in each elif case. I plan to add a switch statement to jingoo.

{% switch foo.bar %}
  {% case "foo" %}{{ 1 }}
  {% case "bar" %}{{ 2 }}
  {% case baz case toto %}{{ 3 }}
  {% default %}{{ -1 }}
{% endswitch %}

Also, it would be a more efficient way to compare values (if the value to compare needs some calculation for instead, you would need to use a set or every elif would be a waste of ressources.)

Any idea/objection?

EDIT:

{% case baz case toto %}

Would be a problem for the parser if you want to allow 'no statements' as right hand part of the case

{% switch foo.bar %}
  {% case "foo" %}
  {% case "bar" %}{{ 2 }}
{% endswitch %}

Would be the same as

{% switch foo.bar %}
  {% case "foo" case "bar" %}{{ 2 }}
{% endswitch %}

Which is obviously not what is wanted.

let rec jg_eq_eq_aux left right =
  match left, right with
    | Tint x1, Tint x2 -> x1=x2
    | Tfloat x1, Tfloat x2 -> x1=x2
    | Tstr x1, Tstr x2 -> x1=x2
    | Tbool x1, Tbool x2 -> x1=x2
    | Tlist x1, Tlist x2
    | Tset x1, Tset x2 -> jg_list_eq_eq x1 x2
    | Tarray x1, Tarray x2 -> jg_array_eq_eq x1 x2
    | Tobj _, Tobj _ ->
      begin
        try unbox_bool @@ jg_apply (jg_obj_lookup left "__eq__") [ left ; right ]
        with _ -> jg_obj_eq_eq left right
      end
    | ((Thash _ | Tobj _ | Tpat _) as left), ((Thash _ | Tobj _ | Tpat _) as right) ->
      begin
        try unbox_bool @@ jg_apply (jg_obj_lookup left "__eq__") [ left ; right ]
        with _ -> false
      end
    | _, _ -> false

With the current behavior, it is impossible to write a test such as {% if x == null %}. It seems counter-intuitive. It is particularly true when trying to handle {% case null %}, which is not possible.

Is there a reason why null != null? If so, we should document this. If not, do we change the behavior?

Another way to solve this is to specialize the test perform when it is with LiteralExpr Tnull. I would actually prefer this. But I have no strong opinion on this, so if you prefer to simplify things jg_eq_eq_aux Tnull Tnull = true would solve the issue as well.

Using the 4.02.3 switch and running opam install jingoo, I get the following output:

$ opam install jingoo
The following actions will be performed:
  - install jingoo 1.2.10

=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[default] https://opam.ocaml.org/archives/jingoo.1.2.10+opam.tar.gz downloaded

=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[ERROR] The compilation of jingoo failed at "gmake".
Processing  1/1: [jingoo: ocamlfind remove]
#=== ERROR while installing jingoo.1.2.10 =====================================#
# opam-version 1.2.1
# os           freebsd
# command      gmake
# path         /home/vagrant/.opam/4.02.3/build/jingoo.1.2.10
# compiler     4.02.3
# exit-code    2
# env-file     /tmp/jingoo/jingoo-18644-e7966f.env
# stdout-file  /tmp/jingoo/jingoo-18644-e7966f.out
# stderr-file  /tmp/jingoo/jingoo-18644-e7966f.err
### stdout ###
# ocamlfind ocamlopt -g -package unix,dynlink,pcre,batteries -c jg_types.mli jg_types.ml jg_utils.ml jg_stub.mli jg_stub.ml  jg_parser.mli jg_parser.ml jg_lexer.ml  jg_runtime.ml jg_interp.ml  jg_template.mli jg_template.ml
# [...]
# ocamlfind ocamlopt -g -package unix,dynlink,pcre,batteries -o jingoo.cmxa -a 
# ocamlfind ocamlopt -g -thread -c 
# ocamlfind ocamlc -g -thread -c 
# ocamlfind ocamlopt -g -o compiler -linkpkg -package unix,dynlink,pcre,batteries jingoo.cmxa jg_cmdline.ml
# *** Error code 2
# 
# Stop.
# make[1]: stopped in /usr/home/vagrant/.opam/4.02.3/build/jingoo.1.2.10/src
# Makefile:2: recipe for target 'all' failed
### stderr ###
# Warning 3: deprecated: String.create
# [...]
# Use Bytes.create instead.
# File "jg_utils.ml", line 95, characters 14-27:
# Warning 3: deprecated: String.create
# Use Bytes.create instead.
# File "jg_cmdline.ml", line 1:
# Error: No implementations provided for the following modules:
#          Jg_template referenced from jg_cmdline.cmx
#          Jg_types referenced from jg_cmdline.cmx
# gmake: *** [all] Error 1



=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions failed
  - install jingoo 1.2.10

No changes have been performed

Would you be against a new function keyword, which is basically a macro, but returning a tvalue instead of something meant to be print?

Something like:

{% function aux (str) %}
  {{ (upper (str[0]), str) }}
{% endfunction %}

{% for grouper, list in data | map ("aux") | groupby ("[0]") %}
  <ul>
    {{ grouper }}
    {% for s in list %}<li>{{ s }}</li>{% endfor %}
  </ul>
{% enfor %}

Is there any technical problem? I think it is useful for deploying and client-side rendering.

In Jinja, it is possible to pass data to template via some function calls. May I know if similar thing can be done in Jingoo?

Thanks in advance!

Hey !

First, thank you for your work, I was very glad to find a jinja parser/interpreter for OCaml.

I started a lot of modifications (you already saw some of my PR). Another thing that I have done is to implement lazyness for tvalue. I did it with a new Tlazy of tvalue Lazy.t constructor. It is very useful (and even mandatory) for recursive data structure without wrapping everything in functions (i.e. a person has a mother, which is a person herself).

Also, I plan (but I do not know when) to functorize jingoo in order to allow the user to use a custom type 'a value = ... | Taplha of 'a.

My question is : are you interested in having a Tlazy constructor? If not, are you interested in having jingoo as a functor (with default values, likeHashtbl module). If you are not interested in laziness but you are interested in functor, I could implement the lazyness as a custome type so it would be okay for me. If none of these option suit you, well... I guess that it would be ok to keep my fork up to date with your repo...

Problem: When using a {% for .. %} loop, if the object being looped-over doesn't exist, an exception is thrown: Exception: Failure "jg_iter:not iterable object".
Example:

{% for p in m.params %}
  {{ p }}
{% endfor %}

Here if m does not have a params member, the exception is thrown.

Expected output: The loop should be silently ignored (i.e. treat m.params as empty list). This is how Nunjucks (jinja port for node.js) works.

Rationale: Due to this bug every single for loop must check that the object exists (and is iterable - not even sure how to check that!):

{% if m.params %}
  {% for p in m.params %}
    {{ p }}
  {% endfor %}
{% endif %}

This adds needless boilerplate to all templates and also deviates from other implementations' behaviour.

Same for ./advanced, in the examples.

It should be possible to make a PPX that generates a function that maps an instance of some type "t" into the typed jingoo model representation. For projects with complex DB models, this could significantly cut down on boilerplate.

I often need to define custom compare functions while sorting data in templates, for instance:

{%- function cmp_fn (a, b) -%}
  {{ alphabetic (a.first_name, b.first_name) }}
{%- endfunction -%}
{%- set list = list | sort (compare = cmp_fn) -%}

In some case, it would be nice to be able to define function on the fly, like this:

 list | sort (compare = ((a, b) => alphabetic (a.first_name, b.first_name)))

Would you agree with this kind of feature?

Do you have a better syntax in mind? I picked

LPAREN separated_list(COMMA, ident) RPAREN => expr

because it should interact nicely with current parser.

Also, it is a form used by some fat arrows functions in javascript.

Note that it would restrict anonymous function to very simple functions (no instruction, only one expression as function body), which is probably better anyway.

Right now the escape function uses numeric ASCII codes like >. This if functional, but can be hard for a human reading that HTML. It's also a bit contrary to the way many other tools work. Symbolic entities like > are easier to read.

However, I did look in the code and I see why you went with numeric codes—since all required codes consist of two digits, the length of the substitute substring is constant: one character like > is always replaced by a 5-character string like >. A variable-length implementation is more annoying to make.

If you aren't categorically opposed to a variable-length implementation, I'm ready to help with it. Just want to check first.

I sometime use a lot of {% set %} instructions in my templates in order to conditionally concatenate string in functions. Supporting += operator for affectations would simplify a little bit their writing.

{% set res = res + 'foo' %}
{% set res += 'foo' %}

Would it worth it to add this feature?

I've try to release jingoo 1.3.0 today, but it failed...

Maintainer is guessing that "the repository revision it was made from doesn't contain all the dependencies of the package".

ocaml/opam-repository#13422 (comment)

@sagotch I have no idea about this, do you know anything about this?

I have a usage question: is it possible to extend a string template (Jg_template.from_string) from another string template?

I'm in the situation in which templates should not be distribuited and I'm keeping those in memory, as OCaml strings.

For each template language, there is the same debate if no extension is mentioned by the creator, so I would suggest to pick .jingoo (or anything you like) and mention it in the doc as a not-mandatory-but-preferred file extension. It is a common practice to use the full name of the markup language as file extension.

For instance, Jinja2 has no official extension, but emacs jinja2-mode uses .jinja2 https://github.com/paradoxxxzero/jinja2-mode/blob/master/jinja2-mode.el#L324

Hello, thanks for creating this fantastic project!

Just I am having trouble to append item into list in template. For some reason I am using version 1.2.21(and probably have to stay with it for some time). Basically, below code does not work,
`{% set a = [] %}
{{ a.append("a") }}
{{ a | length }}

OUTPUT of above code: 0
`
Is this normal/expected?

I see there is an addition of overload of jg_plus to support list operation in v1.3.0, would upgrade to 1.3.0 solves my issue?

Thanks a lot in advance!

A mistake I do quite often is to use {% else if ... %} instead of elif or elseif. Would you mind supporting else if as well? Of course, I can propose a PR for this if you agree.

With these two files:

<html><head></head><body>{% block body %}{% endblock %}</body></html>

{%- function foo () -%}{{ "HELLO" }}{%- endfunction -%}

This wont work

{%- extends "template.jingoo" -%}
{%- include 'utils.jingoo' -%}
{%- block body -%}{{ foo () }}{%- enblock -%}

This will

{%- extends "template.jingoo" -%}
{%- block body -%}{%- include 'utils.jingoo' -%}{{ foo () }}{%- enblock -%}

Because functions are treated as variables and not hoisted as macro are, we can easily run into this kind of errors.

Should not functions be treated as macros?

f0432b5 introduce a lot of new frames, when previous behavior only added values to current scope.

I do not know if a proper fix exists (or if it is needed), but we could imagine this

diff --git a/src/jg_interp.ml b/src/jg_interp.ml
index 6849b7f..5b03fdb 100755
--- a/src/jg_interp.ml
+++ b/src/jg_interp.ml
@@ -162,6 +162,14 @@ and eval_statement env ctx = function
   | ExpandStatement(expr) ->
     jg_output ctx (value_of_expr env ctx expr) ~autoescape:env.autoescape ~safe:(is_safe_expr expr)
 
+  | SetStatement(SetExpr([IdentExpr ident]), expr) ->
+    { ctx with frame_stack = match ctx.frame_stack with
+          | hd :: tl ->
+            let expr = value_of_expr env ctx expr in
+            (fun s -> if s = ident then expr else hd s) :: tl
+          | [] -> assert false
+    }
+
   | SetStatement(SetExpr(ident_list), expr) ->
     jg_frame_table ctx (fun table ->
       jg_bind_names table (ident_names_of ident_list) (value_of_expr env ctx expr)

I only wrote this example for a single value, but in case of multiple definitions ({% set a,b,c = %}) I think that building an associative list and using

fun s -> match List.assoc_opt s list with Some v -> v | None -> hd s

would be better than pushing a new Hashtbl.t. In real life, the list will never be long enough so Hashtbl.t would be better than a simple list.

Unless I'm missing something, in jg_cmdline.ml, let models = ref [] is never written to:

When I compile a simple program that doesn't use thread:

• ocamlfind ocamlc -c -g -I src/lib/logger -package calendar -package cmdliner -package fileutils -package jingoo -I src/bin -I src/lib/logger -o src/bin/Data.cmo src/bin/Data.ml
  ocamlfind: Error from package `threads': Missing -thread or -vmthread switch

This is because there is a require on "threads".

I recommend to split the package into another package (jingoo.threads) which contains the only file that depends on thread (a Mutex call in jingoo_mt.ml). It maybe be better to have an init function call to ensure the time it will be called.

(the fix is simple: just add threads to the package of my library, but this is a workaround, I think the design of the library doesn't want that).

The constructor is called Tset, but a set is a collection of unique items.

Tset is definitely more considered as a tuple rather than a set. So we probably should rename this constructor.

I am not sure that tuples are really useful in jingoo. If there really is no benefit from using tuples instead of list, maybe that we could remove this data type in order to keep jinja simpler, remove some conflicts and simplify the parser.

This is a regression introduced by 8d1073c

Because default might be a token, it is returned as a token whenever you are in a Logic lexer_mode.

{% set pg = default (0, env.pg) %}

will raise

Jingoo.Jg_types.SyntaxError("Error line _, col _, token default ()")

I will take a look at it when I have time.

Do you have a example where not using mutex would be a problem?
I do not understand what could go wrong if we remove this lock.

I need to write loops and if statement, so that I have no empty lines.

I am trying to write:
Build-Depends: debhelper (>= 7.0.50~),
{% for deps in extra_build_depends %}
{{deps}},
{% endfor %}
opam2debian

But it is transformed into:
Build-Depends: debhelper (>= 7.0.50~),

opam2debian

Jinja2:
http://jinja.pocoo.org/docs/templates/#whitespace-control