Giter VIP home page Giter VIP logo

ppx_deriving_yaml's Introduction

ppx_deriving_yaml

This ppx is based on ppx_yojson and ppx_deriving_yojson because of the many similarities between JSON and yaml. In particular many of the way the OCaml values are encoded to yaml types are the same as those implemented by the Yojson ppx.

Basic Usage

For converting OCaml values to yaml values ppx_deriving_yaml will do the conventional dropping of the type name if it is t. Otherwise the type name is the prefix to the to_yaml function.

to_yaml produces a Yaml.value which is compatible with the Ezjsonm.value type.

of_yaml produces OCaml types wrapped in a result -- this is how ocaml-yaml also handles errors i.e. not using exceptions. Based on your type this should let you move between yaml and OCaml values.

# #require "ppx_deriving_yaml";;

Here is a small example.

type person = { name : string; age : int } [@@deriving yaml]
type users = person list [@@deriving yaml]

This will produce four functions, a _to_yaml and _of_yaml for both a person and the users. For example:

# person_to_yaml;;
- : person ->
    [> `O of (string * [> `Float of float | `String of string ]) list ]
= <fun>
# users_of_yaml;;
- : [> `A of [> `O of (string * Yaml.value) list ] list ] ->
    (person list, [> `Msg of string ]) result
= <fun>

If you make polymorphic types, then you will have to supply the function to convert the unknown to a yaml value. For example:

type 'a note = { txt : 'a } [@@deriving yaml]

produces the following function.

# note_to_yaml;;
- : ('a -> Yaml.value) -> 'a note -> [> `O of (string * Yaml.value) list ] =
<fun>

Finally, if you only need the encoder (to_yaml) or the decoder (of_yaml) then there are single versions of the deriver for those.

# type x = { age : int }[@@deriving to_yaml];;
type x = { age : int; }
val x_to_yaml : x -> [> `O of (string * [> `Float of float ]) list ] = <fun>

Attributes

Key and Name

Record field names cannot begin with a capital letter and variant constructors must start with one. This limits what the generated yaml can look like. To override the yaml names you can use the [@key <string>] and [@name <string>] attributes for records and variants respectively.

For example:

type t = {
  camel_name : string [@key "camel-name"]
}[@@deriving to_yaml]

Will produce Yaml of the form

# Yaml.to_string_exn (to_yaml { camel_name = "Alice" });;
- : string = "camel-name: Alice\n"

Default Values

You can also specify default values for fields.

type t = {
  name : string;
  age : int [@default 42]
}[@@deriving yaml]

These will be used in the absence of any fields when decoding yaml values into OCaml ones.

# Yaml.of_string_exn "name: Alice" |> of_yaml;;
- : (t, [> `Msg of string ]) result = Ok {name = "Alice"; age = 42}

Custom encoding and decoding

Sometimes you might want to specify your own encoding and decoding logic on field by field basis. To do so, you can use the of_yaml and to_yaml attributes.

type t = {
  name : string [@to_yaml fun i -> `String ("custom-" ^ i)]
}[@@deriving yaml]

The to_yaml function will use the custom encoder now instead.

# Yaml.to_string_exn (to_yaml { name = "alice" });;
- : string = "name: custom-alice\n"

Partially Decoding

There is a ~skip_unknown flag for telling the deriver to simply ignore any fields which are missing. This is particularly useful when you only wish to partially decode a yaml value.

Consider the following yaml:

let yaml = "name: Bob\nage: 42\nmisc: We don't need this!"

If we try to do the normal decoding of this but only partially extract the fields, it will throw an error.

type t = {
  name : string;
  age : int;
}[@@deriving yaml]

Note that the error is often rather confusing. There is room for improvement (PRs welcome!).

# Yaml.of_string_exn yaml |> of_yaml;;
- : (t, [> `Msg of string ]) result =
Error (`Msg "miscWe don't need this!\n")

Instead we tell the deriver to ignore unknown fields.

type t = {
  name : string;
  age : int;
}[@@deriving yaml ~skip_unknown]
# Yaml.of_string_exn yaml |> of_yaml;;
- : (t, [> `Msg of string ]) result = Ok {name = "Bob"; age = 42}

Implementation Details

One important thing is that 'a option values within records will return None if the Yaml you are trying to convert does not exist.

OCaml Type Yaml Type
int `Float
float `Float
string `String
bool `Bool
None `Null
list `A []
array `A []
record e.g { name : string } `O [("name", `String s)]
A of int or [`A of int] `O [("A", `A [`Float f])]

ppx_deriving_yaml's People

Contributors

andreypopp avatar maurobringolf avatar patricoferris avatar prosper74 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

ppx_deriving_yaml's Issues

Improving error messaging

Feature Request

Error handling in the deriver is not great which was helpfully discovered this thanks to @guptadiksha307.

The biggest improvement would be to provide better error messages when the wrong type of Yaml value is uncovered. For example this code could be a lot better and at least described what it was expecting to find https://github.com/patricoferris/ppx_deriving_yaml/blob/main/lib/value.ml#L162 !

An example

Consider the following little program:

type place = {
  name : string;
  population : string;
} [@@deriving yaml]

type t = {
  places : place list
} [@@deriving yaml]

let yml = {|
places:
  - name: Belfast
    population: 634594|}

let () = 
  match Yaml.of_string_exn yml |> of_yaml with
    | Ok _ -> print_endline "Parsed Nicely :)"
    | Error (`Msg m) -> print_endline m

In the data (the yml string) you can the population of Belfast is given as a number 634594 rather than a string "634594". The type of population in place is a string however so the generated of_yaml function will fail because it will try to read a string and instead find a number!

We might expect the error message to tell us something informative like Was expecting a string but got a <x>, but instead we get the following useful error message!

err

If it is useful the code above was compiled with:

(executable
 (name main)
 (libraries yaml)
 (preprocess
  (pps ppx_deriving_yaml)))

Omit `[@default]` values in `to_yaml`

The [@default] attribute added in #31 is considered in of_yaml, but ignored in to_yaml. It would be nice, if to_yaml would omit fields with their default values. That is how ppx_deriving_yojson does it at least.

In particular, this would be useful for fields with option type and [@default None]: of_yaml doesn't require the field in input (making it None) and the subsequent to_yaml wouldn't print out the field with empty value (because it's unnecessary noise for an optional field).

Embed errors in the AST instead of raising

Currently, when your PPX encounter an error, it uses the raise_errorf function to raise a located error.

The exception is caught by ppxlib, which in this case:

  • Catch the error,
  • stops the rewriting process
  • add the error (as a [%%%ocaml.error] extension node) to the last valid ast
  • Use the resulting AST

The interruption of the rewriting is quite bad for the user experience! The implication for the users are:

  • Since your PPX runs at the "context-free" phase, the "last valid AST" is before the context-free phase. So, no other derivers/extenders get run, which generates a lot of noise in the errors (such as "uninterpreted extensions" or "unbound identifiers")
  • Only one (meaningful) error from your PPX is reported at a time.
Example

For instance:

type t = int -> int [@@deriving yaml]
type u = int -> int [@@deriving yaml]
type v = int [@@deriving yaml]

let _ = v_of_yaml

would generate several errors:

  • Cannot derive anything for t
  • unbound value v_of_yojson

when the correct set of errors would be:

  • Cannot derive anything for t
  • Cannot derive anything for u

You can find more information about error reporting in PPXs in this section of the ppxlib manual.

โ“ Would you be willing to accept contributions to this issue? I'm considering assigning its resolution as part of an outreachy internship: see more information here.

Add extra deriving option for constant variant constructors

A very common pattern is to want to encode string values as a polymorphic (or not) variant with constant constructors. For example:

module Member = struct 
  type t = [ `Student | `Professor ][@@deriving yaml]
end 

type t = { name : string; member : Member.t }[@@deriving yaml]

The standard [@@deriving yaml] deriver has to deal with the possibility of non-constant constructors, so these constant ones are encode as Student: [] which is annoying because then we have:

let person = { name = "Alice"; member = `Student } 

encoding to the following yaml:

name: Alice
member: 
  Student: []

What would be nice is to opt into some string only version that generates the following code:

let of_yaml = function
    | `String "Student" -> Ok `Student
    | `String "Professor" -> Ok `Professor
    | _ -> Error (`Msg "Unknown value ")

let to_yaml = function
  | `Student -> `String "Student"
  | `Duplicate -> `String "Professor"

And additionally an extra little [@value string] to override the default of using the constructor name verbatim.

Variant names (record field names) can't express YAML names

Problem

Currently there is no way to specify your own names for keys in the `O of (string, value) list type (which variants and records transform to). For example:

type t = { camel : int } [@@deriving yaml]

Will only read/produce YAML of the form

camel: 1

We can't have Camel or cam-el as these are not allowed as record field names. The same problem exists for variants like type t = Camel of int.

Solution

Add a name attribute to the ppx in order to specify a transformation -- exactly like ppx_deriving_yojson :)

Is it possible to derive for a field that has arbitrary string keys?

I'm trying to model a YAML structure like:

name: something
jobs:
  - <arbitrary name>:
    standard-key1: value
    standard-key2: value
  - <arbitrary name>:
    standard-key1: value
    standard-key2: value

I can achieve this serialized output by constructing my ocaml value directly from `O and `A literals.

But I would like to define a record type like:

type action = {
  name: string;
  jobs: (string * job) list;
}
[@@deriving yaml]

Is there a recipe for this in ppx_deriving_yaml currently?

ocaml-yaml defines `O of (string * value) list so I was hoping that jobs: (string * job) list would work, but it seems that the (string * value) tuples are serialized as a list instead of key-value object, so the yaml string comes out as:

name: something
jobs:
  - - <arbitrary name>:
    - standard-key1: value
      standard-key2: value
  - - <arbitrary name>:
    - standard-key1: value
      standard-key2: value

I am new to OCaml so I apologise if I am missing something obvious that I've done wrong.

Thanks for this useful library!

(I get the same problem from my record type using ppx_yojson_conv and output as JSON too)

Error loading package in toplevel

A fatal error was encountered when loading the package into utop. Please see error message below.

utop # #require "ppx_deriving_yaml";;
Line 1, characters 0-4Parse error: illegal begin of top_phrase
Fatal error: exception Exit
Raised at file "string.ml", line 115, characters 25-34
Called from file "src/sexp.ml", line 113, characters 13-47

Thanks.

Separate `to_yaml` and `of_yaml` derivers

It would be nice if one could just derive to_yaml or of_yaml (with those corresponding deriver names). Then the yaml deriver can just be a ppxlib deriver alias for the previous two.

For example, ppx_deriving_yojson registers three derivers: to_yojson, of_yojson and yojson.

Support `inline` annotations

That would be nice to be able to do something like:

type x = { a: int; b: int } [@@deriving yaml]
type t = { 
  x: x; [@inline]; 
  y: string
} [@@deriving yaml]

That would be able to parse:

t:
  - a: 1
  - b: 2
  - y: foo

(or whatever yaml is for a single record ;p)

into:

{ x = { a=1; b=2}; y="foo" }

Add `default` attribute

Feature Request

Lots of derivers have a default attribute. These let you specify that in the absence of the key (the key in the example below is name) then the default value shall be what is supplied in the attribute.

type t = {
  name : string; [@default "Alice"]
}[@@deriving yaml]

ppx_deriving_yaml already has a very simplistic version of this for 'a option types where in their absence None is supplied (rather than failing to parse some yaml without that key). A good place to start (thanks @pitag-ha!) would be to how https://github.com/janestreet/ppx_yojson_conv/blob/master/src/ppx_yojson_conv.ml does it using Ppxlib.Attributes https://ocaml-ppx.github.io/ppxlib/ppxlib/Ppxlib/Attribute/index.html

A concrete example

Consider the yaml:

places:
  - name: Belfast
    continent: Europe

And the OCaml type that tries to describe it:

type place = {
  name : string;
  continent : string;
  is_city : bool;
} [@@deriving yaml]

type t = {
  places : place list
} [@@deriving yaml]

With the current implementation this will fail when trying to parse Belfast's information because there is no key for is_city.

Didn't find the function for key: is_city

(Output from this example program: https://gist.github.com/patricoferris/4f54f170a7868e27a30c9f6f87b0ec5b)

Most of our places might be cities and so it would be nice to supply a default value. In which case we'd have:

type place = {
  name : string;
  continent : string;
  is_city : bool; [@default true]
} [@@deriving yaml]

type t = {
  places : place list
} [@@deriving yaml]

And the program above (with this change) would successfully parse our list of places :)

Recursive types are not supported

Right now it looks like the serializers and deserializers are defined as Nonrecursive:

pstr_value ~loc Nonrecursive

This means that a type like the following won't work:

type t = {
  name : string;
  children : t list
}
[@@deriving yaml]

The compiler produces this error:

Unbound value to_yaml

(To be clear, I'm not suggesting that simply making the functions recursive will fix this problem, but it's at least part of why this doesn't work. Maybe setting Nonrecursive if the type is specifically nonrec would fix it?)

Parsing with unknown fields

I would like to use the ppx to partially define the type, so as to only fetch the needed fields.

For example, using a type:

type t = {
  type_ : string option; [@key "type"]
}

And a yaml:

name: vaudit
type: "task"

It fails with an error namevaudit

Would it be possible to support parsing while skipping any unknown elements in the yaml ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.