lordnull / rec2json Goto Github PK

View Code? Open in Web Editor NEW

46.0 8.0 23.0 376 KB

Compile erlang record definitions into modules to convert them to/from json easily.

License: Apache License 2.0

Makefile 0.39% Erlang 99.61%

erlang json json-converter

rec2json's People

Contributors

Stargazers

Watchers

Forkers

jvliwanag sankark morgul chinnurtb hairyhum cjimison karelwar arjan andekar liwooood yzprofile niney yunba stalkermn aminhp lucian stephb9959 cannadayr miloudeloumri qyon-brazil fnchooft

rec2json's Issues

Accessor funs

This parse transform generate additional accessor funs for each field in record. Is it needed for general use of from_json an to_json or it is just additional functionality?
I want to add option to disable accessors when i need to define my own and want to know that it won't break anything

It seems broken with atom

-record(main, {
          test :: atom
         }).

Record = #main{
            test = lalala
           },
Json = Record:to_json(),

** exception error: bad argument: {test,lalala,{specific,[undefined,atom]}}
     in function  rec2json:'-to_json/2-fun-0-'/3 (src/rec2json.erl, line 126)
     in call from lists:foldl/3 (lists.erl, line 1261)
     in call from rec2json:to_json/2 (src/rec2json.erl, line 118)
     in call from main:test/0 (main.erl, line 20)

Unprocessed error

Trying to convert below record to json:

#eth{src = "00:50:56:C0:00:08",dst = "01:00:5E:00:00:FB",
     type = ipv4,
     data = #ipv4{vsn = 4,hlen = 5,diffserv = 0,totlen = 73,
                  id = 17452,flags = [],frag_offset = 0,ttl = 255,proto = udp,
                  hdr_csum = correct,
                  src = <<172,16,216,1>>,
                  dst = <<224,0,0,251>>,
                  options = [],
                  data = #udp{src_port = <<"mdns">>,dst_port = <<"mdns">>,
                              length = 53,csum = correct,
                              data = [{dns_header,0,0,0,0,0,0,0,0,0},
                                      [{dns_query,"_ipp._tcp.local",ptr,in},
                                       {dns_query,"_ipps._ipps._tcp.local",ptr,in}],
                                      [],[],[]]}}}

But getting below error:

Unprocessed: eth,to_json,[{eth,"00:50:56:C0:00:08","01:00:5E:00:00:FB",ipv4,{ipv4,4,5,0,73,17452,[],0,255,udp,correct,<<172,16,216,1>>,<<224,0,0,251>>,[],{udp,<<"mdns">>,<<"mdns">>,53,correct,[{dns_header,0,0,0,0,0,0,0,0,0},[{dns_query,"_ipp._tcp.local",ptr,in},{dns_query,"_ipps._ipps._tcp.local",ptr,in}],[],[],[]]}}}],[]

Errors when a type mismatches on to_json are not helpful

When rec2json cannot convert a term to a json representation, it throws a exception of type {badarg, field_name(), field_value(), field_types()}. This is correct as far as adhering to erlang error output structure, but ends up writing something like the following to the console:

** exception error: bad argument: 
                    {products,
                        [{product,undefined,undefined,undefined,undefined,
                             undefined,undefined,undefined,undefined,
                             undefined,undefined,undefined,undefined,
                             undefined,undefined}],
                        {specific,
                            [undefined,
                             {list,{specific,[{product,product,[]}]}}]}}
     in function  rec2json:'-to_json/2-fun-1-'/3 (src/rec2json.erl, line 145)
     in call from lists:foldl/3 (lists.erl, line 1262)
     in call from rec2json:to_json/2 (src/rec2json.erl, line 135)

Sure, if you stare at that and the rec2json source code, or you know the rec2json error format above, it's slightly helpful, otherwise it's a mismash of wtf. Often, when this happens, it is not meant to be handled and recovered, but fixed by a developer. This means developer understanding of the error is more important that efficient consumption by a computer. Given that, the format should change:

erlang:error({badarg, {error_details, #{
    field_name => field_name(),
    field_value => field_value(),
    allow_any => boolean(),
    expected_types => [field_type()]}}})

With a completely faked error, the console output becomes:

** exception error: bad argument: 
                    {error_details,#{allow_any => false,
                                     expected_types => [undefined],
                                     field_name => name,
                                     filed_value => value}}

Use of a map because it looks nicer than a prophets, though if pre-18 is required, that can be waved.

issues generating json from list of nested records

Hi, i'm not sure if this is supported or not, but i'm having the following issue:

i'm learning erlang passing a Node.js service to it, but i'm stock trying to map relationships between records and mapping them to json; i found your awesome library and it works perfectly until i tried to map to json a has many relationship.
I have some question is it supported and i'm doing something wrong? or it isn't? if it isn't can you point me what i need to make it work? or maybe is a bad idea to map those kind of relationships in erlang? as i said i'm just learning but there isn't anything said anywhere.
thanks in advance.

-module (collection).
-compile([{parse_transform, rec2json}]).

-export ([new/5, add_product/2]).

-export_type([collection/0]).

-type status() :: active | inactive.

-record(collection, {
  id :: binary,
  title :: binary(),
  status :: status(),
  code  :: binary(),
  image  :: binary(),
  products :: [product:product()]
}).

-opaque collection() :: #collection{}.

-spec add_product(product:product(), collection()) -> collection().
add_product(Product, Collection) ->
  ProductList = Collection#collection.products,
  ProductSet = sets:from_list(ProductList),
  ProductSet2 = sets:add_element(Product, ProductSet),
  ProductList2 = sets:to_list(ProductSet2),
  Collection#collection{products = ProductList2}.

-module (product).
-compile([{parse_transform, rec2json}]).

-export ([new/14]).

-export_type([product/0]).

-record (product,
{
  sku_base :: binary(),
  title :: binary(),
  description :: binary(),
  short_description :: binary(),
  fabric :: binary(),
  care :: binary(),
  col = collection:collection(),
  colors :: list(),
  sheerness :: pos_integer(),
  measurements :: list(),
  images :: list(),
  thumbnail :: binary(),
  price :: pos_integer(),
  sizes :: list()
}).

-opaque product() :: #product{}.

Col1 = collection:new(<<"Col1">>, active, <<"SM16">>, <<"url">>, []).
Prod = product:new(<<"">>, <<"">>, <<"">>, <<"">>, <<"">>, <<"">>, Col1, [], 7, [], [], <<"">>, 4000, ["00"]).
Col2 = collection:add_product(Prod, Col1).
product:to_json(Prod).
[{sku_base,<<>>},
 {title,<<>>},
 {description,<<>>},
 {short_description,<<>>},
 {fabric,<<>>},
 {care,<<>>},
 {col,{collection,undefined,<<"Col1">>,active,<<"SM16">>,
                  <<"url">>,undefined,[]}},
 {colors,[]},
 {sheerness,7},
 {measurements,[]},
 {images,[]},
 {thumbnail,<<>>},
 {price,4000},
 {sizes,["00"]}]

collection:to_json(Col2).
** exception error: bad argument: {products,[{product,<<>>,<<>>,<<>>,<<>>,<<>>,<<>>,
                                                      {collection,undefined,<<"Col1">>,active,<<"SM16">>,
                                                                  <<"url">>,undefined,[]},
                                                      [],7,[],[],<<>>,4000,
                                                      ["00"]}],
                                            {specific,[undefined,
                                                       {list,{specific,[{product,product,[]}]}}]}}
     in function  rec2json:'-to_json/2-fun-1-'/3 (/home/kiro/Documents/Erlang/ec_api/_build/default/lib/rec2json/src/rec2json.erl, line 145)
     in call from lists:foldl/3 (lists.erl, line 1262)
     in call from rec2json:to_json/2 (/home/kiro/Documents/Erlang/ec_api/_build/default/lib/rec2json/src/rec2json.erl, line 135)

Allow to_json to return maps instead of proplists.

Maps represent a json object more performantly than proplists. Furthermore, checking for an empty object avoids the ugly [{}] hack. However, backwards compatibility is desirable, so we can't just swap, at least not without a major version change.

The options are:

Use a -d macro check to allow all rec2json transformed modules to use maps or not, thus altering an entire project at once.
use the -rec2json attribute and place an option there to switch use of maps on an individual module basis

How to use rec2json with a record that has a field with the same name?

If we add a field test_person :: binary() to the test_person record in file test/test_rec.hrl, we get an error when running make tests:

test/test_person.erl:260: function test_person/1 already defined
test/test_person.erl:1: Warning: function test_person/1 already exported

We are seeing this warning in our project that has a record with a field fo the same name. Is there a way to use rec2json with records that have a field with the same name? If so, how?

-Sincerely

undefined function

I have hrl file where i have defined the records.

-compile([{parse_transform, rec2json}]).

%% For user File location

-record(filelocation,{dc_id= 5 :: integer(),
                      volume_id= -1 :: integer(),
                      local_id= -1 :: integer(),
                      secret= -1 :: integer(),
                      ext= <<"">> :: binary()}).



-type(tl_fileLocationUnavailable() :: #filelocation{dc_id  :: integer(),
                                                    volume_id :: 0,
                                                    local_id :: 0,
                                                    secret  :: 0,
                                                    ext  :: binary()}).



%% For user photos

-record(photo,{photo_id = -1 :: integer(),
               photo_big=  #filelocation{} :: tl_fileLocation() | tl_fileLocationUnavailable(),
               photo_small= #filelocation{} :: tl_fileLocation() | tl_fileLocationUnavailable()}).

-type(tl_photoEmpty() :: #photo{photo_id     :: -1,
                                       photo_big ::  tl_fileLocationUnavailable(),
                                       photo_small :: tl_fileLocationUnavailable()
                                        }).

later i have another module where i am using those structures
as

Big = #filelocation{dc_id = BDc_id,local_id = BLocal_id,ext = BExt,secret = BSecret,volume_id = BVol_id},

Small = #filelocation{dc_id = SDc_id,local_id = SLocal_id,ext = SExt,secret = SSecret,volume_id = SVol_id},

P = #photo{photo_id = BMedia_id, photo_big = Big,photo_small = Small},
?DEBUG("to json photo  ~p",[P:to_json()]),

and i get error
mod_contacts:prepare_user(<<"cinoabnt.com">>,<<"6789asd122">>).
** exception error: undefined function filelocation:to_json/1

Please help @lordnull

Automatically create user defined type for record.

It is a common pattern to define a record, then define a type for the record, and then export the user defined type. This does not work nicely with to_json or from_son without also defining the appropriately
named functions.

To make development faster and work as expected, rec2json could automatically generate the type and functions.

A developer should be able to change the name of the type (and therefore the exported conversion functions), as well as pass in options to the to_json and from_json functions, such as how to handle null / undefined, and mutator functions.

question for use library

Hello

I'm sorry for post here
I have a problem for use the libray
When I execute rec2json:to_json(Record), it shows

exception error: undefined function user:field_types/0
in function rec2json:to_json/2 (src/rec2json.erl, line 116)

I think I may have missed some steps
Please help me how to resolve the problem
thank you

here is my code

-module(test_json).

-compile([{parse_transform, rec2json}]).

-record(user, {
        userid :: binary(),
        username :: binary()
}).

-export([test/0]).

test()->
    Record = #user{userid= <<"james">>, username= <<"james chen">>},
    io:format("record=~p~n", [Record]),
    rec2json:to_json(Record).

here is console

Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false]

Eshell V6.2  (abort with ^G)
1> io:format("~p~n", [code:get_path()]).
[".","/usr/local/lib/erlang/lib/kernel-3.0.3/ebin",
 "/usr/local/lib/erlang/lib/stdlib-2.2/ebin",
 .....
 "/Users/james/Develop/erlang_imports/deps/jsx/ebin",
 "/Users/james/Develop/erlang_imports/deps/meck/ebin",
 "/Users/james/Develop/erlang_imports/deps/proper/ebin",
 "/Users/james/Develop/erlang_imports/deps/protobuffs/ebin",
 "/Users/james/Develop/erlang_imports/deps/ranch/ebin",
 "/Users/james/Develop/erlang_imports/deps/rec2json-2.0.4/ebin",
 ok
2> test_json:test().
record={user,<<"james">>,<<"james chen">>}
** exception error: undefined function user:field_types/0
     in function  rec2json:to_json/2 (src/rec2json.erl, line 116)

Is there any option to set one field do not be translated?

Example:

-record(r, {
    i = 0 :: integer(),
    n :: ignore()
}).

a = #a{i = 0, n = xxx}

after a:to_json(),
generate:

{"i" : 0}

Error when convert nested list of records

Hello, My problem is i have these two records

-record (a, {
  id :: integer()
}

-record (b, {
  id :: integer(),
  a :: [#a{}]
}

It gives me error like this

{erlang,apply,[[a],from_json,[]],[]}

Is this the right syntax of list of records?
or it's not implmented in the lib?

types do not resolve to simplest type

Hi,

If I define a record like this:

-record( thing , 
{
 prop1 :: binary() ,
prop2 :: mylist()
}).

And in an include file I have defined

-type mylist :: [ integer() ].

-export_type ([mylist/0]).

I am finding that rec2json does not reduce mylist() to [integer()] and therefore never processes prop2.

Am I missing something or is this intended behaviour or should the be filed as an enhancement?

Thanks

Possible Error

Hi,
In context of "Json to record" what if I don't have a field that is specified in the record definition. How can I handle these scenario.

Am having an application which sends a very large Json, I try to minimize the size by deleting some fields of the Json depending on the operation. So even though I use the same Json definition some fields may be missing.

Any suggestion for the problem would be highly appreciable

Does not support rebar3?

Hi! I use rebar3.

===> Verifying dependencies...
===> Fetching rec2json ({git,"https://github.com/lordnull/rec2json",
                                    {tag,"3.2.0"}})
===> Dependency failure: Application rec2json not found at the top level of directory