fogfish / datum Goto Github PK

pure functional and generic programming for Erlang

License: Apache License 2.0

Makefile 1.87% Erlang 98.13%

foldable traversable erlang lenses category-pattern data-structures streams algebraic-data-types monads generic-programming

datum's Introduction

datum : "scrap your boilerplate" for Erlang

You could do this with a macro, but... the best macro is a macro you don't maintain

datum is a pure functional and generic programming for Erlang. It had its origins in Purely Functional Data Structures by Chris Okasaki, on implementing a various higher rank functional abstractions and patterns, on dealing with scrap your boilerplate and gaining experience from other functional languages primary Scala and Haskell. The library is still testing the limits of functional abstractions in Erlang.

Key features

The feature overview provides an introduction to datum features, use-cases and reasoning of they existence:

option and either type notations
a set of generic data types that can be inspected, traversed, and manipulated with common behavior: foldable, traversable and map-like.
pure functional data types: binary search tree, red-black tree, heap, queues, and others
streams or lazy lists are a sequential data structure that contains on demand computed elements.
resembles concept of getters and setters (lens) for complex algebraic data types.
mapping of algebraic data types to they generic representation and back
define a category pattern, monads and they composition for Erlang applications. You might be familiar with this concept as pipe, flow or function composition.
generic do-notation with pattern matching.
typecasts of primitive data types
supports OTP/18.x or later release

Getting started

The latest version of the library is available at its master branch. All development, including new features and bug fixes, take place on the master branch using forking and pull requests as described in contribution guidelines.

The stable library release is available via hex packages, add the library as dependency to rebar.config

{deps, [{datum}]}.

Please follow the feature overview to start leaning all available features; then continue to library examples and to source code.

How To Contribute

The library is Apache 2.0 licensed and accepts contributions via GitHub pull requests:

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Added some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

The development requires Erlang/OTP version 19.0 or later and essential build tools.

Build and run service in your development console. The following command boots Erlang virtual machine and opens Erlang shell.

git clone https://github.com/fogfish/datum
cd datum
make
make run

commit message

The commit message helps us to write a good release note, speed-up review process. The message should address two question what changed and why. The project follows the template defined by chapter Contributing to a Project of Git book.

Short (50 chars or less) summary of changes

More detailed explanatory text, if necessary. Wrap it to about 72 characters or so. In some contexts, the first line is treated as the subject of an email and the rest of the text as the body. The blank line separating the summary from the body is critical (unless you omit the body entirely); tools like rebase can get confused if you run the two together.

Further paragraphs come after blank lines.

Bullet points are okay, too

Typically a hyphen or asterisk is used for the bullet, preceded by a single space, with blank lines in between, but conventions vary here

bugs

If you experience any issues with the library, please let us know via GitHub issues. We appreciate detailed and accurate reports that help us to identity and replicate the issue.

Specify the configuration of your environment. Include which operating system you use and the versions of runtime environments.
Attach logs, screenshots and exceptions, in possible.
Reveal the steps you took to reproduce the problem, include code snippet or links to your project.

License

datum's People

Contributors

Stargazers

Watchers

Forkers

linearregression brigadier franksongfeng barleyj gitter-badger krzysiekj abhilashr2020

datum's Issues

Document recursion flow with category/monads

E.g. How to implement following code using m_state

sync_github(#account{} = Account, Page) ->
   case m_http:once( sync_github_at(Account, Page) ) of
      {ok, []} ->
         {ok, []};
      {ok, Head} ->
         case sync_github(Account, Page + 1) of
            {ok, Tail} ->
               {ok, Head ++ Tail};
            {error, _} = Error ->
               Error
         end;
      {error, _} = Error ->
         Error
   end.

sync_github(#account{} = Account, Page) ->
   [m_state ||
      Head <- sync_github_at(Account, Page),
      Tail <- next(Head, Account, Page),
      cats:unit(Head ++ Tail)
   ].

next([], _, _) ->
   fun(State) -> [[] | State] end;
next(_, Account, Page) ->
   sync_github(Account, Page + 1).

partial application do not support multiple arguments

lists:fold(a(_, _), ...)

fails with

terminated with reason: fun called with wrong arity of 2 instead of 1

Export datum:lens() type

Generic Codec

We do have a ADT - Erlang record

-record(adt, {id, title, ...}).

external serialization works with maps

jsx:encode(#{<<"id">> => ..., <<"title">> => ...})

How to automate process of ADT to maps and back transformation

-ifndef(decode).
-define(decode(Type, Struct),
   list_to_tuple([Type | 
      [lens:get(lens:at(typecast:s(X)), Struct) || X <- record_info(fields, Type)]]
   )
).
-endif.

-ifndef(encode).
-define(encode(Type, Struct),
   maps:from_list(
      [{typecast:s(Key), Value} ||
         {Key, Value} <- lists:zip(
            record_info(fields, Type),
            tl(tuple_to_list(Struct))
         ),
         Value /= undefined,
         Value /= null
      ]
   )
).
-endif.

Evaluate usage of Concrete Conditional and Abstract Conditional for lenses

Lenses to facilitate unit tests of complex structures

define flatmap for streams

stream:flatmap(
   fun(X) -> stream:build([X, X + 1, X * 2]) end
  stream:build([1,2,3])
).

flatmap produces continues stream

[1,2,2,2,3,4,3,4,6]

Fails to compile nested lists comprehension

   [either ||
      cats:unit(binary:split(Pckt, <<$\n>>, [trim, global])),
      cats:sequence([decode(X) || X <- _]),
      cats:sequence(upstream(FMap, _))
   ]

gives an error

variable '_' is unbound

add monad utility functions

sequence

dynamic builder of do notation

lift

lift scalar type monoid to monads, syntax sugar

do([Monad ||
   X <- Mx,
   Y <- My,
   return (X + Y)  %% + is monoid
])

Product lens to spawn multiple fields

A = lens:p({ lens_name(), lens_age(), lens_city() }).
{"Verner Pleishner", 64, "Berne"} = lens:get(lens_name_age_city(), person()).

B = lens:p([ lens_name(), lens_age(), lens_city() ]).
["Verner Pleishner", 64, "Berne"] = lens:get(lens_name_age_city(), person()).

Use product lenses for isomorphism

Backport Pattern from release 3.x due matching causes error

The following example fails to compile:

-module(monad_test).

-compile({parse_transform, category}).

-export(
   [foo/0]).

foo() ->
   [m_identity ||
      {A, B} <- unit({100, 101}),
      unit(A)
   ].

The error is:

error in parse transform 'category': {function_clause,
                                      [{datum_cat_kleisli,'.',
                                        [m_identity,
                                         {monad,'_Vx56',
                                          {call,11,
                                           {remote,11,
                                            {atom,11,m_identity},
                                            {atom,11,'>>='}},
                                           [{call,11,
                                             {remote,11,
                                              {atom,11,m_identity},
                                              {atom,11,unit}},
                                             [{var,11,'A'}]},
                                            {'fun',11,
                                             {clauses,
                                              [{clause,11,
                                                [{var,11,'_Vx55'}],
                                                [],
                                                [{call,11,
                                                  {remote,11,
                                                   {atom,11,m_identity},
                                                   {atom,11,unit}},
                                                  [{var,11,'_Vx55'}]}]}]}}]}},
                                         {generate,10,
                                          {tuple,10,
                                           [{var,10,'A'},{var,10,'B'}]},
                                          {call,10,
                                           {remote,10,
                                            {atom,10,m_identity},
                                            {atom,10,unit}},
                                           [{tuple,10,
                                             [{integer,10,100},
                                              {integer,10,101}]}]}}],
                                        [{file,
                                          "src/category/datum_cat_kleisli.erl"},
                                         {line,21}]},
                                       {datum_cat,join,3,
                                        [{file,"src/category/datum_cat.erl"}, 
                                         {line,274}]},
                                       {datum_cat,category,3,
                                        [{file,"src/category/datum_cat.erl"},
                                         {line,71}]},
                                       {category,exprs,1,
                                        [{file,"src/category/category.erl"},
                                         {line,381}]},
                                       {category,clause,1,
                                        [{file,"src/category/category.erl"},
                                         {line,139}]},
                                       {category,clauses,1,
                                        [{file,"src/category/category.erl"},
                                         {line,130}]},
                                       {category,function,3,
                                        [{file,"src/category/category.erl"},
                                         {line,124}]},
                                       {category,form,1,
                                        [{file,"src/category/category.erl"},
                                         {line,81}]}]}

Tested on 4.3.3. If the line containing pattern matching is replaced with A <- unit({100, 101}), there is no longer a compilation error.

Unify interface for bst and rbtree

Supports category transformer in nested expressions.

As a developer
I want to use a category transformers (cats:) within nested expressions
So that the code block is reusable when Type of category is changed.

nested_cats() ->
   [either ||
      X <- one(),
      Y <- two(),
      sum(cats:unit(X), cats:unit(Y))
   ].

one() ->
   {ok, 1}.

two() ->
   {ok, 2}.

sum({ok, X}, {ok, Y}) ->
   {ok, X + Y}.

code fails with

([email protected])1> t:nested_cats().
** exception error: undefined function cats:unit/1

The usage of nested transformers helps to build inline expressions as part of composition.

export lens() type.

Lens crash if focus is not defined.

Here is the behaviour

lens:put(lens:at(<<"id">>), 1, #{}).
** exception error: {badkey,<<"id">>}
     in function  lens:'-at/1-anonymous-1-'/3
     in call from lens:put/3

lens:put(lens:at(<<"id">>, undefined), 1, #{}).
#{<<"id">> => 1}

The investigation is required about lenses without Omega value:

Is it type safe to crash if field is not know at structure?
Is it usable to demand Omega value for each lens?
Should lens return optional type always?

supporting legacy rebar

Hello,

Do you think it's possible to still support the legacy rebar ? Almost all projects still support it.

Right now on OTP 20 and rebar this fails:

src/maplike/heap.erl:19: behaviour maplike undefined
Compiling src/maplike/heap.erl failed:
ERROR: compile failed while processing /home/silviu/Desktop/projects/erlkaf/deps/datum: rebar_abort

Removing warnings_as_errors fix this for the legacy rebar

Silviu

Update documentation about do-notation

The documentation is missing

monad interface specification
transformers
explanation and examples about existing monads

Use undefined for empty streams.

streams module uses {} for empty (undefined) stream.

It requires explicit pattern match before usage

do({}) -> stream:new();
do(Stream) -> stream:head(Stream).

The usage of undefined makes stream compatible with maybe (optional) category.

IO category transformers

%% we have a function that returns io monad
io() ->
   [m_io ||
      ...
   ].

%% we need to compose this function with other category (e.g. reader)
dot() ->
  [reader ||
     Result =< io()
     ...
  ].

The Result is IO function. We need to design a transformer to evaluate this function to actual IO result or fail.

built-in value fmap to xor category

case f() of
  {ok, _} -> ...
  {error, _} -> ...
  ok -> {ok, undefined}
  X  -> {ok, X}
end

Implicit return for do-notation

do([m_maybe || 
   _ <- a(), 
   b(_)
]).

instead of

do([m_maybe || 
   _ <- a(), 
   _ <- b(_),
  return(_)
]).

Missing identity lens

wrong Mio signature

See https://wiki.haskell.org/IO_inside
The IO function should return a new RealWorld state.

add common test for tree(s)

Support MFA Generator with stream:unfold

Improve lens interface

Introduce following features:

lens:at/1 read, write the value associated with a key in a Map-like containers. ~~keylists, maps~~.
deprecate lens:map/1 ~~and create lens:maps/1~~
~~deprecate lens:pair/1~~
improve functions that takes a scalar or function as focus.
lens:traverse/1

Enable arbitrary inline expression as part of category composition.

How to efficiently implement following example with category composition.

if_safe_div_zero_2(X, Y, Fun) ->
    [option ||
        Result <- case Y == 0 of
                      true  -> fail("Cannot divide by zero");
                      false -> unit(X / Y)
                  end,
        unit(Fun(Result))
    ].

add unique filter as example algorithm to stream_SUITE

%% remove duplicated elements
-spec(unique/1 :: (datum:stream()) -> datum:stream()).

unique({s, Head, _}=Stream) ->
   stream:new(Head, fun() -> unique(dropwhile(fun(X) -> X =:= Head end, Stream)) end);
unique({}) ->
   stream:new().

Category compilation fails with `=<` and `case` as first statement.

Example code

if_safe_div_zero_3(X, Y, Fun) ->
   [option ||
      % io:format("--> ~n"),
      Result =< case Y == 0 of
         true  -> ?None;
         false -> X / Y
      end,
      unit(Fun(Result))
   ].

Gives an error

error in parse transform 'category': {function_clause,
                                      [{datum_cat,is_partial,
                                        [{'case',10,
                                          {op,10,'==',
                                           {var,10,'Y'},
                                           {integer,10,0}},
                                          [{clause,11,
                                            [{atom,11,true}],
                                            [],
                                            [{atom,11,undefined}]},
                                           {clause,12,
                                            [{atom,12,false}],
                                            [],
                                            [{op,12,'/',
                                              {var,12,'X'},
                                              {var,12,'Y'}}]}]}],
...

However, it is able to compile this code

if_safe_div_zero_3(X, Y, Fun) ->
   [option ||
      do_something(...),
      Result =< case Y == 0 of
         true  -> ?None;
         false -> X / Y
      end,
      unit(Fun(Result))
   ].

Lenses for binary

<<"prefix", Focus/binary>>

<<Skip:16/binary, Focus:16/binary, _/binary>> 

<<Focus:16/binary, _/binary>>

Undefined category

The category is not option. It exists if function returns defined value

[undefined ||
  a(), b(), c()
]

Invalid coverage report for category structure

The head structure of category comprehension takes a wrong line number.

        |  'IDLE'({connect, Uri}, Pipe, #state{} = State0) ->
     1..|     case 
        |        [either ||
     0..|           connect(Uri, State0),
     1..|           time_to_live(_),
     1..|           time_to_hibernate(_),
     1..|           time_to_packet(0, _),
     1..|           pipe_to_side_a(Pipe, established, _),
     1..|           config_flow_ctrl(_)
        |        ]

Pattern match an empty data structure

The library has used {} as empty structure. Recently, structures has been replaces with #queue{} explicit definition. We need a tools (approach) to match empty empty structures in the functions.

e.g.

do_somthing(?empty_queue()) -> ...

define NONE macro

Define a global NONE macro instead of type specific NULL definition.

Unify interface of 'collection' types

Validate feasibility of defining a behavior as 'interface' umbrella for collections.
Check if we can abstract map, fold, dropwhile, split, splitwith, etc wishing this interface.

Required for each datatype

map
flatmap

Category transformers are not defined for monads

[m_state ||
   ...
   cats:eitherT( ... )
   ..
].

fails with

{undef,
   [{m_state,eitherT, ...

Missing constant lens

tryT a category transformer

A new category transformer to catch exceptions.

E.g. the following code fails if input is not valid json

[either ||
   ...
   cats:unit(jsx:decode(_))
]

some one might protect it with

[either ||
   ...
   cats:unit(catch jsx:decode(_))
]

Usage of tryT transformer that is able to catch error and inject it into "sequence" simplify error handling.

[either ||
   ...
   cats:tryT(jsx:decode(_))
]

a similar to Try is Scala

Variable '_Vx44' exported from 'case'

t() ->
   [option ||
      A =< case 1 of _ -> 1 end,
      B =< case 1 of _ -> 3 end,
      cats:unit(A + B)
   ].

Any match at case generates warning:

variable '_Vx44' exported from 'case' (line NN)

 30 t() ->
 31     case
 32         datum_cat_option:unit(case 1 of
 33                                   _Vx44 ->
 34                                       1
 35                               end)
 36     of
 37         undefined ->
 38             undefined;
 39         A ->
 40             case
 41                 datum_cat_option:unit(case 1 of
 42                                           _Vx44 ->
 43                                               3
 44                                       end)
 45             of
 46                 undefined ->
 47                     undefined;
 48                 B ->
 49                     datum_cat_option:unit(A + B)
 50             end
 51     end.

Define iostream, compatible with stream interface

support Ordering functor at tree(s)

bst
rbtree
htree

Typecast lenses

Support a typecast operation via lens

e.g. parsing AWS API response

lens:c(
      lens:traverse(),
      lens:pair(instances_set),
      lens:traverse(),
      lens:pair(instance_id)
     %% here we need a lens to convert list to binary
   ).

Improve the documentation of =< operator in context of category pattern.

The documentation and examples requires better description of cats:unit(...) and =< operator.
We need to improve the documentation at Section: Composition with state. Additionally, create a dedicated section to define a role of cats:unit(...).