This project is a dependency of Rebar 3. It has recently started failing bootstrap on Erlang/OTP 24 (master) due to the following warning:

/path/to/rebar3-src/_build/default/lib/erlware_commons/src/ec_cmd_log.erl:160: Warning: ambiguous call of overridden auto-imported BIF error/3
 - use erlang:error/3 or "-compile({no_auto_import,[error/3]})." to resolve name clash

Most likely this warning is relatively new but I did not spend any time trying to trace it to a specific OTP commit.

Hey guys,

As a vim user, I was wondering if you'd be open to a brief pull request that inserts the vim headers in the comments at the top to help with formatting.

Basically, somewhere in the comments at the top of each module, in order to be consistent with your current intendation, I'd want to add:

%% vim: shiftwidth=4 tabstop=4 expandtab

I know that emacs is the standard editor for erlang, but I like my vim :) Would you be open to this?

Thanks,

-Jesse

The README file should spell out the licensing of the code.

Various module seem to be proprietary (i.e. have no OSI approved licensing statement and therefore default to not being usable without violating the copyright). Ideally, these would be licensed under an open source license or alternatively the proprietary and OS components should be split into separate apps.

• commit b23bf73
• erlang/OTP : 21-rc2

For the file owned by root user, ec_file:coy fails if executed by non-root user.

Steps to reproduce:

(default) % rm -f a && sudo touch a && sudo chmod 755 a
(default) % ls -al a
-rwxr-xr-x  1 root  staff  0 Jun  7 14:07 a
(default) % ./rebar3 shell
===> Verifying dependencies...
===> Compiling erlware_commons
Erlang/OTP 21 [RELEASE CANDIDATE 2] [erts-10.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:0] [hipe] [sharing-preserving]

Eshell V10.0  (abort with ^G)
1> ec_file:copy("a", "b").
{error,{write_file_info_failed,eperm}}

This does not fail with OTP 20.3.4.

From the doc of 21-rc2, file_info record has been added some fields.

• http://erlang.org/documentation/doc-10.0-rc2/lib/kernel-6.0/doc/html/file.html#type-file_info
• http://erlang.org/doc/man/file.html#type-file_info

Therefore, this is caused by (potential) incompatibility of Erlang/OTP.
If you decide this is not a bug of erlware_commons, please just close ;)

For a library this isn't useful, it would be better to throw an
exception with relevent information.

or should I handle the includes myself?

  ec_compile:erl_source_to_core_ast("../samples/access1.erl").
  ** exception error: no match of right hand side value 
                {error,{999999,erl_parse,["syntax error before: ",[]]}}
 in function  ec_compile:'-erl_string_to_erl_ast/1-fun-1-'/2 (src/ec_compile.erl, line 100)
 in call from lists:foldl/3 (lists.erl, line 1261)
 in call from ec_compile:erl_string_to_core_ast/1 (src/ec_compile.erl, line 110)

If I remove the comment at the end, I get just error, which isn't that better.

If I inline the .hrl file, it works nicely.

I need to parse & compile .erl files into .core internal structure.

where access1.erl is

-module(access1).
-compile(export_all).
-include("rec1.hrl").
-spec build1(string()) -> #contact{}.
build1(Name) ->
    #contact{name=Name}.
access1(Contact) ->
    Contact#contact.name.
%% to_core() ->
%%     compile:file("../samples/access1.erl",[to_core]).

and rec1.hrl is

-record(contact, {
      name = "name_default" :: string(),
      age = -1 :: integer()
     }).

-record(contacts, {
      company = "company_default" :: string(),
      contacts = [] :: [#contact{}]
   }).


-record(device, {
      name = "name_default" :: string(),
      ip_address = "0.0.0.0" :: string()
     }).

 -type mail() :: #contact{} | #device{}.

I noticed that runmany_wrap doesn't actually call the function passed to it, it only returns it. Looks like some parens were dropped during the refactor (see https://github.com/erlware/erlware_commons/blob/master/src/ec_plists.erl#L878)

I think that's slipped through testing because the ec_plist map tests only test the default malt, which uses the local runmany local code path.

This seems straightforward, so I wrote a quick fix here: https://github.com/chef/erlware_commons/tree/ma/fix_for_ftmap. Happy to make it a PR if you like.

To see this in action?

mkdir a
touch a/x
cd a
ln -s x y

From Erlang:

ec_file:copy("a", "b", [recursive]).

Doing an ls -lah shows that the symlinks have not been preserved.

Is it supposed to do this? It does not seem right to me.

I get the following error when trying to build master or tag v0.17.0 with rebar3 on windows with
System version: Erlang/OTP 19 [erts-8.0] [64-bit] [smp:4:4] [async-threads:10]

Compiling _build/default/lib/erlware_commons/src/ec_plists.erl failed
_build/default/lib/erlware_commons/src/ec_plists.erl:220: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:226: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:250: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:256: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:279: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:285: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:300: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:305: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:326: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:336: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:343: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:355: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:361: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:372: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:384: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:397: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:403: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:418: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:427: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:438: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:450: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:459: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:472: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:484: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:522: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:590: syntax error before: '/'
_build/default/lib/erlware_commons/src/ec_plists.erl:625: syntax error before: '/'

_build/default/lib/erlware_commons/src/ec_plists.erl:209: type el_fun() is unused

The COPYING file has something that looks like an MIT license inside of it. Meanwhile hex.pm metadata states that erlware_commons is licensed under the Apache license.

It should use try catch, and probably should be moved to a ec_int
module or something along those lines. At the very least it should be
moved to try catch.

Running the dialyzer, i got the following message:

Line 110 Column 5: Unknown function compile:forms/2
Line 118 Column 5: Unknown function compile:forms/2

These messages comming from erlware_commons/src/ec_compile.erl

I'd like to ask this (master?) to be published to Hex.pm, in order for it to be imported into rebar3 as per one of the actions identified in erlang/rebar3#2515. Thanks.

Edit: master handles, among other stuff, erlang/rebar3#2515 (comment).

There isn't anything wrong with that, but we should be consistant
across erlware commons modules. We need to define an exception
handling bit, perferably across all of erlware_commons. Then port
everything over to the new apy.

Since we already have md5sum in erlware_commons, can we have sha1sum which is a better check for files nowadays ?

    lists:flatten(cf:format("~!" ++ [Color] ++"~s~s", [?PREFIX, Msg]));

in ec_cmd_log should probably be

    lists:flatten(cf:format("~!" ++ [Color] ++"~ts~ts", [?PREFIX, Msg]));

This is for rebar3 log (used from plugins)

I just upgraded one of my packages, exrm 1.0-rc7 -> rc8, and with it came a new dependency, erlware_commons. However, I can't seem to fetch it:

$ mix deps.get
Running dependency resolution
==> erlware_commons
Error evaluating rebar config script ./rebar.config.script:16: evaluation failed with reason error:{badmatch,undefined} and stacktrace [{erl_eval,expr,3,[]}]
Any dependency defined in the script won't be available unless you add them to your Mix project

There seems to be nothing wrong with rebar3 on my system:

$ rebar3 version
rebar 3.0.0-beta-2 on Erlang/OTP 18 Erts 7.2.1

Any clues as to what may be wrong? Is this even a problem? I'm using Elixir 1.2.2.

parses

> ec_date:parse("2017-01-30T10:15:00.100000Z").
{{2017,1,30},{10,15,0,100000}}

doesn't parse

> ec_date:parse("2017-01-30T10:15:00.100000+01:00").
** exception throw: {ec_date,{bad_date,"2017-01-30T10:15:00.100000+01:00"}}

Have forked. Will fix.

This is not a smart implementation, we should be pulling the tmp from
more well recognized sources. Also it would probably be nice if tmp
was exported as a public api once this occures.

Currently if you run any string with the check on type is will return directory. I am using this from elixir so it's something like this:

iex(1)> :ec_file.type('xxxyy')
:directory

It should at least use :filelib.is_dir to check on the Path that is being passed in.

As in erlang/docker-erlang-otp#327,
https://travis-ci.org/github/erlang/docker-erlang-otp/jobs/760303241
The error msg is :

/usr/src/rebar3-src/_build/default/lib/erlware_commons/src/ec_cmd_log.erl:160:5: Warning: ambiguous call of overridden auto-imported BIF error/3
 - use erlang:error/3 or "-compile({no_auto_import,[error/3]})." to resolve name clash

A new tag is needed to be made.

The parse clauses

parse([Year,X,Month,X,Day,Hour,$:,Min,$:,Sec,$., Ms | PAM], _Now, _Opts)
  when ?is_meridian(PAM) andalso
       (?is_us_sep(X) orelse ?is_world_sep(X))
       andalso ?is_year(Year) ->
    {{Year, Month, Day}, {hour(Hour, PAM), Min, Sec}, {Ms}};

and

parse([Year,X,Month,X,Day,Hour,$:,Min,$:,Sec,$., Ms], _Now, _Opts)
  when  (?is_us_sep(X) orelse ?is_world_sep(X))
        andalso ?is_year(Year) ->
    {{Year, Month, Day}, {hour(Hour,[]), Min, Sec}, {Ms}};

in the file ec_date.erl (lines 155 to 178) seem to be equivalent, since they both match lists without a value for PAM.

6> [Year,X,Month,X,Day,Hour,$:,Min,$:,Sec,$., Ms | PAM] = [2014,$-,8,$-,28,23,$:,39,$:,10,$.,1].   
[2014,45,8,45,28,23,58,39,58,10,46,1]
7> PAM.
[]
8> [Year,X,Month,X,Day,Hour,$:,Min,$:,Sec,$., Ms] = [2014,$-,8,$-,28,23,$:,39,$:,10,$.,1].
[2014,45,8,45,28,23,58,39,58,10,46,1]

In the first case PAM is set to [] which would pass the ?is_meridian(PAM) guard

-define( is_meridian(X), (X==[] orelse X==[am] orelse X==[pm]) ).

and result in hour(Hour, PAM) being equivalent to hour(Hour, []).

This affects the four clauses below (lines 173, 176, 184, 187), too. The only difference is that
in this case the first two clauses use the ?is_year(Year) guard and the latter two ?is_month(Month).

Note: I am not too familiar with Erlang and might be missing something.

In the format parameter string, 'G' should have represented 24-hour format of an hour without leading zeros.
But it actually results in 24-hour format of an hour with leading zeros.

erlware_commons version: "1.3.1"
rebar 3.14.2+build.4877.reffb15036e on Erlang/OTP 23 Erts 11.1.1

<span style="background-color:#97ea9799">===> Performing cover analysis... </b></span><span style="background-color:#ff3434b3">===> <b>Uncaught error in rebar_core. Run with DIAGNOSTIC=1 to see stacktrace or consult rebar3.crashdump </b></span></b></span>escript: exception error: bad argument in function io_lib:format/2 called as io_lib:format("~!c~s~s</b></span>", ["===> ","Uncaught error: ~p~n"]) in call from ec_cmd_log:colorize/4 (/root/butler_envs/rebar/_build/test/lib/erlware_commons/src/ec_cmd_log.erl, line 254) in call from ec_cmd_log:debug/3 (/root/butler_envs/rebar/_build/test/lib/erlware_commons/src/ec_cmd_log.erl, line 132) in call from rebar3:handle_error/2 (/root/lib/rebar3/src/rebar3.erl, line 357) in call from escript:run/2 (escript.erl, line 758) in call from escript:start/1 (escript.erl, line 277) in call from init:start_em/1 in call from init:do_boot/3

I've used this package for a while but only just noticed that (last checked 874f2dc)

1> ec_date:format_iso8601({{2014,10,12},{3,55,23}}).
"2014-10-12T3:55:23Z"

This is incorrect. It needs to return

"2014-10-12T03:55:23Z"

because the spec says that "each date and time value has a fixed number of digits that must be padded with leading zeros".

The ec_string module needs to be renamed to ec_version or something
along those lines. It actually has little or nothing to do with
strings aside from the fact that it parses them in a specific way.

Hi,

btw... imho there is a security bug/problem in erlware-commons/ec_file.
The function in question is mkdtemp which calls mkdir_path but which is not
using O_EXCL flag to create the temp directory, thus making it prone to link high-jacking.
Additionally it uses a completely monotone and predictable random-part which
would make it quite an easy target.
It also doesn't error out if the directory already exists, thus an attacker could highjack
the mkdtemp call without the application noticing and possible get to sensitive data.
Not setting permissions to 0700 for the temp dir is also something unexpected.

Imho if the name of a function is the same as from a standard defined function with special
security guarantees it's deadly to have the same name but none of the security guarantees.

michael

Running the dialyzer, i got the following message:

Line 248 Column 1: The pattern <State, Color, 'true', Msg> can never match the type <,99 | 103 | 109 | 114,'false',>
Line 253 Column 1: The pattern <State, Color, 'true', Msg> can never match the type <,99 | 103 | 109 | 114,'false',>

These messages comming from erlware_commons/src/ec_cmd_log.erl

Erlang/OTP 26

I think it would make erlware commons much more approachable for those
that have not used it if the readme, displayed on github, had a list
of or at least a partial list of the functionality found within.

Thanks :)

It seems there is no tag 1.0.5 . Hex has the version: erlware_commons | Hex
https://hex.pm/packages/erlware_commons/1.0.5

9eae901 is the one?

This may be the case but is by no means required. I think
ec_file:consult needs to follow on of the following paths.

• Return all terms in the file
• return a single term but be renamed to something like ec_file:consult_one

the one off behavior here needs to be obvious and documented.

The receivefrom function (https://github.com/erlware/erlware_commons/blob/master/src/ec_plists.erl#L771) matches all DOWN messages, not only from spawned processes (BadPid instead of Pid).

receivefrom(Pid) ->
    receive
        {Pid, R} ->
            R;
        {'DOWN', _, _, BadPid, Reason} when Reason =/= normal ->
            erlang:throw({BadPid, Reason});
        {timerrang, _} ->
            erlang:throw({nil, timeout})
    end.

This results e.g. ec_plists:map failure when DOWN from other sources is in message queue.

Ref from {Pid, _} = erlang:spawn_monitor(F), is also ignored (https://github.com/erlware/erlware_commons/blob/master/src/ec_plists.erl#L755).

I was trying to go through some of the libraries which rebar3 depends on, and it uses hex packages, so I was looking at the dependency in rebar3 and it has version 0.19.0 of erlware_commons, but I see that the .app.src in master has version 0.18.0, and the last tagged version is v0.17.0. The most recent version on hex.pm is labeled as 0.20.0. Is there a reason for all the discrepancies? I'd like to be able to override the rebar3 config file to pull all packages from source, but am unable to determine the actual source corresponding to 0.19.0 as listed.

The rebar.config.script does not work with the latest version of rebar3, since rebar3 has changed how the vsn application variable is formatted - it is no longer Major.Min.Patch-Hotfix but has changed to Major.Min.Patch+hash. This is not handled in the rebar.config.script file as it is now.

syntax error when running make on 17.0:

$ make
expr: syntax error
...

The syntax error is in this line:
https://github.com/erlware/erlware_commons/blob/master/Makefile#L31

It seems that last commit broke project compiling.

.../portal/rebar compile
WARN:  Expected .../portal/deps/cf to be an app dir (containing ebin/*.app), but no .app found.
==> erlware_commons (compile)
WARN:  Expected .../portal/deps/cf to be an app dir (containing ebin/*.app), but no .app found.
Dependency not available: cf-.* (undefined)
ERROR: compile failed while processing .../portal/deps/erlware_commons: rebar_abort
make: *** [compile-full]

With R15B01 and erlware_commons 0.8.0, rebar doc fails with the following error message:
==> erlware_commons-0.8.0 (doc)
2990- fatal: {invalid_name," 3.0.0"}
./src/ec_semver.erl, function pes/2: at line 134: error in XML parser: {fatal,{{invalid_name," 3.0.0"},
{file,file_name_unknown},
{line,144},
{col,45}}}.
edoc: skipping source file './src/ec_semver.erl': {'EXIT',error}.
edoc: error in doclet 'edoc_doclet': {'EXIT',error}.
ERROR: doc failed while processing /var/tmp/portage/dev-libs/erlware_commons-0.8.0/work/erlware_commons-0.8.0: {'EXIT',error}

Can we change
https://github.com/erlware/erlware_commons/blob/master/src/ec_cmd_log.erl#L240
into dumb as default?

Currently the message output via the logging module on windows produces a lot of garbage because it tries to colorize.

←[0;32m===> Verifying dependencies...
←[0m←[0;32m===> Linking d:/gitrepos/rebar3/_build/default/lib/erlware_commons to
 d:/gitrepos/rebar3/_build/bootstrap/lib/erlware_commons
←[0m←[0;32m===> Linking d:/gitrepos/rebar3/_build/default/lib/getopt to d:/gitre
pos/rebar3/_build/bootstrap/lib/getopt
←[0m←[0;32m===> Linking d:/gitrepos/rebar3/_build/default/lib/mustache to d:/git
repos/rebar3/_build/bootstrap/lib/mustache
←[0m←[0;32m===> Linking d:/gitrepos/rebar3/_build/default/lib/providers to d:/gi
trepos/rebar3/_build/bootstrap/lib/providers
←[0m←[0;32m===> Linking d:/gitrepos/rebar3/_build/default/lib/relx to d:/gitrepo
s/rebar3/_build/bootstrap/lib/relx
←[0m←[0;32m===> Compiling rebar
←[0m←[0;32m===> Cleaning out erlware_commons...
←[0m←[0;32m===> Cleaning out getopt...
←[0m←[0;32m===> Cleaning out mustache...
←[0m←[0;32m===> Cleaning out providers...
←[0m←[0;32m===> Cleaning out rebar...
←[0m←[0;32m===> Cleaning out relx...
←[0m←[0;32m===> Cleaning out rebar...
←[0m←[0;32m===> Verifying dependencies...
←[0m←[0;32m===> Compiling getopt
←[0m←[0;32m===> Compiling providers
←[0m←[0;32m===> Compiling erlware_commons
←[0m←[0;32m===> Compiling mustache
←[0m←[0;32m===> Compiling relx
←[0m←[0;32m===> Compiling rebar

Thats not very useful, we should gather the error up and throw an
exception with a valid reason if it fails.

https://semver.org contains the following statement regarding the semantic version spec for "pre-release identifiers":

9. A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphen [0-9A-Za-z-]. Identifiers MUST NOT be empty. Numeric identifiers MUST NOT include leading zeroes. Pre-release versions have a lower precedence than the associated normal version. A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92.

Notably, ec_semver:parse() correctly parses all four of the above examples:

3> ec_semver:parse(<<"1.0.0-alpha">>).
{{1,0,0},{[<<"alpha">>],[]}}
4> ec_semver:parse(<<"1.0.0-alpha.1">>).
{{1,0,0},{[<<"alpha">>,1],[]}}
5> ec_semver:parse(<<"1.0.0-0.3.7">>).
{{1,0,0},{[<<"0">>,3,7],[]}}
6> ec_semver:parse(<<"1.0.0-x.7.z.92">>).
{{1,0,0},{[<<"x">>,7,<<"z">>,92],[]}}

However, a few [hex] packages contain hyphens in the "pre-release identifer" portion of the semantic version, e.g. https://hex.pm/packages/xclient/0.7.0-vendored-xhttp, and this breaks the parser:

7> ec_semver:parse(<<"0.7.0-vendored-xhttp">>).
{<<"0.7.0-vendored-xhttp">>,{[],[]}}

I've found that simply changing <<"[A-Za-z0-9]">> to <<"[A-Za-z0-9-]">> in src/ec_semver_parser.erl seems to address the problem, although I haven't reviewed the code sufficiently to say that this is the most appropriate fix.

erlware_commons/src/ec_plists.erl:483: type dict/0 is deprecated and will be removed in OTP 18.0; use use dict:dict/0 or preferably dict:dict/2

nivertech@d8569db

Running rebar doc on a clean checkout of master using Erlang version R15B01 gives the following errors.

➜  erlware_commons git:(master) rebar doc
==> erlware_commons (doc)
./src/ec_rbdict.erl, in module header: at line 55: unknown error parsing reference: {56,edoc_parser,
                                  ["syntax error before: ","'-'"]}.
edoc: skipping source file './src/ec_rbdict.erl': {'EXIT',error}.
./src/ec_orddict.erl, in module header: at line 10: unknown error parsing reference: {11,edoc_parser,
                                  ["syntax error before: ","'-'"]}.
edoc: skipping source file './src/ec_orddict.erl': {'EXIT',error}.
./src/ec_gb_trees.erl, in module header: at line 9: unknown error parsing reference: {10,edoc_parser,
                                  ["syntax error before: ","'-'"]}.
edoc: skipping source file './src/ec_gb_trees.erl': {'EXIT',error}.
./src/ec_dictionary.erl, function get/2: at line 86: unknown error parsing throws-declaration: {86,edoc_parser,
                                           ["syntax error before: ","the"]}.
edoc: skipping source file './src/ec_dictionary.erl': {'EXIT',error}.
./src/ec_dict.erl, in module header: at line 10: unknown error parsing reference: {11,edoc_parser,
                                  ["syntax error before: ","'-'"]}.
edoc: skipping source file './src/ec_dict.erl': {'EXIT',error}.
./src/ec_assoc_list.erl, in module header: at line 8: unknown error parsing reference: {9,edoc_parser,
                                  ["syntax error before: ","'-'"]}.
edoc: skipping source file './src/ec_assoc_list.erl': {'EXIT',error}.
edoc: error in doclet 'edoc_doclet': {'EXIT',error}.
ERROR: doc failed while processing /Users/tim/code/erlang/shoutcast_otp/deps/erlware_commons: {'EXIT',error}

It looks like there are 2 classes of errors:

1. @see tags pointing to invalid references, either ec_application or dict
2. @throws tags with extra information after the type.

When a major/minor/patch version is an alpha the comparison should be done using alpha ordering. Many projects have made the mistake of using git refs as their version. While we don't want to encourage this, we do need to support things that are in the wild.

This change should not affect the correct evaluation of versions that are proper semver versions.