Generated elixir_make app
==> crc
C checksum_xor.c
In file included from /app/deps/crc/c_src/nif/checksum_xor.c:4:
/app/deps/crc/c_src/nif/checksum_xor.h:7:10: fatal error: stdlib.h: No such file or directory
7 | #include <stdlib.h>
| ^~~~~~~~~~
compilation terminated.
make: *** [Makefile:163: /app/_build/prod/lib/crc/obj/checksum_xor.o] Error 1
could not compile dependency :crc, "mix compile" failed. You can recompile this dependency with "mix deps.compile crc", update it with "mix deps.update crc" or clean it with "mix deps.clean crc"

Since the NIFs are load via erlang files and not elixir now, they are being auto loaded. This breaks cross compiling the module for a nerves project.

Hello,

According to this paper, it is possible to detect errors with a longer hamming distance with CRC-32 by using other polynomial than the usual one.

Do the implementation use the suggestion of the paper?

Where can I find a documentation of hamming distance errors for the various CRC implemented?

Would it make sense to add that to the README?

For small binaries an elixir implementation can be more performant especially for nerves devices.

If we implement non-NIF version for CRC algorithm we could add an optional parameter to use the non-NIF calculation for users who know their payloads are small and would prefer to not load the NIF.

During implementation we have the NIFs to unit test against.

If useful we could also look at adding an optional threshold param to use vs not use the NIF.

Goals:

• pure elixir CRC algorithm implementation
• Support for all models the NIF supports.
• Unit test elixir calculation with NIF implementation.
• crc/2 & calculate/2 takes optional param to choose not to use NIF, should default to current behavior of using NIF
• ☝️ default switched to not use NIF

Hello, I'm try to compile it on Windows 10 using visual studio 2019 pre , Program Maintenance Utility Version is 14.20.27607.1 , something wrong with " cannot open file '..\priv\crc_nif.dll'" and no dll file was generated in floder , logs detail as follows

Copyright (C) Microsoft Corporation.  All rights reserved.

        echo # Auto-generated as part of Makefile.win, do not modify. > Makefile.auto.win
        erl -eval "io:format(\"~s~n\", [lists:concat([\"ERTS_INCLUDE_PATH=\", code:root_dir(), \"/erts-\", erlang:system_info(version), \"/include\"])])" -s init stop -noshell >> Makefile.auto.win
        nmake /                   /F Makefile.win app

Microsoft (R) Program Maintenance Utility Version 14.20.27607.1
Copyright (C) Microsoft Corporation.  All rights reserved.

        cl /O2 /EHsc /I"c:/Program Files/erl10.1/erts-10.1/include" /LD /MD /Fe"..\priv\crc_nif.dll"  nif\xnif_slice.c nif\checksum_xor.c nif\crc_8.c nif\crc_nif.c nif\crc_algorithm.c nif\crc_model.c nif\crc_resource.c
                      
Microsoft (R) C/C++ Optimizing Compiler Version 19.20.27607.1 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

xnif_slice.c
checksum_xor.c
crc_8.c
nif\crc_8.c(37): warning C4333: '>>': right shift by too large amount, data loss
crc_nif.c
crc_algorithm.c
c:\myprojectpath\deps\crc\c_src\nif\crc_algorithm_fast.c.h(60): warning C4333: '>>': right shift by too large amount, data loss                                                                                     crc_model.c
crc_resource.c
Generating Code...

Microsoft (R) Incremental Linker Version 14.20.27607.1
Copyright (C) Microsoft Corporation.  All rights reserved.

/dll
/implib:..\priv\crc_nif.lib
/out:..\priv\crc_nif.dll
xnif_slice.obj
checksum_xor.obj
crc_8.obj
crc_nif.obj
crc_algorithm.obj
crc_model.obj
crc_resource.obj
LINK : fatal error LNK1104: cannot open file '..\priv\crc_nif.dll'
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.20.27607\bin\HostX86\x86\cl.EXE"' : return code '0x2'                                                                                 Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.20.27607\bin\HostX86\x86\nmake.EXE"' : return code '0x2'
Stop. 

What should I do to fix this ?

Error

noven/deps/crc on  main [!] is 📦 v0.9.2 via 💧 1.11.0 (OTP 23) via  23.2 
🕙 [ 19:21:17 ] ❯ elixir -v
Erlang/OTP 23 [erts-11.1.4] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe]

Elixir 1.11.0 (compiled with Erlang/OTP 23)

noven/deps/crc on  main [!] is 📦 v0.9.2 via 💧 1.11.0 (OTP 23) via  23.2 
🕙 [ 19:21:32 ] ❯ uname -a 
Linux palladium 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux

noven/deps/crc on  main [!] is 📦 v0.9.2 via 💧 1.11.0 (OTP 23) via  23.2 
🕙 [ 19:21:41 ] ❯ mix compile\

 LD     crc_nif.so
/usr/bin/ld: cannot find -lerl_interface
collect2: error: ld returned 1 exit status
make: *** [Makefile:151: /home/loictosser/github.com/KalvadTech/noven/deps/crc/priv/crc_nif.so] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".

Notes

• everything is set up (base-devel) erlang

It seems the CRC makefile may not be using the correct compiler when CROSSCOMPILE is set by Nerves.

Comments added to crc_src/makefile

$(info "**** CRC PLATFORM set to [$(PLATFORM)] ****")
$(info "**** CRC CROSSCOMPILE set to [$(CROSSCOMPILE)] ****")

Error when compiling crc ...

>MIX_TARGET=bbb mix deps.compile crc
==> nerves
==> atmos

Nerves environment
  MIX_TARGET:   bbb
  MIX_ENV:      dev

==> crc
"**** CRC PLATFORM set to [darwin] ****"
"**** CRC CROSSCOMPILE set to [/Users/taun/.nerves/artifacts/nerves_toolchain_armv7_nerves_linux_gnueabihf-darwin_arm-1.8.0/bin/armv7-nerves-linux-gnueabihf] ****"
 C      checksum_xor.c
armv7-nerves-linux-gnueabihf-gcc: error: unrecognized command-line option '-arch'; did you mean '-march='?
make: *** [/Users/taun/Development/Elixir/atmos/_build/bbb_dev/lib/crc/obj/checksum_xor.o] Error 1
could not compile dependency :crc, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile crc --force", update it with "mix deps.update crc" or clean it with "mix deps.clean crc"
==> atmos
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

It seems the above is indicating, the platform is darwin which is correct but the crc source should be compiled using the cross compiler armv7-nerves-linux-gnueabihf-gcc and not compiled using darwin's cc which is what the makefile attempts.

Originating issue: nerves-project/nerves#444

This is an issue with the crc library. It builds the NIF in the source directory and not under _build. Since the source directory is shared between all targets and environments, make doesn't know that it needs to be rebuilt.
This is a common problem with NIF projects. Could you file an issue with crc or PR an update to its Makefile to build under _build? The Elixir Circuits projects are NIFs that have Makefiles that put object files and libraries in the right place. See https://github.com/elixir-circuits/circuits_spi/blob/master/Makefile#L21 for an example.

It would be really cool if crc's Makefile were to be altered, so that it is possible to compile for various targets without having to manually clean up the built files in between 🙂 .

CRC algorithm nifs were released in v0.8.0 and they add lots of new functionality that needs to be documented.

• Using named models
• extending named models
• creating models at runtime
• using an initialized model resource instead of its name etc

Some of these will require some usage and testing to document best practices. I'm curious if there is any performance gains to using a model's resource over just calling it by name, especially in gen servers that are calculating crc very often.

Hello,

Using crc on Linux causes the following problem:

iex(1)> CRC.crc_16("hello")
2017-11-23 12:37:09.102 pid=<0.1289.0> [warn] : The on_load function for module crc_nif returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'/home/user/projects/xyz/_build/dev/lib/crc/priv/crc_nif.so: invalid ELF header\''}}

** (UndefinedFunctionError) function :crc_nif.crc_16/1 is undefined (module :crc_nif is not available)
    (crc) :crc_nif.crc_16("hello")

The error is due to crc-0.7.0 package, containing the files c_src/env.mk and priv/crc_nif.so. Since these files are built and environment dependent, they must not be included in the package. priv/crc_nif.so is a MACH format compiled binary, causing the error.

I get ** (ArgumentError) argument error when computing the CRC of a binary constituted of 34599 bytes with CRC.ccitt_16

Is it to be expected??

Requested from @fire here

Lately I get a lot of this kind of error when running my tests locally:

** (UndefinedFunctionError) function :crc_nif.crc_fast/2 is undefined (module :crc_nif is not available)

When re-running with e.g. mix test --failed, the same tests are green and nothing have been compiled since the error.

I'm not sure if this belongs here. Perhaps it is an issue of elixir_make, elixir or erlang.
I recently updated to erlang 23.0, and I think this is where the problems started.

I have no similar problems on the CI server (that also runs tests) or in my production or staging environments. I'm also not able to reproduce the error. It just comes some times - I see it perhaps around 2-3 times per 8 hour working day.

Has anyone seen something similar or know what might be wrong?

I'm running:

• crc 0.10.0
• elixir_make 0.6.0
• elixir 1.10.3
• erlang 23.0.1
• Ubuntu

Initially discussed in #11 under Possible Improvements implementing crc/2, crc_init/1, crc_update/2, and crc_final/1 will allow implementing new algorithms by just passing the required params to crc_init/1 and using the context provided with the other functions. This would allow reduction of C code to a single implementation, instead of separate files for different widths etc.

Erlang version: 22.1
Elixir version: 1.9.1
crc version: 0.9.2

I'm getting the following error while compiling the dependencies with mix deps.compile:

could not compile dependency :crc, "mix compile" failed. You can recompile this dependency with "mix deps.compile crc", update it with "mix deps.update crc" or clean it with "mix deps.clean crc"
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

I have both gcc and make installed and they're added properly in the PATH.

In python3,

>>> from binascii import crc32
>>> crc32(b"123", 34) # Provide an initial value
3011637596

In elixir using this library:

iex(36)> CRC.crc(%{extend: :crc_32, init: 34}, "123")
2297841019

Is this a bug?

For larger binaries sending the entire value to a NIF could possibly cause issues with the scheduler, if execution time in the NIF is too great.

Some testing should be done to see what a sensible max binary size is. With that information we should limit calls to the NIF functions with binaries smaller than that, and for larger ones chunk calls to the NIF and calculate the CRC as we go. This will require modifications to the NIF functions to take continue a CRC instead of always starting from a seed.

• Decide on Max binary size to send to NIFs
• Add ability to continue a CRC to NIFs
• Chunk calls with binaries larger than max to make multiple calls to NIFs

BREAKING CHANGE

Remove the legacy algorithm specific functions in favor of the single calculate/2 with parameters.

The NIF seems to require specific LDFLAGS to be set in the Makefile in order to compile. The Makefile uses the ?= conditional operator to set these flags—if LDFLAGS are already configured, this means that the library cannot compile.

This caused us some problems today, because we are using crc and fast_tls in the same project, and the latter only compiles for us on M1 Macs if LDFLAGS include the path to openssl.

We got around this by copying the specific flags from this Makefile into the LDFLAGS that we set for our project, but if another operator such as += was used here, it might make it so that crc could compile without issues, but without conflicting with other projects.

Anyways, hopefully this helps others!