What are the thoughts on invoking make clean to clean up the object files and binaries it creates as part of mix clean? I would expect most Makefiles to have a clean target since it's standard practice.

I know this was discussed on the original 'make' pull request thread (in the main elixir repo), but it was unclear what conclusions people had reached.
Anyway, this proposal is to add support to define a different 'make' program if the environment variable MAKE_EXECUTABLE is set, thereby giving the user more freedom to customize the build.
If there's any interest, I can send a PR.

@josevalim and @wojtekmach I noticed that the build has failed for Elixir 1.9.4 and OTP 20.3 and the likely reason is that the ubuntu-latest is pointing to a hex otp build list that doesn't contains the OTP 20.3 build, as seen here.

I was trying to contribute to helping find out the root cause of the build error with a simple PR, but it seems to require fixing the build of all these otp versions. I can do it with the proper guidance because I'm wiling to contribute more to the ecosystem.

NetBSD is one of the platforms our project supports, so we would appreciate having a released version of elixir_make that works with it (this commit is needed: b46a5e8)

In my case, to compile the NIF I need few supporting libraries. These libraries are configured as Makefile target prerequisite, so that when I try to build NIF, it automatically fetches these files and builds the NIF. These libraries are platform specific, so I need to fetch it different files based for different arch/os/abi.

Issue

mix elixir_make.precompile tries to build the package for all supported platforms. But during that, it does not clean before starting the next target compilation. Because of that, I am not able to create archive for multiple targets.

$ MIX_ENV=prod mix elixir_make.precompile
Current compiling target: x86_64-apple-darwin
Fetching support libraries at https://example.com/support-darwin-x64.tar.gz
Compiles successfully...
Creating precompiled archive: /***-apple-darwin-0.15.1.tar.gz

Current compiling target: aarch64-apple-darwin
** Fails **

It fails because make won't be able to recognize the change in target, so it does not attempt to fetch a different package. Another issue I think is intermediate files (*.o) might be outdated, which we won't be able to catch here.

I think we should run make clean before compiling the next target. That way, I can ensure to clean up by deleting all intermediate files.

Please let me know if you have suggestions to handle this differently.

I removed a dependency from deps/0 in mix.exs and realised that it didn't remove it from the lock file.

Is mix deps.clean --unlock --unused the correct way to fully remove a dependency?

If so I was thinking most developers probably don't realise this. Perhaps a new alias could be added called deps.sync.

eg.

"deps.sync": ["deps.get", "mix deps.clean --unlock --unused"]

Hey, I'm not sure if you would verify it as bug or feature, but in one specific case it looks like a bug, so it's why I wanted to describe it here.

In ScenicFramework/scenic_driver_glfw#26 we have an edge-case while using elixir_make. The problem is that we already have there a priv directory (with included fonts directory), so your if would always return false for our use case:

Immediately I got some ideas. Some better and some not. I would like to describe well all of them in order to show my point of view:

1. Do not include fonts
   If there is need for hex package then why we can't download "fonts"? However for other edge-cases it's not a real fix - it's only workaround. What if some project have important not-public images or other resources? Sure, it's still possible to download them, but for me it looks like overcomplication for me. Easier one is 2nd idea.

2. Store files in (let's say priv-tmp) and later copy them to priv using Makefile/Makefile.vim
   It's again workaround, but really easy to introduce.

3. Always structure priv
   Probably simplest idea. Just forgot about checking everything and simply always run that call. Of course it's good only in it's simplicity. The if is there only for a "bad joke".

4. Dynamically build priv directory list
   This is my favorite one. It's easiest for developer as it does not requires to do anything. Finally it's idea which does not looks like a workaround.

5. Create a new dependency for 5th point
   Well … It does not requires to change code here, but is it really worth to create a standalone dependency for such simple feature?

Maybe you would ask why not adding our own custom compiler task for just restructuring build directory. Here are 2 main reasons:

1. Possibly (i.e. in theory) long list of compilers added to app. When programming I'm thinking of future - no matter how far. If possible just don't cause problems (even only theoretical - it's because history already show how things changes rapidly).

2. Problem with testing - we need to compile before we can call our own mix compiler task (otherwise mix does not see it) and we need mix compiler task before compile, so here we go again to point 5th (create extra dependency).

Optional idea: If we would go for comparing list of files then maybe it's also good idea to consider ensuring checksum for files in priv directory?

Which idea you see best? What do you think about it? Is it worth to add support for checksum?

While it's problem of scenic_driver_glfw I believe it's not that bad idea to discuss it here as other developers could also hit such edge-case. As said from my point of view it looks like a bug, but contributors of this project could not agree with me.

ping @axelson, @boydm, @josevalim and @whatyouhide

I'm trying to include https://github.com/chitika/cberl as an app dependency in my mix file. It's an erlang lib with NIFs for couchbase with a makefile.

The makefile in that erlang lib doesn't seem to be run under a normal mix compile...

I tried adding this lib with this setup:

 make_makefile: "../../deps/cberl/Makefile",
 compilers: [:elixir_make] ++ Mix.compilers,

And it tried to run. However it's looking for rebar in the wrong spot:

'.../apps/couchbase/rebar get-deps'
'.../apps/couchbase/rebar: No such file or directory'

I'm probably going about this all wrong =) Do you know a good way to run make on an erlang dependency? Or really any dependency I guess?

I was using asdf to install the latest Apple Silicon compiled versions of Erlang and Elixir to use on a Phoenix project that employs the new gen-auth stuff.

When running elixir 1.11.2-otp-23 + Erlang 23.2.1 for Apple Silicon, bcrypt_elixir fails to compile, noting it needs Erlang 20 or better. The bcrypt.nif never gets created.

When I revert back to my last Intel version of Erlang, 23.0.3, I can rebuild the project and the bcrypt.nif gets created. Keeping Apple or Intel Elixir makes no difference. It comes down to the Erlang base system.

As for my project, I'll keep running the Intel Erlang for the time being. It works.

The maintainer of bcrypt_elixir suggested I seek input over here. It seems like an issue with elixir_make could possibly be adjusted to address apple silicon? I'm honestly not sure; I admit I’m still a newbie in this ecosystem, so I’m at a roadblock for where to go to seek the knowledge to get this resolved. I imagine this could eventually come to bite others.

It should at least print out the final command that the :elixir_make compiler is executing. Mentioned in elixir-lang/elixir#4082.

Hi,

Sample code is here:
https://github.com/zacky1972/elixir_make_test

When running iex -S mix on the project, the error "Failed to load NIF" occurs.

Redefining the module ElixirMakeTest will eliminate this error. Removing "clean" of :compile of :aliases in project of mix.exs will also eliminate this error.

I feel it may be some kind of bugs, or it requires some workaround.

I've tested it on Elixir 1.11.2, Erlang 23.1.5 and elixir_make 0.6.2.

With the introduction of Mix.target(), nerves now stores built dependencies in the same directory for all targets, make will not pick up changes between targets. here is an example.

in the main_app Nerves application we have a dependency on c_library.

• Call MIX_TARGET=rpi0 mix deps.get
• Call MIX_TARGET=rpi0 mix firmware
• c_library gets compiled twice.
• make sees that c_library.c hasn't changed.
• make doesn't recompile priv/c_library
• priv/c_library is compiled for x86, not ARM.

When working with nifs under windows there is a group of people including myself that uses the linux compatible msys2 compiler suite under windows. At the moment elixir_make assumes windows always means nmake and must be forced manually with MAKE=make to do the right thing on msys2. Would be nice if elixir_make could check for msys2 (e.g. uname -s returns "MINGW....") at least in those cases when nmake can't be found.

If you're open to that idea I can prepare a pull request

Hi,

I proposed #41 but it was rejected because Application.app_dir(:my_app, "somewhere") doesn't work.

So, I propose an alternative approach that uses :code.priv_dir(:my_app), again. I've proved it works by the following code:

defmodule Mix.Tasks.MixTest do
  use Mix.Task
  require Logger

  def run(args) do
  	Logger.info(:code.priv_dir(:mix_test))
  end
end

On my machine, the following tests fail after upgrading to Erlang 19:

• test specifying a makefile (Mix.Tasks.Compile.ElixirMakeTest)
• test specifying targets to run when cleaning (Mix.Tasks.Compile.ElixirMakeTest)
• test specifying targets (Mix.Tasks.Compile.ElixirMakeTest)
• test specifying env (Mix.Tasks.Compile.ElixirMakeTest)
• test specifying a cwd (Mix.Tasks.Compile.ElixirMakeTest)
• test running with a makefile (Mix.Tasks.Compile.ElixirMakeTest)
• test --verbose (Mix.Tasks.Compile.ElixirMakeTest)

After downgrading to Erlang 18.3, all the tests passed.

I'm using Erlang 19, Elixir 1.4-dev, and I'm on Arch Linux

Edit:
For each error, the os specific build error is printed, and this is the stacktrace:

stacktrace:
(mix) lib/mix.ex:274: Mix.raise/1
(elixir_make) lib/mix/tasks/compile.make.ex:91: Mix.Tasks.Compile.ElixirMake.run/1
(ex_unit) lib/ex_unit/capture_io.ex:146: ExUnit.CaptureIO.do_capture_io/2
(ex_unit) lib/ex_unit/capture_io.ex:119: ExUnit.CaptureIO.do_capture_io/3
test/mix/tasks/compile.make_test.exs:51: anonymous fn/0 in Mix.Tasks.Compile.ElixirMakeTest.test running with a makefile/1
(elixir) lib/file.ex:1145: File.cd!/2
test/mix/tasks/compile.make_test.exs:45: (test)

I recently reported this issue for muontrap failing when built in a project, which had a space in a folder of the absolute path: fhunleth/muontrap#21. @fhunleth suggested adding a warning to this project instead of muontrap, as it's relevant to make in general.

Hi,

Current Pelemay builds native code by calling make from itself after generating code and Makefile by a macro. If elixir_make will support a feature building after executing a macro, Pelemay can use it.

I'd like to contribute enhancement of elixir_make.

For a Phoenix project, when trying to add dependencies such as argon2_elixir or bcrypt_elixir. The user is prompted to set the MAKE environment variable to the path of "nmake.exe" which requires installing the latest visual studio (or using chocolatey to install VisualCppBuildTools).

Compiling the project leads to an error:

makefile(34) : fatal error U1000: syntax error : ')' missing in macro invocation
Stop.

For bcrypt_elixir, the error is the same except for makefile(34); instead it's makefile(3) (or some other single digit number).

I was told this is probably happening because It's probably not checking if the executable specified by MAKE has the filename 'nmake' or 'nmake.exe' regardless of directory, instead of simply comparing against the string 'nmake'. and to report this bug. (kudos to legoscia on https://stackoverflow.com/questions/55616187/nmake-fails-when-attempting-to-compile-argon2-elixir)

Would it be possible to make a new release and post it to hex? I'm most interested in the :make_clean update. Not a huge rush, but now that I've converted all my projects to elixir_make, that's the main thing I miss when using the current hex package.

Thanks!!

It's not a super big thing, but I want to do a full, hex-based, integration test of a project I'm working on; the project relies on the PR that got merged in.

So I don't want to chase people into action or something, but is there a plan to release a new version shortly? If not, I'll probably want to spend some time on a work-around for my end-to-end test but that's fine; would just like to hear about timelines :)

TIA,

Cees

Instead, we should allow something such as:

versions: fn arg1, arg2 -> ... end

This will lead to a smaller surface API. :)

But let's do this after merging #79. /cc @cocoa-xu

So #37 made me think. The majority of Elixir projects will write Makefile for GNU make, so developers expects that elixir_make will always use it.

That PR added netbsd, but there's also solaris, smartos, dragonfly, …. How can we find a way to ensure we'll use gmake consistently ?

Could we just call gmake if :os.type() isn't {:unix, :linux} ? I can't really think of any other unices that bundles gnu make as a default.

Hi @cocoa-xu!

I have been looking at the code and I think we can simplify the precompiler implementation if we move the logic for zipping/checksuming to elixir_make.

The idea would be to roughly have precompile return a something like {:ok, [{target, cache_dir}]}. Then we can go through those lists and tar.gz and add the relevant metadata. By doing this, I think we can simplify the precompile callback. I also think we can remove both available_nif_urls() and current_target_nif_url callbacks, as long as we add a make_precompiled_url to elixir_make.

The idea is to make the precompiler worry only about precompilation and none of the actual artefact management.

WDYT?

Hello,

with version 0.6.1, the permission of some files has changed.

$> tail -n 1 deps/elixir_make/hex_metadata.config
{<<"version">>,<<"0.6.0">>}.
$> ll -a deps/elixir_make
total 40
drwxr-xr-x   9 kruse  staff   288B Dec  2 17:07 .
drwxr-xr-x  79 kruse  staff   2.5K Dec  2 17:07 ..
-rw-r--r--   1 kruse  staff     0B Dec  2 17:07 .fetch
-rw-r--r--   1 kruse  staff    67B Dec  2 17:07 .formatter.exs
-rw-r--r--   1 kruse  staff   275B Dec  2 17:07 .hex
-rw-r--r--   1 kruse  staff   1.4K Dec  2 17:07 README.md
-rw-r--r--   1 kruse  staff   504B Dec  2 17:07 hex_metadata.config
drwxr-xr-x   3 kruse  staff    96B Dec  2 17:07 lib
-rw-r--r--   1 kruse  staff   596B Dec  2 17:07 mix.exs
...
$> tail -n 1 deps/elixir_make/hex_metadata.config
{<<"version">>,<<"0.6.1">>}.
$> ll -a deps/elixir_make
total 48
drwxr-xr-x  10 kruse  staff   320B Dec  2 17:09 .
drwxr-xr-x  79 kruse  staff   2.5K Dec  2 17:09 ..
-rw-r--r--   1 kruse  staff     0B Dec  2 17:09 .fetch
-rw-------   1 kruse  staff    67B Dec  2 17:09 .formatter.exs
-rw-r--r--   1 kruse  staff   275B Dec  2 17:09 .hex
-rw-r--r--   1 kruse  staff   153B Dec  2 17:09 CHANGELOG.md
-rw-------   1 kruse  staff   1.4K Dec  2 17:09 README.md
-rw-r--r--   1 kruse  staff   523B Dec  2 17:09 hex_metadata.config
drwxr-xr-x   3 kruse  staff    96B Dec  2 17:09 lib
-rw-------   1 kruse  staff   845B Dec  2 17:09 mix.exs

That caused some problems for me when I was trying to create a docker image from my app.

which includes DragonFly support. That makes my life easier :)

Hi,

I propose Add a feature to put Makefile in Application.app_dir(:app, "somewhere") , especially in case that the Makefile is generated by a macro, e.g. in case of Pelemay.

The following error occurs by setting :make_makefile to Applicatoin.app_dir(:app, "somewhere/Makefile") in project in mix.exs:

❯❯❯ mix deps.get
** (ArgumentError) unknown application: :elixir_make_test
    (elixir 1.11.2) lib/application.ex:896: Application.app_dir/1
    (elixir 1.11.2) lib/application.ex:923: Application.app_dir/2
    mix.exs:11: ElixirMakeTest.MixProject.project/0
    (mix 1.11.2) lib/mix/project.ex:790: Mix.Project.get_project_config/1
    (mix 1.11.2) lib/mix/project.ex:114: Mix.Project.push/3
    (stdlib 3.13.2) lists.erl:1267: :lists.foldl/3

When I run mix test on the root I get error:

mix test
==> elixir_make
Compiling 1 file (.ex)
Generated elixir_make app
echo "done"
done
** (RuntimeError) trying to access Mix.Project.app_path for an umbrella project but umbrellas have no app
    (mix) lib/mix/project.ex:332: Mix.Project.app_path/1
    (mix) lib/mix/project.ex:397: Mix.Project.build_structure/2
    lib/mix/tasks/compile.make.ex:73: Mix.Tasks.Compile.ElixirMake.run/1
    (mix) lib/mix/task.ex:296: Mix.Task.run_task/3
    (elixir) lib/file.ex:1145: File.cd!/2
    (mix) lib/mix/project_stack.ex:86: Mix.ProjectStack.root/1
    (elixir) lib/enum.ex:1184: Enum."-map/2-lists^map/1-0-"/2
    (mix) lib/mix/tasks/compile.all.ex:19: anonymous fn/1 in Mix.Tasks.Compile.All.run/1

I can also create the Makefile in the specific app, still it doesn't work as I expected(need to fix the makefile path in the main mix.exs file) and if I make it work still get the same error

directory structure:

.
├── Makefile
├── README.md
├── apps
│   └── my_make
│       ├── README.md
│       ├── config
│       │   └── config.exs
│       ├── lib
│       │   └── my_make.ex
│       ├── mix.exs
│       └── test
│           ├── my_make_test.exs
│           └── test_helper.exs
├── config
│   └── config.exs
├── mix.exs
└── mix.lock

environment:

• OSX 10.11.5
• Erlang/OTP 18 [erts-7.3]
• Elixir 1.3.1

files:

Makefile is :

all:
    echo done

apps/my_make/mix.exs :

defmodule MyMake.Mixfile do
  use Mix.Project

  def project do
    [app: :my_make,
     version: "0.1.0",
     build_path: "../../_build",
     config_path: "../../config/config.exs",
     deps_path: "../../deps",
     lockfile: "../../mix.lock",
     elixir: "~> 1.3",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     compilers: [:elixir_make] ++ Mix.compilers,
     deps: deps]
  end

  def application do
    [applications: [:logger]]
  end

  defp deps do
    [{:elixir_make, "~> 0.1.0"}]
  end
end

Hi,

I propose add a feature to call configure before building, in order to enable NIFs to be configurable by Autoconf.

I have a plan to apply it to Pelemay.

This issue was opened yesterday. It seems that in some cases, line 247 - [erts_dir] = Path.wildcard(Path.join(root_dir, "erts*"))- is returning an empty list, and that is resulting in an error.

This project can be a great place to document best practices for writing NIFs and add some helper functions to do so.

I think the first, and most important part, is where to store the compiled artifacts and how to load them.

It could be a good idea as well to provide a function to load NIFs from the advised location or find the port executable. This would help with further standardization of how elixir handles NIFs/Ports.

It would be beneficial for windows build tools (alright, I mean just nmake) to get Windows-compatible environment variables.

For example ERTS_INCLUDE_DIR in nmake resolves to c:/Program Files/erl10.3/erts-10.3/include instead of c:\Program Files\erl10.3\erts-10.3\include.

While the compiler and nmake itself can deal with that, other tools like copy can't and throw an error. This is especially important on Windows and nmake as useful tools like tr are not available by default and nmake itself is quite limited in features.

Since by now probably some number of projects have worked around that in some way or other, it may also be an option to define additional variables with backslashes, so maintainers have the option to use back- or forward slashes depending on their requirements.

With several projects that use NIFs, the so / dll file is stored in the priv directory. This raises the following question(s):

• If this is the standard place to store the file(s) created, do we want to create the priv directory if it is not there?
• Alternatively, do we want to add something to the documentation saying that the developer might need to create an empty priv directory as part of the package?

Could you update the package in hex?

I need @recursive true change for umbrella projects.

Thanks