REBAR_CACHE_DIR and global config clash about rebar3 HOT 13 OPEN

Some things that are looking for global configuration will actually end up in REBAR_CACHE_DIR/.config/rebar3/... which is wrong (in my opinion).

I faced this issue today when I tried to use rebar3 with kerl, where kerl defines the REBAR_CACHE_DIR environment variable specific for each OTP installation. The rebar3 new command didn't look up ~/.config/rebar3/templates, and it took a few hours for me to find out why by thoroughly reviewing the source code.

from rebar3.

I looked at this, and reading @eproxus' comment, I'm as confused regarding what should be set. I do agree that it seems odd we're setting global_rebar_dir for REBAR_CACHE_DIR (this appears to be done twice), but I didn't dig much further.

from rebar3.

Yeah I think I recall digging in slack, and the unfortunate state is that this stuff is all very inconsistent and will be hard to fix without breaking anyone's setup. This would need a full re-mapping and clean-up to re-define where the values are set, what they mean, and how they're used.

It's very possible right now that it was intended one way in a code area, and was used another one in another area, and that there's no way to fix this without willingly breaking either pattern in a specific context.

from rebar3.

One thing I'd like to suggest is that this behavior of ${REBAR_CACHE_DIR/.config/rebar3/templates and the other affected settings should be documented. I couldn't find the document.

from rebar3.

I just came across this issue as well (with an automated template install that wasn't being recognized), and my solution was to make a little plugin for outputting directories.

https://github.com/choptastic/rebar3_dirs and https://hex.pm/packages/rebar3_dirs

At least this way, I can use something that, should this functionality ever change, will hopefully remain backwards compatible, rather than having to know and then code around the internal logic for where the config directory ends up.

$ rebar3 dirs template_dir
/home/gumm/kerl/26.2.4/.cache/rebar3/.config/rebar3/templates

$ rebar3 dirs
base_dir: /home/gumm/code/myapp/_build/default
deps_dir: /home/gumm/code/myapp/_build/default/lib
root_dir: /home/gumm/code/myapp/.
checkouts_dir: /home/gumm/code/myapp/_checkouts
checkouts_out_dir: /home/gumm/code/myapp/_build/default/checkouts
plugins_dir: /home/gumm/code/myapp/_build/default/plugins
lib_dirs: apps/*lib/*.
project_plugin_dirs: plugins/*
global_config_dir: /home/gumm/kerl/26.2.4/.cache/rebar3/.config/rebar3
global_config: /home/gumm/kerl/26.2.4/.cache/rebar3/.config/rebar3/rebar.config
template_dir: /home/gumm/kerl/26.2.4/.cache/rebar3/.config/rebar3/templates

from rebar3.

@choptastic That should be built into Rebar itself! ❤️

from rebar3.

@choptastic That should be built into Rebar itself! ❤️

I certainly appreciate the appreciation.

I'll gladly submit a PR to rebar3 if the team wants.

from rebar3.

I wouldn't be opposed to having this sort of stuff built in, though there's some potential confusion around what we do for subtasks and hooks' env vars: https://github.com/erlang/rebar3/blob/main/apps/rebar/src/rebar_env.erl

from rebar3.

I'm facing a related issue, and I'm not sure what is the right way to proceed. We use kerl to install Erlang on multi-user systems, and the recent change involving REBAR_CACHE_DIR requires that those users have write access to the location in which kerl has installed the Erlang package.

===> Could not write to "/opt/kerl/lib/26.2.5/.cache/rebar3/hex". Please ensure the path is writeable.

The pain point for me as system administrator, is that I would rather not expose this path as writeable to the users on the system.

I believe before the change in kerl (#457), rebar would have used the default ~/.cache/rebar3/hex.

I'm unsure if my question belongs in the rebar project or kerl project, but discussion on this topic seems to be centered here for now.

As a temporary workaround, I've instructed users to use the following when activating a kerl installation:

. /opt/kerl/lib/26.2.5/activate
unset REBAR_CACHE_DIR
unset REBAR_PLT_DIR

from rebar3.

@JesseStimpson, this might also be of interest: kerl/kerl#495.

from rebar3.

At this point I don't think it is possible to solve this, we just made too many inconsistent choices within Rebar3 over the years. The only thing that seems reasonable is to map out all the use cases and breakage points, and pick and choose what is a clearer future behavior, and whatever option we pick will be backwards incompatible for some users and use cases.

from rebar3.

Ugh, was global_rebar_dir inherited from rebar2? It seems off...

from rebar3.

Also was meaning forever to move templates to XDG_DATA_HOME or ~/.local/share/rebar3

from rebar3.