ansible: environment difference while running debops playbooks about mitogen HOT 9 OPEN

had that debops.sshd issue as well here. Just fyi.

from mitogen.

If it works with Mitogen (I wonder how) but doesn't work without it, it seems like the dpkg command doesn't like to use non-existent locales. Your en_gb.UTF-8 locale is passed through SSH because by default LC_* environment variables are whitelisted by sshd.

Perhaps setting the environment as LC_ALL: C for this task could solve it.

from mitogen.

I just pushed issue_122.yml which I was using earlier to try and reproduce the difference. Running vimdiff on the output with/without Mitogen shows no environment difference. Probably I'm still doing something wrong to reproduce the original scenario, will look at it later in the week.

No matter what the current behaviour is, Mitogen needs to emulate it. It needs to let both the SSH invocation and the sudo invocation do whatever horridness is done to the environment and reproduce exactly that, otherwise more playbooks will be broken behind closed doors

from mitogen.

@dw Ah, so you want to make sure that Mitogen fails the same way that Ansible fails with the same environment, good idea.

from mitogen.

Discoveries so far:

• OS X ssh_config does not have SendEnv by default
• Debian ssh_config has SendEnv by default in /etc/ssh/ssh_config
• Fedora ssh_config sets SendEnv in /etc/ssh/ssh_config.d/05-redhat.conf

As usual the correct behaviour isn't exactly clear. I dislike having bits of the environment leak into the tooling (it makes it impossible to test), but we have to emulate Ansible.

Relevant bugs:

• ansible/ansible#9635
• ansible/ansible#10698

Relevant spaghetti:

• C.CONTROLER_LANG is set from LANG environment variable
• C.DEFAULT_MODULE_LANG defaults to C.CONTROLER_LANG unless ANSIBLE_MODULE_LANG is present in the environment, or module_lang is present in [defaults] in ansible.cfg
• plugins/shell/__init__.py uses C.DEFAULT_MODULE_LANG or (lol) the LANG environment variable if C.DEFAULT_MODULE_LANG is unset, to build a dict of defaults environment variables for any shell command. The dict contains LANG, LC_ALL and LC_MESSAGES forced to that value, but it only does this if C.DEFAULT_MODULE_SET_LOCALE is set, which is set if the ANSIBLE_MODULE_SET_LOCALE environment variable is set, or the module_set_locale key exists in [defaults] of ansible.cfg.
• plugins/action/__init__.py uses ShellBase.env_prefix() from _compute_environment_string() to build the snippet for setting the environment using the active shell, but _compute_environment_string() does not return those variables in its output dictionary, only in the returned snippet.
• plugins/action/__init__.py calls _compute_environment_string() from _execute_module() only. _low_level_execute_command() does not get this environment treatment

Sphaghetti round 2:

• actions/script.py (which is the failing action) is not actually using _execute_module(), but calling _compute_environment_string() directly.

Can't reproduce this locally, going to have to setup two Gcloud boxes again

from mitogen.

@stefangweichinger do you still have access to your setup that triggered the locale error? If so, can you please --start-at="Check if OpenSSH is installed (or half-installed)" and attach "-vvv" output both for Ansible and for Mitogen.

If not don't worry, I'll pick this up next week

from mitogen.

@dw yes, I have. pls more details: with or without your new debug variable exported, mitogen.log from controller?

from mitogen.

Just -vvv is fine, MITOGEN_ROUTER_DEBUG is only useful for lower level IO issues

from mitogen.

ansible-playbook ~/.local/share/debops/debops/ansible/playbooks/common.yml -vvv --start-at="Check if OpenSSH is installed (or half-installed)" -l tx100 > debops_common.log

debops_common.log

can't find a mitogen.log in /tmp now, though

note: afk soon ...

from mitogen.