lueschem / edi Goto Github PK

View Code? Open in Web Editor NEW

38.0 38.0 12.0 2.84 MB

Embedded development infrastructure.

Home Page: https://www.get-edi.io

License: GNU Lesser General Public License v3.0

Python 97.09% Shell 2.02% Smarty 0.07% Makefile 0.01% Jinja 0.81%

ansible debian embedded image iot jinja2 lxc lxd python python3 raspberry-pi ubuntu yaml

edi's People

Contributors

Stargazers

Watchers

Forkers

erickeller timofurrer fkfang randombenj yeekzhang soccentric vraevsky matthiasklein 0xbadcoffe devninja5 shubhamsaini1585

edi's Issues

Add minimal xorg playbook

Idea:
Add a playbook that provides a minimal xorg installation.

Benefit:
Using "ssh -Y ..." one could access the container and launch a gui application there if needed. E.g. for native debugging, doing a commit, ...

Prerequisite:
playbook for ssh and ssh key handling

Support zstd compressed deb packages.

deb packages might soon be zstd compressed:

https://www.phoronix.com/scan.php?page=news_item&px=Ubuntu-Zstd-Deb-Packages

edi should add this type of compression in order to successfully download qemu in the future.

debuild -uc -us fails on python tests on Ubuntu 16.10

steps to reproduce:
http://docs.get-edi.io/en/latest/getting_started.html#working-with-the-edi-source-code

lol@[~/workspace/edi]$ cat /etc/os-release                                                                            *[master] 
NAME="Ubuntu"
VERSION="16.10 (Yakkety Yak)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.10"
VERSION_ID="16.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="http://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=yakkety
UBUNTU_CODENAME=yakkety
lol@[~/workspace/edi]$ debuild -us -uc                                                                                *[master] 
 dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: info: source package edi
dpkg-buildpackage: info: source version 0.4.5
dpkg-buildpackage: info: source distribution xenial
dpkg-buildpackage: info: source changed by Matthias Lüscher (Launchpad) <[email protected]>
 dpkg-source --before-build edi
dpkg-buildpackage: info: host architecture amd64
 fakeroot debian/rules clean
dh clean --with python3,sphinxdoc --buildsystem=pybuild --fail-missing
   dh_testdir -O--buildsystem=pybuild -O--fail-missing
   dh_auto_clean -O--buildsystem=pybuild -O--fail-missing
I: pybuild base:184: python3.5 setup.py clean 
running clean
removing '/home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.5' does not exist -- can't clean it
   dh_clean -O--buildsystem=pybuild -O--fail-missing
 dpkg-source -b edi
dpkg-source: info: using source format '3.0 (native)'
dpkg-source: info: building edi in edi_0.4.5.tar.xz
dpkg-source: info: building edi in edi_0.4.5.dsc
 debian/rules build
dh build --with python3,sphinxdoc --buildsystem=pybuild --fail-missing
   dh_testdir -O--buildsystem=pybuild -O--fail-missing
   dh_update_autotools_config -O--buildsystem=pybuild -O--fail-missing
   dh_auto_configure -O--buildsystem=pybuild -O--fail-missing
I: pybuild base:184: python3.5 setup.py config 
running config
   debian/rules override_dh_auto_build
make[1]: Entering directory '/home/mariole/workspace/edi'
dh_auto_build
I: pybuild base:184: /usr/bin/python3 setup.py build 
running build
running build_py
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi
copying edi/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests
copying tests/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests
copying tests/test_command_line_interface.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/clean.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/config.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/image.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/lxc.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/qemu.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/target.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
copying edi/commands/version.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/archivehelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/commandfactory.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/configurationhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/configurationparser.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/debhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/edicommand.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/helpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/keyhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/mockablerun.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/networkhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/playbookrunner.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/sharedfoldercoordinator.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/shellhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
copying edi/lib/versionhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/lib
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/configcommands
copying edi/commands/configcommands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/configcommands
copying edi/commands/configcommands/configclean.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/configcommands
copying edi/commands/configcommands/configinit.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/configcommands
copying edi/commands/configcommands/configmerge.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/configcommands
copying edi/commands/configcommands/configssh.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/configcommands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/imagecommands
copying edi/commands/imagecommands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/imagecommands
copying edi/commands/imagecommands/bootstrap.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/imagecommands
copying edi/commands/imagecommands/imageclean.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/imagecommands
copying edi/commands/imagecommands/imagelxc.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/imagecommands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/lxccommands
copying edi/commands/lxccommands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/lxccommands
copying edi/commands/lxccommands/importcmd.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/lxccommands
copying edi/commands/lxccommands/launch.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/lxccommands
copying edi/commands/lxccommands/lxcclean.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/lxccommands
copying edi/commands/lxccommands/lxcconfigure.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/lxccommands
copying edi/commands/lxccommands/profile.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/lxccommands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/qemucommands
copying edi/commands/qemucommands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/qemucommands
copying edi/commands/qemucommands/fetch.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/qemucommands
copying edi/commands/qemucommands/qemuclean.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/qemucommands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/targetcommands
copying edi/commands/targetcommands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/targetcommands
copying edi/commands/targetcommands/targetconfigure.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/commands/targetcommands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands
copying tests/commands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
copying tests/lib/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
copying tests/lib/test_archivehelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
copying tests/lib/test_configurationhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
copying tests/lib/test_configurationparser.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
copying tests/lib/test_debhelpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
copying tests/lib/test_playbookrunner.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
copying tests/lib/test_sharedfoldercoordinator.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/lib
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/libtesting
copying tests/libtesting/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/libtesting
copying tests/libtesting/helpers.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/libtesting
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands/configcommands
copying tests/commands/configcommands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands/configcommands
copying tests/commands/configcommands/test_configmerge.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands/configcommands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands/imagecommands
copying tests/commands/imagecommands/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands/imagecommands
copying tests/commands/imagecommands/test_bootstrap.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/commands/imagecommands
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/libtesting/fixtures
copying tests/libtesting/fixtures/__init__.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/libtesting/fixtures
copying tests/libtesting/fixtures/configfiles.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/libtesting/fixtures
copying tests/libtesting/fixtures/datadir.py -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/libtesting/fixtures
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates
copying edi/plugins/config_templates/debian-jessie-amd64.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates
copying edi/plugins/config_templates/debian-stretch-amd64.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree
copying edi/plugins/config_templates/project_tree/PROJECTNAME-build.yml.edilink -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree
copying edi/plugins/config_templates/project_tree/PROJECTNAME-develop.yml.edilink -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree
copying edi/plugins/config_templates/project_tree/PROJECTNAME-run.yml.edilink -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree
copying edi/plugins/config_templates/project_tree/PROJECTNAME-test.yml.edilink -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration/base
copying edi/plugins/config_templates/project_tree/configuration/base/PROJECTNAME.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration/base
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration/overlay
copying edi/plugins/config_templates/project_tree/configuration/overlay/PROJECTNAME-build.global.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration/overlay
copying edi/plugins/config_templates/project_tree/configuration/overlay/PROJECTNAME-develop.global.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration/overlay
copying edi/plugins/config_templates/project_tree/configuration/overlay/PROJECTNAME-run.global.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration/overlay
copying edi/plugins/config_templates/project_tree/configuration/overlay/PROJECTNAME-test.global.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/configuration/overlay
copying edi/plugins/config_templates/project_tree/gitignore.edihidden -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins/playbooks
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook
copying edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook/roles
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook/roles/sample_role
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook/roles/sample_role/tasks
copying edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook/roles/sample_role/tasks/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates/project_tree/plugins/playbooks/sample_playbook/roles/sample_role/tasks
copying edi/plugins/config_templates/raspbian-jessie-armhf.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/config_templates
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_profiles
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_profiles/general
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_profiles/general/lxc_networking
copying edi/plugins/lxc_profiles/general/lxc_networking/default_interface.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_profiles/general/lxc_networking
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_profiles/general/lxcif0
copying edi/plugins/lxc_profiles/general/lxcif0/lxcif0.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_profiles/general/lxcif0
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates/debian
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates/debian/hostname
copying edi/plugins/lxc_templates/debian/hostname/hostname.tpl -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates/debian/hostname
copying edi/plugins/lxc_templates/debian/hostname/hostname.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates/debian/hostname
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates/debian/hosts
copying edi/plugins/lxc_templates/debian/hosts/hosts.tpl -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates/debian/hosts
copying edi/plugins/lxc_templates/debian/hosts/hosts.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/lxc_templates/debian/hosts
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/base_system
copying edi/plugins/playbooks/debian/base_system/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/base_system
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/base_system/roles
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/base_system/roles/apt_setup
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/base_system/roles/apt_setup/tasks
copying edi/plugins/playbooks/debian/base_system/roles/apt_setup/tasks/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/base_system/roles/apt_setup/tasks
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities
copying edi/plugins/playbooks/debian/development_user_facilities/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/host_user_as_target_user
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/host_user_as_target_user/tasks
copying edi/plugins/playbooks/debian/development_user_facilities/roles/host_user_as_target_user/tasks/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/host_user_as_target_user/tasks
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/host_user_as_target_user/templates
copying edi/plugins/playbooks/debian/development_user_facilities/roles/host_user_as_target_user/templates/no_passwd -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/host_user_as_target_user/templates
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/shared_folder_mountpoints
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/shared_folder_mountpoints/tasks
copying edi/plugins/playbooks/debian/development_user_facilities/roles/shared_folder_mountpoints/tasks/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/development_user_facilities/roles/shared_folder_mountpoints/tasks
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings
copying edi/plugins/playbooks/debian/inherit_host_proxy_settings/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles/inherit_host_proxy_settings
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles/inherit_host_proxy_settings/tasks
copying edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles/inherit_host_proxy_settings/tasks/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles/inherit_host_proxy_settings/tasks
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles/inherit_host_proxy_settings/vars
copying edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles/inherit_host_proxy_settings/vars/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/inherit_host_proxy_settings/roles/inherit_host_proxy_settings/vars
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking
copying edi/plugins/playbooks/debian/lxc_networking/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking/roles
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking/roles/lxc_interfaces
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking/roles/lxc_interfaces/tasks
copying edi/plugins/playbooks/debian/lxc_networking/roles/lxc_interfaces/tasks/main.yml -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking/roles/lxc_interfaces/tasks
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking/roles/lxc_interfaces/templates
copying edi/plugins/playbooks/debian/lxc_networking/roles/lxc_interfaces/templates/default_interface.j2 -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/edi/plugins/playbooks/debian/lxc_networking/roles/lxc_interfaces/templates
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
copying tests/data/test_debhelpers/Release -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
copying tests/data/test_debhelpers/binary-all_Packages -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
copying tests/data/test_debhelpers/binary-amd64_Packages -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/bin
copying tests/data/test_debhelpers/foo-1.0/bin/foo -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/bin
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian
copying tests/data/test_debhelpers/foo-1.0/debian/changelog -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian
copying tests/data/test_debhelpers/foo-1.0/debian/compat -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian
copying tests/data/test_debhelpers/foo-1.0/debian/control -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian
copying tests/data/test_debhelpers/foo-1.0/debian/copyright -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian
copying tests/data/test_debhelpers/foo-1.0/debian/foo.install -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian
copying tests/data/test_debhelpers/foo-1.0/debian/rules -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian
creating /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian/source
copying tests/data/test_debhelpers/foo-1.0/debian/source/format -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers/foo-1.0/debian/source
copying tests/data/test_debhelpers/foo_1.0_amd64.deb -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
copying tests/data/test_debhelpers/test-archive-key.asc -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
copying tests/data/test_debhelpers/test-pubring.gpg -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
copying tests/data/test_debhelpers/test-secring.gpg -> /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build/tests/data/test_debhelpers
PYTHONPATH=. http_proxy='127.0.0.1:9' sphinx-build -N -bman docs/ build/man # Manpage generator
Running Sphinx v1.4.8
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: 5 added, 0 changed, 0 removed
reading sources... [ 20%] command_cheat_sheet
reading sources... [ 40%] getting_started
reading sources... [ 60%] index
reading sources... [ 80%] introduction
reading sources... [100%] reference_list

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... edi.1 { introduction getting_started reference_list command_cheat_sheet } 
build succeeded.
make[1]: Leaving directory '/home/mariole/workspace/edi'
   dh_auto_test -O--buildsystem=pybuild -O--fail-missing
I: pybuild base:184: cd /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest tests
============================= test session starts ==============================
platform linux -- Python 3.5.2+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/mariole/workspace/edi, inifile: 
collected 26 items

tests/test_command_line_interface.py .
tests/commands/configcommands/test_configmerge.py .
tests/commands/imagecommands/test_bootstrap.py .
tests/lib/test_archivehelpers.py .
tests/lib/test_configurationhelpers.py ..
tests/lib/test_configurationparser.py .....
tests/lib/test_debhelpers.py .F
tests/lib/test_playbookrunner.py .
tests/lib/test_sharedfoldercoordinator.py ............

=================================== FAILURES ===================================
________________________ test_package_download_with_key ________________________

datadir = local('/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0')

    def test_package_download_with_key(datadir):
>       do_package_download(datadir, 'https://www.example.com/keys/test-archive-key.asc')

tests/lib/test_debhelpers.py:155: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/lib/test_debhelpers.py:129: in do_package_download
    repository_mock.sign_release()
tests/lib/test_debhelpers.py:106: in sign_release
    subprocess.run(cmd, input=None, timeout=None, check=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

input = None, timeout = None, check = True
popenargs = (['gpg', '--batch', '--yes', '--homedir', '/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0', '--no-default-keyring', ...],)
kwargs = {}, process = <subprocess.Popen object at 0x7f9abccb8860>
stdout = None, stderr = None, retcode = 2

    def run(*popenargs, input=None, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        The other arguments are the same as for the Popen constructor.
    
        If universal_newlines=True is passed, the "input" argument must be a
        string and stdout/stderr in the returned object will be strings rather than
        bytes.
        """
        if input is not None:
            if 'stdin' in kwargs:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired:
                process.kill()
                stdout, stderr = process.communicate()
                raise TimeoutExpired(process.args, timeout, output=stdout,
                                     stderr=stderr)
            except:
                process.kill()
                process.wait()
                raise
            retcode = process.poll()
            if check and retcode:
                raise CalledProcessError(retcode, process.args,
>                                        output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['gpg', '--batch', '--yes', '--homedir', '/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0', '--no-default-keyring', '--secret-keyring', '/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0/test-secring.gpg', '--keyring', '/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0/test-pubring.gpg', '--digest-algo', 'SHA256', '-abs', '-o', '/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0/Release.gpg', '/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0/Release']' returned non-zero exit status 2

/usr/lib/python3.5/subprocess.py:708: CalledProcessError
----------------------------- Captured stderr call -----------------------------
gpg: WARNING: unsafe permissions on homedir '/tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0'
gpg: /tmp/pytest-of-mariole/pytest-1/test_package_download_with_key0/trustdb.gpg: trustdb created
gpg: no default secret key: No secret key
gpg: signing failed: No secret key
===================== 1 failed, 25 passed in 2.10 seconds ======================
E: pybuild pybuild:276: test: plugin distutils failed with: exit code=1: cd /home/mariole/workspace/edi/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest tests
dh_auto_test: pybuild --test --test-pytest -i python{version} -p 3.5 returned exit code 13
debian/rules:6: recipe for target 'build' failed
make: *** [build] Error 25
dpkg-buildpackage: error: debian/rules build gave error exit status 2
debuild: fatal error at line 1376:
dpkg-buildpackage -rfakeroot -D -us -uc failed

I did not dig into the python failure but it seems to be something related to some url pointing to https://www.example.com/keys/test-archive-key.asc which is not really hosting any asc file...

Make sure output node gets properly merged in postprocessing_commands

Desired Behaviour

The output node shall be properly merged even if some sub nodes get overwritten in overlays.

Actual Behaviour

The overlay just replaces the parent.

Reduce disk usage by optionally excluding documentation.

Check this for details:
https://askubuntu.com/questions/129566/remove-documentation-to-save-hard-drive-space

Provide edi as a snap package

For non Ubuntu/Debian users it would be beneficial if edi could get installed as a snap package.

Edi fails to create host user within container when group already exists

Initial Position

The host user has the following settings:

uid: foo(1000)
gid: users(100)

Note: user name is not equal to group name.

Actual Behavior

edi fails to create the host user equivalent within the container.

Two problems exist:

user name is not equal to group name.
group users already exists.

Issue Type

Bug report

Displayed Warnings

TASK [openssh_server_keys : Identify openssh-server keys.] *************************************************************************
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|succeeded` instead use `result is succeeded`.
 This feature will be removed in version 2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in 
ansible.cfg.
ok: [edi-tmp-ed7bf51689500a32fc35]

TASK [openssh_server_keys : Remove openssh-server keys.] ***************************************************************************
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|succeeded` instead use `result is succeeded`.
 This feature will be removed in version 2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in 
ansible.cfg.
changed: ...

TASK [openssh_server_keys : Add a systemd service that re-generates the openssh-server keys.] **************************************
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|succeeded` instead use `result is succeeded`.
 This feature will be removed in version 2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in 
ansible.cfg.
changed: [edi-tmp-ed7bf51689500a32fc35]

TASK [openssh_server_keys : Enable the systemd service that re-generates the openssh-server keys.] *********************************
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|succeeded` instead use `result is succeeded`.
 This feature will be removed in version 2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in 
ansible.cfg.

Enable preprocessing commands.

Issue Type

Feature request

Environment detail

Similar to the postprocessing_commands, allow preprocessing_commands.

Use Cases

check availability of ssh pub keys for current user (see #24)
generate profile for loop device pass through.

error: Missing parent 'lxdbr0' for nic 'lxcif0'

Environment:

cat /etc/os-release
NAME="Ubuntu"
VERSION="16.10 (Yakkety Yak)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.10"
VERSION_ID="16.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="http://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=yakkety
UBUNTU_CODENAME=yakkety
+ python3 --version
Python 3.5.2+
+ ansible-playbook --version
ansible-playbook 2.1.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
+ lxc --version
2.4.1
+ dpkg -l
+ grep \(lxd\|ansible\)
ii  ansible                                         2.1.1.0-1                                        all          Configuration management, deployment, and task execution system
ii  lxd                                             2.4.1-0ubuntu1.1                                 amd64        Container hypervisor based on LXC - daemon
ii  lxd-client                                      2.4.1-0ubuntu1.1                                 amd64        Container hypervisor based on LXC - client

Step to reproduce

mkdir my-first-edi-project
cd my-first-edi-project
edi config init my-project raspbian-jessie-armhf
sudo edi -v lxc configure my-first-edi-container my-project-develop.yml

The following error occurs:

Profile privileged_f2c9cc4cc9ef28d536cf created
Success: Created lxc profile privileged_f2c9cc4cc9ef28d536cf.
INFO:root:Running command: ['sudo', '-u', 'mariole', 'lxc', 'profile', 'edit', 'privileged_f2c9cc4cc9ef28d536cf']
Success: The following profiles are now available: lxcif0_0c4a88500d0670949c8f, privileged_f2c9cc4cc9ef28d536cf
Going to launch container.
INFO:root:Running command: ['sudo', '-u', 'mariole', 'lxc', 'launch', 'local:my-project-develop_edicommand_lxc_import', 'my-first-edi-container', '-p', 'lxcif0_0c4a88500d0670949c8f', '-p', 'privileged_f2c9cc4cc9ef28d536cf']
Creating my-first-edi-container
Starting my-first-edi-container
error: Missing parent 'lxdbr0' for nic 'lxcif0'
Try `lxc info --show-log local:my-first-edi-container` for more info
Error: Command '['sudo', '-u', 'mariole', 'lxc', 'launch', 'local:my-project-develop_edicommand_lxc_import', 'my-first-edi-container', '-p', 'lxcif0_0c4a88500d0670949c8f', '-p', 'privileged_f2c9cc4cc9ef28d536cf']' returned non-zero exit status 1
For more information increase the log level.

This is probably done by the default configuration of the lxd package when it's installed as a dependency.

Use RAUC project to install/update target

Issue Type

Enhancement

Environment detail

N./A.

More details

have a look at https://github.com/rauc/rauc

Generate image with robust A/B update using mender.io.

Goal

Provide an edi configuration similar to edi-pi (see also this blog post) with full support for mender.io.

The configuration can be used to generate a Debian image for the Raspberry Pi that allows robust A/B updates of a big fleet of devices. Furthermore the configuration shall also produce a cross compilation environment for faster development.

Sub Goals

Allow a hybrid update strategy (mender for image based upgrades, apt for package based updates)
The hybrid update strategy shall also allow a kernel update (based on edi-boot-shim) using apt
Directly build the mender compliant image without making the detour over mender-convert (However: mender-convert is a really great tool!)
Vanilla u-boot shall be used as bootloader - the mender specific patches shall get replaced by a specially crafted hush script (boot.scr).
Mender client shall be installed as a Debian package (many thanks to Andreas Henriksson!)

Partition Layout

vfat boot partition

The following stuff is on the vfat partition:

Pi bootloader
u-boot (unpatched)
uboot.env (persistent storage of environment variables, gets written on first boot)
boot.scr: does the mender glue logic, loads the boot.scr of the active ext4 partition

1x ext4 data partition

The data partition is shared between the A and the B system. The partition has a fixed size.

2x ext4 system partition (A/B)

The system partitions have the following characteristics:

Dynamically adjusted size on first boot to make use of the full disk.
Contain a full Debian (currently stretch) root file system including kernel
Contain a boot.scr that describes how to load the kernel/device-tree/initrd (the boot.scr gets updated by edi-boot-shim)

Design Reasons

I would like to keep the possibility of apt based updates to a) speed up updates during development and to b) have reduced network traffic for security updates of an IoT fleet
I would like to have an unmodified u-boot in order to get rid of maintaining a forked u-boot (I am a lazy guy where possible)
I would like to directly generate mender compliant images to have feature parity with Yocto, shorter build cycles and more flexibility
I would like to use Debian kernels directly (without moving around dtbs, kernels and initrds) (see also this blog post)

Debootstrap fails to create ../rootfs/test-dev-null during lxc container creation process

Issue Type

Bug report

Environment detail

sh -x << EOF
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

python3 --version
Python 3.5.3

ansible-playbook --version
ansible-playbook 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

lxc --version
3.13

dpkg -l | grep "\(lxd\|ansible\)"
ii  ansible                                                     2.2.1.0-2+deb9u1                  all          Configuration management, deployment, and task execution system
EOF

Edi installation method: Apt
Ansible installation method: OS package
LXD installation method: OS package

Desired Behaviour

No error.

Actual Behaviour (Bug report only).

Please note that the failing folder is not encrypted. But it is on a RAID10 setup. This failure does not occur when I use a folder outside of the RAID drive.
Please see console output below..
kyukim@Ryzen7:~/my-first-edi-project$ sudo edi -v lxc configure my-first-edi-container my-project-develop.yml
[sudo] password for kyukim:
INFO:root:Load time dictionary:
edi_configuration_name: my-project-develop
edi_configure_remote_target: false
edi_create_distributable_image: false
edi_current_user_gid: 1000
edi_current_user_host_home_directory: /home/kyukim
edi_current_user_name: kyukim
edi_current_user_ssh_pub_keys:

/home/kyukim/.ssh/id_rsa.pub
edi_current_user_target_home_directory: /home/kyukim
edi_current_user_uid: 1000
edi_edi_plugin_directory: /home/kyukim/edi/edi/plugins
edi_host_ftp_proxy: ''
edi_host_hostname: Ryzen7
edi_host_http_proxy: ''
edi_host_https_proxy: ''
edi_host_no_proxy: ''
edi_host_socks_proxy: ''
edi_log_level: INFO
edi_lxd_version: '3.13'
edi_project_directory: /home/kyukim/my-first-edi-project
edi_project_plugin_directory: /home/kyukim/my-first-edi-project/plugins
edi_work_directory: /home/kyukim/my-first-edi-project

INFO:root:Using base configuration file 'my-project-develop.yml'
INFO:root:Using overlay configuration file 'configuration/overlay/my-project-develop.global.yml'
INFO:root:Merged configuration:
bootstrap:
architecture: amd64
repository: deb http://deb.debian.org/debian/ stretch main
repository_key: https://ftp-master.debian.org/keys/archive-key-8.asc
tool: debootstrap
general:
edi_compression: gz
edi_required_minimal_edi_version: 1.1.0
parameters: {}
lxc_profiles:
100_lxc_networking:
path: lxc_profiles/general/lxc_networking/default_interface.yml
200_default_root_device:
path: lxc_profiles/general/default_root_device/default_root_device.yml
lxc_templates:
100_etc_hostname:
path: lxc_templates/debian/hostname/hostname.yml
200_etc_hosts:
path: lxc_templates/debian/hosts/hosts.yml
playbooks:
100_base_system:
parameters:
create_default_user: true
install_openssh_server: true
path: playbooks/debian/base_system/main.yml
200_development_user_facilities:
path: playbooks/debian/development_user_facilities/main.yml
300_sample_playbook:
path: playbooks/sample_playbook/main.yml
900_base_system_cleanup:
path: playbooks/debian/base_system_cleanup/main.yml
postprocessing_commands:
100_sample_command:
output:
sample_output: my-project-develop.result
parameters:
message: We could do something useful within this command!
path: postprocessing_commands/sample_command/custom_command
qemu: {}
shared_folders:
edi_workspace:
folder: edi-workspace
mountpoint: edi-workspace

Going to bootstrap initial image - be patient.
INFO:root:Running command: ['debootstrap', '--arch=amd64', '--variant=minbase', '--include=python,sudo,netbase,net-tools,iputils-ping,ifupdown,isc-dhcp-client,resolvconf,systemd,systemd-sysv,gnupg', '--components=main', '--force-check-gpg', '--keyring=/home/kyukim/my-first-edi-project/tmpeemtv_sf/temp_keyring.gpg', 'stretch', '/home/kyukim/my-first-edi-project/tmpeemtv_sf/rootfs', 'http://deb.debian.org/debian/']
/usr/sbin/debootstrap: 1454: /usr/sbin/debootstrap: cannot create /home/kyukim/my-first-edi-project/tmpeemtv_sf/rootfs/test-dev-null: Permission denied
E: Cannot install into target '/home/kyukim/my-first-edi-project/tmpeemtv_sf/rootfs' mounted with noexec or nodev
Error: Command '['debootstrap', '--arch=amd64', '--variant=minbase', '--include=python,sudo,netbase,net-tools,iputils-ping,ifupdown,isc-dhcp-client,resolvconf,systemd,systemd-sysv,gnupg', '--components=main', '--force-check-gpg', '--keyring=/home/kyukim/my-first-edi-project/tmpeemtv_sf/temp_keyring.gpg', 'stretch', '/home/kyukim/my-first-edi-project/tmpeemtv_sf/rootfs', 'http://deb.debian.org/debian/']' returned non-zero exit status 1
For more information increase the log level.
kyukim@Ryzen7:~/my-first-edi-project$

systemd reports degraded state on first boot.

On first boot of an edi created image, everything works as expected but systemd reports that the system is in degraded state.

Root cause:
openssh-server-key-generator.service startup conflicts with ssh.service startup.

Config parsing shall not fail if the overlay is empty.

Issue Type

Bug report

Environment detail

Edi installation method: ppa

Desired Behaviour

edi shall be able to handle empty overlay files.

Actual Behaviour (Bug report only)

edi fails if the overlay file is empty

---
# nothing to skip - full setup

with this stack trace:

Traceback (most recent call last):
  File "/usr/bin/edi", line 9, in <module>
    load_entry_point('edi==0.7.2', 'console_scripts', 'edi')()
  File "/usr/lib/python3/dist-packages/edi/__init__.py", line 76, in main
    get_command(command_name)().run_cli(cli_args)
  File "/usr/lib/python3/dist-packages/edi/commands/lxc.py", line 46, in run_cli
    self._run_sub_command_cli(cli_args)
  File "/usr/lib/python3/dist-packages/edi/lib/edicommand.py", line 91, in _run_sub_command_cli
    self._get_sub_command(cli_args.sub_command_name)().run_cli(cli_args)
  File "/usr/lib/python3/dist-packages/edi/commands/lxccommands/lxcconfigure.py", line 50, in run_cli
    cli_args, ['lxc_templates', 'lxc_profiles', 'playbooks']))
  File "/usr/lib/python3/dist-packages/edi/commands/lxccommands/lxcconfigure.py", line 53, in run
    self._setup_parser(config_file)
  File "/usr/lib/python3/dist-packages/edi/lib/edicommand.py", line 67, in _setup_parser
    self.config = ConfigurationParser(config_file)
  File "/usr/lib/python3/dist-packages/edi/lib/configurationparser.py", line 204, in __init__
    merge_1 = self._merge_configurations(base_config, global_config)
  File "/usr/lib/python3/dist-packages/edi/lib/configurationparser.py", line 253, in _merge_configurations
    element)
  File "/usr/lib/python3/dist-packages/edi/lib/configurationparser.py", line 265, in _merge_key_value_node
    overlay_node = overlay.get(node_name, {})
AttributeError: 'NoneType' object has no attribute 'get'

Make use of --cache option delivered with latest debootstrap.

Recently debootstrap introduced the --cache option. This is also useful for edi to speed up the bootstrapping process.

Things to consider:

edi needs to detect if debootstrap is recent enough to make use of the --cache option.
If available, then edi can by default make use of it.
The default caching path could go to /var/lib/edi/cache/debootstrap.
It shall be possible to overrule the cache path (make sure it gets turned into an absolute path).
It shall be possible to turn the cache off by providing an empty path (or cache: None).
The caching settings shall be part of the bootstrap section:

bootstrap:
  cache: /path/to/my/cache

Use ANSIBLE_PIPELINING to speed up image builds.

It looks like Ansible pipelining can give edi a significant speedup:
https://docs.ansible.com/ansible/latest/reference_appendices/config.html

The idea is that pipelining can be enabled on edi config level.

Generate an artifact that contains the source code of a corresponding artifact and its build dependencies.

Use Case

To be compliant with the obligations of various open source licenses you should be able to provide the "corresponding source code" with its build environment

Implementation

The idea is to make the whole corresponding source code including its build dependencies available as a container. To simplify the setup it can be made available within an emulated container so that the packages do not need to be cross compiled.

To exclude proprietary Debian packages a white/black listing approach is needed.

ssh: Verify that at least an ssh pub key is available for the current user.

Issue Type

Feature request

Desired Behaviour

During a preprocessing step we can verify, that at least one ssh pub key is available for the current user. If not: Print a warning or stop with an error (see also #22).

Use cdn/fastly URL for sources.list(.d).

Issue Type

Feature request

Desired Behaviour

Use the cdn/fastly url for the sources.list entries: deb.debian.org

Actual Behaviour

Currently the deprecated httpredir is used.
Hint: This needs to be changed for edi-pi too.

Restart a container if its list of applied profiles has changed

Some configuration changes might require a restart of the container. If edi detects that additional or other profiles should get applied then it might want to trigger a restart of the container.

Fix ssh test on Ubuntu artful.

Issue:

The ssh test fails on Ubuntu artful due to improved security checks of ssh:

Warning: Permanently added '10.185.156.71' (ECDSA) to the list of known hosts.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0664 for '/tmp/pytest-of-root/pytest-0/test_build_stretch_container0/test_id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/tmp/pytest-of-root/pytest-0/test_build_stretch_container0/test_id_rsa": bad permissions
Permission denied (publickey).

edi fails to bootstrap edi-raspbian container on Debian buster

Environment detail

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
+ python3 --version
Python 3.7.3
+ ansible-playbook --version
ansible-playbook 2.7.7
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/lueschem-admin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.7.3 (default, Apr  3 2019, 05:39:12) [GCC 8.3.0]
+ lxc --version
3.18
+ grep \(lxd\|ansible\)
+ dpkg -l
ii  ansible                               2.7.7+dfsg-1                         all          Configuration management, deployment, and task execution system

Desired Behavior

Setup should work out of the box.

Actual Behavior

The command

sudo edi -v lxc configure raspbian-buster-cross buster-cross.yml

fails during the playbook run:

PLAY [edi] ***************************************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'distutils.spawn'
fatal: [raspbian-buster-cross]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}
 [WARNING]: Could not create retry file '/usr/lib/python3/dist-packages/edi/plugins/playbooks/debian/base_system/main.retry'.         [Errno 13] Permission
denied: '/usr/lib/python3/dist-packages/edi/plugins/playbooks/debian/base_system/main.retry'

due to missing python3-distutils.

Implement bootstrapping as special case of "preprocessing_commands".

Issue Type

Idea/improvement

Desired Behaviour

To add more flexibility during pre-container bootstrapping the current bootstrapping implementation could be transformed into an implementation that processes a "preprocessing_commands" section. The current debootstrap implementation could then be solved as a "built-in" helper command that can be invoked from the preprocessing_commands section.

This would result in the simple but flexible workflow:

preprocessing_commands
playbooks
postprocessing_commands

edi image create command

Issue Type

Feature request

Desired Behaviour

edi shall be able to build "images". Images can target various environments such as bare metal, lxd, Docker, ... .

Design Ideas

Implement a command edi image create CONFIG.

The create command shall just expose access to plugins, enable introspection and provide the clean functionality.

The configuration section of the create command might look as follows:

postprocessing_commands:
  10_create_iso:
    path: path to iso plugin/script
    output: {{ edi_project_name }}.iso
    require_root: True
    parameters:
        bingo: bongo
    skip: False
  20_create_vmdk:
    path: path to packer plugin/script
    output: {{ edi_project_name }}.vmdk

Hints:

each script plugin will receive the following variables: edi_input_artifact, edi_output_artifact, edi_current_plugin_directory, edi_log_level.
edi_output_artifact will become the edi_input_artifact of the next script plugin.
edi image clean will delete the output artifacts.

Plugin:

A plugin might look as follows:

#!/bin/bash
{{ edi_current_plugin_directory }}/some_command --input {{ edi_input_artifact }} --output {{ edi_output_artifact }}

The plugins will be loaded using the jinja2 template engine and then executed. Attention: It will not get executed within its original folder!

Enable nested edi-pi execution

Issue Type

Feature request

Desired Behaviour

edi-pi shall be able to run within a container (e.g. generated using edi-ubuntu project configuration).

A modified ubuntu artful config did the trick:

lueschem@xps:~/workspace$ lxc config show ubuntu-artful 
architecture: x86_64
config:
  volatile.base_image: 74d4edd4c0bbf4d387d71f944fd57f833b9987cf954eedff85d69048aae7424e
  volatile.idmap.base: "0"
  volatile.idmap.next: '[]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.lxcif0.hwaddr: 00:16:3e:c4:06:f2
devices:
  loop-control:
    path: /dev/loop-control
    type: unix-char
  loop0:
    path: /dev/loop0
    type: unix-block
  loop1:
    path: /dev/loop1
    type: unix-block
  loop2:
    path: /dev/loop2
    type: unix-block
  loop3:
    path: /dev/loop3
    type: unix-block
  loop4:
    path: /dev/loop4
    type: unix-block
  loop5:
    path: /dev/loop5
    type: unix-block
  loop6:
    path: /dev/loop6
    type: unix-block
  loop7:
    path: /dev/loop7
    type: unix-block
  root:
    path: /
    type: disk
ephemeral: false
profiles:
- lxcif0_0c4a88500d0670949c8f
- nesting_2c77c12c691edf069bd1
- privileged_f2c9cc4cc9ef28d536cf
- shared_folder_4768ca9cb2ec30c17b45
stateful: false

Actual Behaviour

Loop device usage is currently not possible within a container that was created using the edi-ubuntu project configuration.

Implementation Ideas

The loop stuff could be prepared as a template with the yet to be implemented preprocessing steps (see #25):

Template gets generated in preprocessing_step e.g. as "generated_template".
Template is loaded via a generic template:

{% include ['{{ generated_template }}', '{{ fallback_template }}'] %}

Tests fail if user and hostname are identical

Issue Type

Bug report

Environment detail

+ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.2 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.2 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
+ python3 --version
Python 3.5.2
+ ansible-playbook --version
ansible-playbook 2.1.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
+ lxc --version
2.0.9
+ dpkg -l+ grep \(lxd\|ansible\)

ii  ansible                          2.1.1.0-1~ubuntu16.04.1                    all          Configuration management, deployment, and task execution system
ii  lxd                              2.0.9-0ubuntu1~16.04.2                     armhf        Container hypervisor based on LXC - daemon
ii  lxd-client                       2.0.9-0ubuntu1~16.04.2                     armhf        Container hypervisor based on LXC - client

Edi installation method: source
Ansible installation method: OS package
LXD installation method: OS package

Desired Behaviour

py.test-3 tests shall pass.

Actual Behaviour (Bug report only)

py.test-3 tests fail if the user name is identical to the hostname (e.g. username=ubuntu, hostname=ubuntu).

edi fails with upstream lxd (2.14): No root device could be found.

Issue Type

Bug report

Environment detail

NAME="Ubuntu"
VERSION="16.04.2 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.2 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
+ python3 --version
Python 3.5.2
+ ansible-playbook --version
ansible-playbook 2.1.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
+ lxc --version
2.14
+ grep \(lxd\|ansible\)
+ dpkg -l
ii  ansible                                    2.1.1.0-1~ubuntu16.04.1                       all          Configuration management, deployment, and task execution system
ii  lxd                                        2.14-0ubuntu3~16.04.1                         amd64        Container hypervisor based on LXC - daemon
ii  lxd-client                                 2.14-0ubuntu3~16.04.1                         amd64        Container hypervisor based on LXC - client

Edi installation method: ppa
Ansible installation method: OS package
LXD installation method: OS package from xenial-backports (sudo apt install lxd/xenial-backports lxd-client/xenial-backports)

Desired Behaviour

edi should work with upstream lxd releases.

Actual Behaviour

edi fails with lxd 2.14:

$ sudo edi -v lxc configure my-project-01 my-project-test.yml
...
edi_host_socks_proxy: ''
edi_lxc_network_interface_name: lxcif0
edi_project_plugin_directory: /home/lueschm1/workspace/my-project/plugins
edi_work_directory: /home/lueschm1/workspace/my-project

INFO:root:Running command: ['sudo', '-u', 'lueschm1', 'lxc', 'profile', 'show', 'lxcif0_0c4a88500d0670949c8f']
INFO:root:Running command: ['sudo', '-u', 'lueschm1', 'lxc', 'profile', 'create', 'lxcif0_0c4a88500d0670949c8f']
Profile lxcif0_0c4a88500d0670949c8f created
Success: Created lxc profile lxcif0_0c4a88500d0670949c8f.
INFO:root:Running command: ['sudo', '-u', 'lueschm1', 'lxc', 'profile', 'edit', 'lxcif0_0c4a88500d0670949c8f']
Success: The following profiles are now available: lxcif0_0c4a88500d0670949c8f
Going to launch container.
INFO:root:Running command: ['sudo', '-u', 'lueschm1', 'lxc', 'launch', 'local:my-project-test_edicommand_lxc_import', 'my-project-01', '-p', 'lxcif0_0c4a88500d0670949c8f']
Creating my-project-01
Error: Launching image 'my-project-test_edicommand_lxc_import' failed with the following message:
error: No root device could be found.

Add command that is able to identify the ssh public keys of the current user.

Issue Type

Feature request

Desired Behaviour

edi ssh list pubkeys shall deliver a list of the public keys available for the current (sudo) user.
Hint ssh -G ... can help to identify the available pub keys.

Generate an artifact that contains all changelogs of a corresponding artifact.

Use Case

To quickly find out what changes made it into an artifact it would be useful to collect all the changelogs of the artifact and make them available as a separate artifact.

Implementation

For each package Debian makes its changelog available. According to the debian policy the Debian changelog will be placed in

/usr/share/doc/package/changelog.Debian.gz: If there is an upstream and a Debian changelog.
/usr/share/doc/package/changelog.gz: If there is only one changelog.

The idea is to collect them and make them available as a tar archive separate from the "main" artifact.

Add general "parameters" node.

Issue Type

Feature request

Desired Behaviour

It would be great to be able to have general parameters that apply to all plugins:

Example:

general:
    parameters:
        my_message: "This message applies to all plugins."

Actual Behaviour

Currently there are no globally available parameters. If required, they have to be specified for each plugin (unwanted redundancy).

Generate an artifact that contains all copyright files of a corresponding artifact.

Use Case

To be compliant with various licenses you should distribute your product with the licenses of the open source software you are using.

Implementation

Debian places the copyright files of the upstream software to the documentation directory:
/usr/share/doc/package/copyright

The idea is to collect all those copyright files of an artifact and make them available as a separate artifact.

Make edi available on Debian

Issue Type

Feature request

Environment detail

> cat /etc/os-release
> python3 --version
> ansible-playbook --version
> lxc --version
> dpkg -l | grep "\(lxd\|ansible\)"
> EOF
+ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
+ python3 --version
Python 3.5.3
+ ansible-playbook --version
ansible-playbook 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
+ lxc --version
3.2
+ dpkg -l
+ grep \(lxd\|ansible\)
ii  ansible                          2.2.1.0-2                      all          Configuration management, deployment, and task execution system

Edi installation method: source
Ansible installation method: OS package
LXD installation method: snap

Desired Behavior

edi should work smoothly on Debian stretch as long as the LXD snap is installed.

Actual Behavior

How to reproduce:

sudo apt install snapd
sudo snap install lxd
source /etc/profile.d/apps-bin-path.sh
sudo usermod -a -G lxd $USER
lxd init
sudo apt install git
git clone https://github.com/lueschem/edi.git
cd edi
sudo apt install -y git-buildpackage dh-make equivs && sudo mk-build-deps -i debian/control
sudo visudo --> add /snap/bin to secure_path
debuild -us -uc

Following fixes were needed:

Remove lxd from build-deps
Check for debootstrap also in /usr/sbin/debootstrap (sbin paths are not in PATH of unprivileged user)
lxc snap executable not visible during debuild (not in PATH - also check for lxc in /snap/bin/)

local_setup fails when using zsh

Desired Behaviour

. local_setup shall also work for zsh.

Actual Behavior

. local_setup fails when run within zsh.

Cross image build (amd64/armhf or arm64) fails for Debian buster

Issue Type

Bug report

Desired Behaviour

edi should be able to build armhf or arm64 Debian buster images on an amd64 host.

Actual Behaviour

systemd gets forced into a tight loop if it runs in an emulated environment.

More Information

The related upstream issue is currently under investigation:
https://bugs.launchpad.net/qemu/+bug/1823790

Possible Workaround

Build the image within a chroot (like this, systemd will not get started).

Generate a machine readable artifact that describes all installed packages of a corresponding artifact.

Use Case

To compare two artifacts it would be nice to have a machine readable file that describes all installed packages of a given artifact.

Possible Implementation

The following command would produce a yaml containing valuable information:

dpkg-query --show --showformat='- package: ${Package}:\n  architecture: ${Architecture}\n  version: \"${Version}\"\n  source_package: ${source:Package}\n  source_version: \"${source:Version}\"\n    status: ${db:Status-Abbrev}\n'

The yaml will look like this:

- package: adduser
  architecture: all
  version: "3.118"
  source_package: adduser
  source_version: "3.118"
  status: ii 
- package: apt
  architecture: amd64
  version: "1.8.2"
  source_package: apt
  source_version: "1.8.2"
  status: ii 
- pacakge: apt-utils
  architecture: amd64
  version: "1.8.2"
  source_package: apt
  source_version: "1.8.2"
  status: ii 
- package: at
  architecture: amd64
  version: "3.1.23-1"
  source_package: at
  source_version: "3.1.23-1"
  status: ii
...

Upgrade edi-pi setup to Debian buster

Debian buster got officially released.

https://github.com/lueschem/edi-pi can now be upgraded to Debian buster.

scan vulnerabilities of the debian environment

Issue Type

Enhancement

Environment detail

N./A.

Desired Behaviour

scan the debian environement and list vulnerabilities using the debsecan tool like following:

ROOTFS=...
DISTRIBUTION_CODENAME=$(grep "VERSION=" ${ROOTFS}/etc/os-release | sed -e "s/.* [(]\(.*\)[)].*/\1/g")
mkdir -p ${ROOTFS}/tmp/security
chroot ${ROOTFS} bash -x << EOF
debsecan --suite ${DISTRIBUTION_CODENAME} --format detail --only-fixed > /tmp/security/output.debsecan
EOF

Allow installation of edi without installing lxd apt package

Option 1 (preferred):
Downgrade lxd dependency to "Recommends".
Attention: Make sure that all lxd and lxc calls check for the executable first.

Option 2:
Provide a "edi-light" package (without lxc) that conflicts with the edi package.

Justification:
Some functionality of edi also makes sense without lxc/lxd or lxc/lxd got installed as a snap.

Add possibility to remove translations in order to save disk space.

https://wiki.ubuntu.com/ReducingDiskFootprint#Documentation

Do not enforce uid and gid for edi_create_distributable_image, edi_configure_target.

Issue Type

Feature request

Desired Behaviour

Within the development_user_facilities playbook edi should not enforce uid/gid if it is building a distributable image or if it configures a target system (edi target configure ...).

Actual Behaviour

edi enforces the uid/gid of the development user also for a distributable image. This will cause problems if another user also wants to add his user that might eventually have the same uid/gid.

Enhance image bootstrap/image lxc commands

Issue Type

enhancement

Ideas

In addition to creating an initial image using debootstrap the bootstrap section shall allow the download of a previously generated lxd image (e.g. tool: lxc-download).
The image bootstrap and image lxc commands shall then be replaced by a lxc download command.

Big plus: The lxc launch command will then apply the right profiles to the pre-created lxd image.

ssh (role): Disable password login and automatically add pub keys to authorized_keys.

Issue Type

Feature request

Desired Behaviour

To improve security, ssh remote access shall not be possible with a (static) (default!) password.
Instead, keys shall be used for login.
For the development user, the public key of the current user shall be added to authorized_keys of the target system (see also #22).
For the default user all keys from a certain folder plus optionally the pub keys of the current user shall be added.

Handling of yaml parser errors

Properly report parser errors if edi hits an invalid configuration file.

This minimal information shall be included in the error message:

path to malformed file
error message of parser

Improve Proxy Handling on Ubuntu 18.04 Desktop

Issue Type

Bug report

Environment detail

+ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
+ python3 --version
Python 3.6.5
+ ansible-playbook --version
ansible-playbook 2.5.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/lueschm1/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]
+ lxc --version
3.0.0
+ dpkg -l
+ grep \(lxd\|ansible\)
ii  ansible                                    2.5.1+dfsg-1                         all          Configuration management, deployment, and task execution system
ii  lxd                                        3.0.0-0ubuntu4                       amd64        Container hypervisor based on LXC - daemon
ii  lxd-client                                 3.0.0-0ubuntu4                       amd64        Container hypervisor based on LXC - client

Edi installation method: source and ppa
Ansible installation method: OS package
LXD installation method: OS package

Desired Behaviour

sudo edi ... and edi ... shall use the same proxy settings.

Actual Behaviour

sudo edi ... does not take the proxy settings of the current user into account. The difference between Ubuntu 18.04 and Ubuntu 16.04 is that Ubuntu 16.04 had the possibility to apply the proxy settings system wide while for 18.04 it is only applied per user.

Provide possibility to run "long" tests.

Currently all tests do mock away lengthy operations such as debootstrap, lxc/lxd interactions, ansible.
This shall be the default behavior also for future releases since those tests must also work within a
debuild environment and eventually within a docker image on github.

Idea:
Optionally run py.test-3 with additional arguments:
py.test-3 --lxc --debootstrap --ansible
The additional arguments could be evaluated within pytest_generate_tests and return a True or False (default).
See also:
https://docs.pytest.org/en/latest/example/parametrize.html

The lengthy operations are then only executed if the respective option is true.

Incomplete removal of artifacts.

sudo edi -v image create --recursive-clean 10 pi3-stretch-arm64.yml

does not remove
qemu-*-static

Furthermore qemu-*-static should be pushed into a sub directory according to the config name. It might be
that different use cases reference different versions of qemu-*-static.

Upgrade edi-raspbian cross compiler to Raspbian buster

Raspbian buster got officially released.

Therefore edi-raspbian (https://github.com/lueschem/edi-raspbian) can be upgraded to Raspbian buster.

Remove proxy passwords from verbose output

Issue Type

Bug report

Environment detail

+ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
+ python3 --version
Python 3.5.2
+ ansible-playbook --version
ansible-playbook 2.1.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
+ lxc --version
2.0.10
+ + grep \(lxd\|ansible\)
dpkg -l
ii  ansible                                    2.1.1.0-1~ubuntu16.04.1                      all          Configuration management, deployment, and task execution system
ii  lxd                                        2.0.10-0ubuntu1~16.04.1                      amd64        Container hypervisor based on LXC - daemon
ii  lxd-client                                 2.0.10-0ubuntu1~16.04.1                      amd64        Container hypervisor based on LXC - client

Edi installation method: ppa
Ansible installation method: OS package
LXD installation method: OS package

Desired Behaviour

edi shall not reveal passwords encoded into the proxy url when building a container using 'edi -v ...'

Actual Behaviour (Bug report only)

edi shows passwords encoded into the proxy url within verbose output. This is especially undesired behavior when edi is run on a build server and logs are available for a big audience.

Cleanup handling of required packages.

It looks like the handling of required packages (requirements.txt/setup.py) needs some cleanup:

pip.req is not available in newer pip versions (see pypa/pip#4700 and https://packaging.python.org/discussions/install-requires-vs-requirements/)