Add a boolean to the recipe structure, disabled, default false. This provides a more convenient way for tonixxx to manage the Vagrant state of recipes that the user wants to skip.

The alternative is to comment the whole recipe, which would leave Vagrant resources behind on tonixxx clean, and also result in a corrupted Vagrant state when vagrant global-status, vagrant destroy <id> are run. When tonixxx clean removes the entire project directory under ~/.tonixxx, then vagrant destroy <id> fails. So we use the boolean to skip over the boil step, but allow the recipe to be otherwise allocated and deallocated.

Which tonixxx tasks should disabled: true continue operating, and which should be skipped?

• Allocate/deallocate host directories under ~/.tonixxx project: Continue
• Generate Vagrantfile: Continue
• Spin up vagrant box: Skip
• Boil: Skip
• Spin down: Continue
• Clean: Continue

• Write a simple Rust application. Use https://github.com/mcandre/toys/tree/master/rust/fizzy as a reference.
• Setup Debian build bot.
• Setup Windows build bot.
• Setup FreeBSD build bot.
• Setup Alpine build bot.
• Setup OpenBSD build bot.
• Setup macOS build bot.
• Setup NetBSD build bot.
• Setup Illumos build bot.
• Setup DragonflyBSD bot.
• Setup uClibC Linux bot.
• Setup x86 AKA i386 bots.
• Setup ARM bots.

• Repack Debian base box with rsync synced folders working.
• Publish vagrant-debian-cpp box featuring clang.

As a user, I want to know how to get the most out of tonixxx for different project setups, so that I can build projects faster and easier.

• Describe how Docker containers are much faster build bots than for Linux-only target cross-compilation, compared to Vagrant VM's.
• Describe how Go applications natively cross-compile even faster than Docker containers and Vagrant VM's, though caveat that some obscure targets like musl Linux and uClibC Linux are currently unsupported in the Go toolchain, requiring either Docker containers or Vagrant VM's to build applications for these targets.
• Provide scaling up tip for hosts: more host CPU cores, more host CPU speed, more host RAM, more available host disk space, faster host network connection.
• Provide scaling up tip for build bot guests: more guest CPU cores, more guest CPU speed, more guest RAM, more available guest disk space, faster guest network adapter.
• Provide minimalism tip; that build bots should limit the services and files they provide, in order to download, boot, and build faster.
• Provide hypervisor tip: VMware tends to be fairly speedy (medium), compared to VirtualBox (moderate) and libvirt/qemu (very slow). Again, for Linux-only target cross-compilation, containers are faster than VM's, so Docker hypervisor, LXD hypervisor, or LXC hypervisor providers are preferred for better build speed.
• Demonstrate parallelism option (pending #10).
• Demonstrate remote scale-wide option (pending #12).
• Describe SSH-based native remote scale-wide option for optimal cross-platform build speeds. (Basically, ditching virtualization and containerization for purchasing dedicated hardware for SSH server build bots for the desired target platforms.)

Early demos:

• fewer

Pending NetBSD fixing its hilarious v7.1 package situation (cmake missing from pkgin, cmake broken from pkg_add).

As a workaround, we could also try

• install cmake via pkgsrc
• configuring a NetBSD buildbot based on NetBSD the v6.x series
• see if NetBSD v7.2 has fixed both pkg_add and pkgin

e.g., getopt

https://www.nuget.org/packages/fix8.dependencies.getopt/

Projects:

• fewer (C)
• palindrome (C++)
• bronze (Rust)

Targets:

(As many as are supported by the relevant programming language)

• Debian
• FreeBSD
• Windows
• OpenBSD
• macOS
• Illumos
• DragonflyBSD
• x86 AKA i386 architecture (e.g. MINIX)

Test Docker-backed Vagrant build bots, for increased performance for Linux targets.

• GNU Linux (Debian)
• musl Linux (Alpine)
• uClibC Linux (OpenWrt 15.05 Chaos Calmer, busybox)

Pending #7.

Integrate build steps back to the host, in order to provide a tighter feedback loop for tight tweak-compile loops using developer tools available on the host, prior to porting an application.

• Provide a tonixxx boil-local subcommand that executes the configured tonixxx.yaml build steps directly on the host machine, with the host source directory as the execution's current working directory. This boil-local subcommand is independent of the typical Vagrant-related subcommands.
• Receive a TONIXXX_LOCAL_LABEL environment variable for users to select an appropriate set of steps for their particular host machine, from the labelled steps in tonixxx.yaml. When blank or undefined, tonixxx attempts the default set of steps.

• Vagrantfile, export.Vagrantfile pair for crafting exportable, reusable downstream Vagrant boxes
• rsync as lowest common denominator for syncing source code and artifacts, for many different guest operating systems

As a user, I want to be able to run build steps against specific recipes, to speed up debugging when a few recipes in particular break and the rest are okay.

Implementation notes:

Expand phase syntax like tonixxx boil:gnu-linux-amd64,windows-amd64, so that users can conveniently run phases against specific recipes. This helps to speed up experimentation, when the rest of the user’s recipes are currently slow, broken, or straining host resources.

When tonixxx clean’s a recipe:

• vagrant destroy -f
• rm -rf .vagrant

• Publish mcandre/vagrant-void-cpp box
• Publish mcandre/vagrant-void-haskell box
• Publish a musl Linux box with rustup (currently unavailable due to poor musl support by rustup)
• Have examples/fewer build musl target with Void
• Have examples/palindrome build musl target with Void
• Have examples/mo build musl target with Void

Environments:

• GNU Linux (Debian to start with)
• FreeBSD
• Darwin
• Windows

Tasks:

• Install a C compiler.
• Install libreadline source.
• Install cppcheck.
• Install splint.
• Install vera++.
• Install drmemory.

• Build fewer for Haiku OS

https://cmake.org/cmake/help/v3.0/command/include_directories.html

Pending mcandre/vagrant-haiku-cpp#1

Theories:

• Manage one Vagrant box clone at a time, destroying the VM and .vagrant/ as tonixxx iterates through the target platforms.
• Experiment with multi-machine Vagrantfile configurations.
• House each Vagrant clone in a different subdirectory, and somehow get /vagrant guest synced folder to read from the right host files (no longer simply current working directory).
• Patch Vagrant to support new CLI flags for overriding the Vagrantfile as well as the synced folder host directory.

Also need to figure out how to exclude .tonixxx/ from infinite synced folder recursion.

• local rsync with --exclude .tonixxx

https://github.com/Redundancy/go-sync

Theories for working around lack of exclude pattern support in go-sync:

• Move .tonixxx/ to $HOME/.tonixxx.
• Place Vagrant box clones in $HOME/.tonixxx/<project-id>/<recipe-name>.
• Require a project name in tonixxx.yaml.
• Have Vagrant operations be relative to the project-id host directory rather than the global .tonixxx directory.
• Have tonixxx clear remove the project-id directory rather than the entire .tonixxx directory.
• Have rsync copy-merge all the per-Vagrant artifacts back together into a common tree in $HOME/.tonixxx/<project-id>/bin.
• Document $HOME/.tonixxx behavior in README.

As a C++ developer, I want an example of CLI long option parsing, so that I have a better understanding of how to parse CLI arguments in a more cross-platform compatible way.

• boost, getopt, docopt?

As a developer, I want the fewer project to pass its configured linters, so that I have a better understanding of how to write modern, highly cross-compatible C code.

Configure the fewer project so that it passes a lint step.

• Tweak fewer source code to pass linter rules.
• Configure linters to disable impractical rules.

Pending #123

• fewer (C)
• palindrome (C++)

Mostly involves copying the FreeBSD build bots and replacing the base boxes with HardenedBSD variants.

Problem: Most developer host machines do not have the 64 GB+ RAM hardware, nor the CPU speed, necessary for running each buildbot in the fewer demo simultaneously. In fact, running any two buildbots simultaneously with each requiring 4 GB of guest memory triggers a freeze of macOS hosts.

Solution: Have tonixxx receive environment variable-controlled configuration for constraining the maximum number of simultaneously running Vagrant boxes. Restructure the distillery control flow so that instead of looping outer over phases and inner over recipes, we loop outer over recipes and inner over phases, to reduce time-wasting boot cycling during builds.

Default the max limit to 1 when unspecified. Treat 0 as no-max-limit.

Test behavior of this configuration against debian-amd64, debian-i386, freebsd buildbots with the fewer project.

Ensure that the --no-tty flag is specified when executing commands over vagrant ssh -c..., to work around an issue with Windows box SSH sessions freezing indefinitely.

As a user, I want to be able to run tonixxx -help before I configure tonixxx.yaml, so that I always have access to the CLI help information.

Allow usage information to be presented to the user from any directory, regardless of whether a valid tonixxx.yaml configuration file is found.

So that users can elect to define buildbot Vagrant configuration in the same directory path as a tonixxx project, without expensively copying any Vagrant boxes or metadata into the guest boxes for tonixxx up / tonixxx boil.

Specifically, panic early rather than return errors for situations that are unlikely to actually fail, such as when accessing the current working directory or the current user's home directory.

As a user, I want to customize the Vagrant provider, so that I can use tonixxx with new kinds of Vagrant boxes.

• When a provider is not specified, then --provider <name> is omitted from vagrant... commands.
• Use vagrant up'ed provider name, as opposed to Vagrant Cloud provider name, which are often differently tagged, e.g. vagrant up --provider <vmware_fusion | vmware_workstation> vs. Vagrant Cloud tag vmware_desktop, vagrant up --provider qemu vs. Vagrant Cloud tag libvirt.

no vera++ package is available on ftp, but mentioned in tonixxx/examples/fewer-buildbot-openbsd/bootstrap.sh

Pending rust-lang/rust#49462

Think about how recipe labeling could become more flexible, in order to handle project structures where an individual buildbot (perhaps with a cross-compilation toolchain installed) may be responsible for artifacts of multiple architectures.

For now, users can either:

• Include the architecture name as a deeper sub-directory path under bin/ for build artifacts (boo!)
• Include the architecture name as a suffix to artifact basenames (boo!)
• List a buildbot twice in tonixxx.yaml under different labels (boo!)

As a developer, I want an example that uses a library dependency, so that I have a better understanding of how to use tonixxx to build complex, cross-platform projects.

Use a C compiler macro to check whether _MSC_BUILD is defined. If it is defined, then assume a Visual Studio environment, and do not include nor use readline. If it is not defined, then use readline to populate a history of commands to fewer, and process Up/Down arrow keys to manipulate the history.

See https://eli.thegreenplace.net/2016/basics-of-using-the-readline-library/ for an example readline usage.

As a user, I want tonixxx to allow multiple recipes to build simultaneously, so that I can build my projects faster.

• Preserve max recipe limit at all times.
• Add -j [<n>] option to execute n per-recipe tasks concurrently.
• n should default to the number of detected host cores.
• Have all per-recipe output prefixed with the recipe label, so that interleaved build output can be read somewhat sensibly.

Reduce custom configuration in cmake-related Windows build bots, by passing --installargs '"ADD_CMAKE_TO_PATH=User"' to the Chocolatey command that installs cmake, for better provisioning code.

As a developer, I want valgrind included in more C/C++ examples, so that I have a better understanding of how to integrate memory leak checking with tonixxx.

Implementation notes:

Flesh out the C and C++ example projects by running a memory leak check as part of the tonixxx build, for OS’s that support a memory leak checker.

Try valgrind.

• fewer
• palindrome

Moving drmemory to #122.

• Allow users to supply important setting overrides in Vagrantfile's for a recipe.
• Reduce boilerplate by configuring tonixxx against a Vagrantfile path, rather than a Vagrant box name.

As a PowerShell user, I want tonixxx to support cmd.exe environments, so that I can write some recipe commands using MS-DOS batch file syntax rather than UNIX sh syntax.

• Vagrant synced folder: C:\vagrant
• Configuring environment variables: set <key>="<value>"
• Step aggregation: <earlier commands> & <next command>

As a developer, I want tonixxx to automate more of the port generation process, specifically collecting all the ports into a single, transferable archive, so that I can share my ports more reliably.

Implementation notes:

• Archive format is TGZ, a reasonable compromise between maturity vs. ability to extract without special tools (sorry Windows!) vs. support for preserving file permissions.
• Leave the original executables on disk as-is, for easy debugging/inspection.
• Ensure that non-EXE binaries receive full executable bits on insertion into the archive.

As a user, I want less noise when I build my app.

Treat Vagrant output as error-level log: Tonixxx will hide the abundant Vagrant text by default, unless an error is detected.

As a user, I want to be able to control tonixxx from one host while processing recipes on remote hosts, so that I can support more parallel builds.

Rewrite tonixxx in server/client terms, to make it more convenient for users to delegate multiple long builds to different server hardware.

Pending ndmitchell/hlint#463 , https://ghc.haskell.org/trac/ghc/ticket/14988#ticket

Potential workarounds:

• Look for an hlint package in the main HardenedBSD pkg repository. (None available)
• Disable hlint as part of the lint task in our Makefile, when HardenedBSD machine is detected.
• Pin HLint at a version that’s not so bloated

Futher sub-issue: Conditional syntax is unportable for [g]make vs. [b]make, so the current Makefile used to organize my Haskell project's build system can't easily express that the lint task should omit hlint for HardenedBSD machines. Long term, I plan to rewrite my Haskell build configurations in a more portable system, Shake. For now, I'll just use the tonixxx steps configuration to override the lint step, for the HardenedBSD recipe.

Tracking work in progress for this issue in https://github.com/mcandre/tonixxx/tree/issue-49 branch.

Have the tonixxx down and clean commands ignore the exit value of vagrant halt, to avoid prematurely terminating cleanup of multiple tonixxx recipes one after the other. This is especially important for any Haiku recipes:

hashicorp/vagrant#9557

So that the source code within each box stays up to date for compilation.

Help users manipulate more complex tonixxx builds, by providing a -f <file> option to override the default path to the CWD/tonixxx.yaml file.

For example, a workflow using multiple tonixxx.yaml files + rsync/ssh would help users to hack a multiple machine remote setup while waiting for #12 to be implemented.

As a PowerShell user, I want tonixxx to support PowerShell, so that I can write some recipe commands using PowerShell rather than UNIX sh syntax.

• Vagrant synced folder: C:\vagrant
• Configuring environment variables: $env:<key> = "<value>"
• Step aggregation: <earlier commands>; if ($?) { <next command> }

Now that it is evident Vagrant syncing often fails to work bidirectionally, we will have to do some more work to extract artifacts emitted during build by guests.

• Experiment with https://github.com/dustymabe/vagrant-sshfs to provide configurable guest-to-host synced files. (Not supported for Windows.)
• Experiment with NFS as a workaround for some kernels, if available and doesn't simply present a VirtualBox FS missing error (FreeBSD, DragonflyBSD, OpenBSD, NetBSD, MINIX, Illumos). In fact, NFS is broken for Windows hosts, so some kind of rsync bidirectional workaround is ideal for a standard, reusable cross-compilation configuration.
• Experiment with https://github.com/smerrill/vagrant-rsync-back .
• Experiment with https://github.com/invernizzi/vagrant-scp .
• Use FTP.

As a developer, I want to see a C project that features multiple binaries for different purposes, so that I have a better understanding of how tonixxx manages porting multi-binary projects.

To test how tonixxx handles multiple binary artifacts per target per project.

As a user, I want more Vagrant commands exposed up to tonixxx, so that I don't have to manually muck with these low-level details.

Add tonixxx ssh:<label> for buildbot debugging convenience.

As a workaround, users can cd $HOME/.tonixxx/<project>/<label>; vagrant ssh.

Have Windows mode use UNIX-style syntax for now, to work around limitations in Vagrant hardcoded bash -l assumption during vagrant ssh -c.

As an ARM user, I want ARM examples, so that I have a better understanding of how to build ARM ports.

Apply qemu/libvirt emulation toward a working ARM example, perhaps with an ARM Raspbian guest.

Strategies:

• [-] Purchase a dedicated Linux host featuring qemu + Vagrant, in order to run arm guest boxes (super lame though).
• [-] Help VirtualBox add support for ARM guests (ha!)
• g++-aarch64-linux-gnu + qemu-user-static
• Help vagrant-libvirt, libvirt support non-Linux host environments.
• Explore alternative Vagrant providers for any support for ARM guests: https://github.com/hashicorp/vagrant/wiki/Available-Vagrant-Plugins#providers
• Create a third tool similar to goxcart, tonixxx, specifically for cross-compiling applications to multiple non-x86 architectures. Where goxcart uses Docker and tonixxx uses Vagrant, this tool would use qemu.
• Publish base boxes featuring guest OS + qemu, running the non-x86 edition of the OS inside qemu. (Requires the outer guest VM's to double or treble their disk and memory requirements.)

Projects:

• fewer (C)
• palindrome (C++)
• mo (Haskell)
• bronze (Rust)

Pending #7.

Add a doctor subcommand to the tonixxx CLI app that runs some fundamental tests for each recipe in tonixxx.yaml.

tonixxx already performs many error checks relating to file paths while preparing recipes to boil. We want to specifically add a new check that tests whether Vagrant synced folders are operating correctly.

• COMSPEC-type VM's should be able to echo "test" >C:\Users\vagrant\test.txt in the guest, observe a test.txt artifact for the recipe on the host, and successfully delete the file from the host with del C:\Users\vagrant\test.txt. I'm not entirely certain that the C:\Users\vagrant path is correct for Vagrant synced folders with Windows hosts, will have to borrow a VM from Vagrant Cloud or pack a fresh box from scratch.
• POSIX-type VM's should be able to echo "test >$HOME/vagrant in the guest, observe a test.txt artifact for the recipe on the host, and successfully delete the file from the host with rm $HOME/vagrant/test.txt.

As a user, I want tonixxx to do a better job cleaning up commented-out, deleted, and edited recipes, so that I have more resources freed on tonixxx clean.

If the user comments out a recipe and then runs tonixxx clean, then tonixxx often leaves behind orphaned Vagrant instances, that are visible to vagrant global-status, but cannot be easily removed (either with tonixxx clean or with vagrant destroy -f...)

Let's brainstorm ways to handle this edge case more gracefully:

• tonixxx could cache the recipes to be spun up, and include these recipes when cleaning up Vagrant boxes, even when they are commented out in tonixxx.yaml