Giter VIP home page Giter VIP logo

clang-build-test's Introduction

Clang-build

Linux and OSX test: Test status Windows test: Test status

Motivation:

  • Meta build systems are inherently the wrong way to go, either the build system or the compiler should be platform-agnostic.
  • Trying to cover all use-cases is the wrong way to go - there is no need to let people do it the wrong way
  • CMake is cumbersome, unnecessarily generic and verbose and people should not need a second programming language to be able to build C++
  • With Clang, finally a properly cross-platform compiler exists
  • With Python we have a language we can use consistently across platforms

Goals:

  • One compiler (Clang), one build system (written in Python)
  • Simple projects should be simple to build
  • Build process for reasonable project structures should still be easy
  • Adding third-party dependencies should be manageable

Related resources:

Usage

In order to run clang-build one should only need to have Python and Clang installed. If you have admin rights, pip install, otherwise just drop-in the clang-build.py script, e.g. curl -O https://raw.githubusercontent.com/GPMueller/clang-build-test/master/clang-build.py

  • clang-build to build the current directory
  • clang-build -d"path/to/dir" to build a different directory (alternatively --directory)
  • clang-build -V to print the called clang commands (alternatively --verbose)

The given directory will be searched for a clang-build.toml file, which you can use to configure your build targets, if necessary. If the build file cannot be found, clang-build will try to create an executable from your project, searching the root and some default folders for sources and headers.

Note: until this is a package on pypi, you need to call python clang-build.py instead of just clang-build...

General Ideas

What should be trivial

This would be things that require only the invocation of clang-build and no build file.

  • build a hello world program (i.e anything with single main and without non-std dependencies)
  • build a reasonable MWE with local dependencies (potentially folder structure with e.g. src, include/MWE and include/thirdparty)
  • include stdlib
  • include anything that can be found by sane default search
  • using command line arguments:
    • specify root/source folder
    • set build type from (last used should be cached/remembered)
    • set build verbosity

Sane defaults and default behaviour:

  • platform-independence
  • build into a "build/" directory, not into toplevel
  • for multiple targets build into "build/target"
  • default search paths for different platforms, including also e.g. "./include", "./lib", "./build/lib", "/usr/local/...", ...

What should be easy

This would be things that only require a minimal TOML project file

  • add dependency / external project from source folder or remote (e.g. github)
    • header-only should be trivial
    • for a regular (not too complicated) library it should be easy to write a build config
  • create a library from one subfolder, an executable from another and link them
  • setting target-specific (note: defaults should be sane!)
    • source file extensions
    • source directories
    • compile and link flags
    • optional version
    • dependencies (which may include non-targets, e.g. configuration steps)
    • properties (required c++ version, definitions/#defines, ...)
  • access to flag "lists" such as flags for
    • coverage
    • cuda
    • openmp
  • set target-specific flags, include folders, etc. which should not be propagated to dependency parents as "private"

What should be possible

Steps that would involve more effort from the user, including possibly some python code

  • a Target configuration step before building (e.g. for more involved version numbering)
  • through the configuration step, inclusion of e.g. CMake-project should be possible
  • packaging: any target may be packaged, meaning it's dependencies are handled and if built, binaries may be bundled
  • external package dependencies
    • binaries on a server
    • source on a server (fallback from binaries)
    • binaries on disk, try to determine version from path and file names
    • source on disk, try to determine version from path and file names

Project File By Example

A single target

# Top-level brackets indicate a target
[hello]
# Note: the following sources settings could be left out.
# .hpp and .cpp files will be searched for in include and src by default
[hello.sources]
file_extensions = [".hpp", ".cpp"]
include_directories = ["include"]
source_directories = ["src"]
# Some properties
[hello.properties]
cpp_version = 17
output_name = "runHello" # name should not contain pre- or suffixes such as lib, .exe, .so

Two targets with linking

# Build a library
[mylib]
target_type = "sharedlibrary"
[mylib.sources]
include_directories = ["mylib/include"]
source_directories = ["mylib/src"]

# Build an executable and link the library
[myexe]
output_name = "runExe"
target_type = "executable"
[myexe.sources]
include_directories = ["myexe/include"]
source_directories = ["myexe/src"]

[myexe.link]
dependencies = ["mylib"]

[myexe.flags]
link = ["-DMYLIB_SOME_DEFINE"]

A package used by a target

mypackage/clang-build.toml

# Build a library
[mylib]
target_type = "library"
[mylib.sources]
include_directories = ["mylib/include"]
source_directories = ["mylib/src"]

myexe/clang-build.toml

# Include an external package/target (i.e. not from this toml file)
[somelib]
external = true
path = "/path/to/sources"

# Build an executable and link the library
[myexe]
[myexe.sources]
include_directories = ["include", "mylib.sources.include_directories"]
source_directories = ["src"]

[myexe.link]
dependencies = ["somelib"]

Packages from server

# Build a library
[mylib]
external = true
url = "https://github.com/trick-17/mylib"
version = 1.1 # will try to git checkout [v]1.1[.*] 

# Build an executable and link the library
[myexe]
[myexe.link]
dependencies = ["mylib"]

Defaults

General

  • all relative paths in a toml are interpreted as relative to that toml file
  • if only one target is built from source, it is built into build/<build_type>
  • if more than one target is built from source, they are built into build/<target_name>/<build_type>

Target properties

  • target and its properties are public by default. The property private = true can be used to set a target to be only visible in the local .toml.

Include Paths

Note: only those paths should be added to the build flags, which the build system finds contain needed files.

Linux

  • .
  • ./include
  • /usr/local
  • /opt
  • $PATH
  • /include

OSX

  • .
  • ./include
  • /usr/local
  • /opt
  • $PATH
  • /include

Windows

  • .
  • ./include
  • %PATH%

External targets

Search paths

The build system should search these paths for folders with names corresponding to the external targets. For paths which were manually specified, the build system should search more deeply to try and find a clang-build.toml and in turn search that for the corresponding target names.

Local:

  • ./
  • ./<target_name>

git server:

  • <url>
  • <url>/<target_name>

clang-build-test's People

Contributors

gpmueller avatar

Watchers

 avatar  avatar

clang-build-test's Issues

Filter output

Currently you are using print statements everywhere. However, if you would use functions like the python logging stuff, you could set different output levels. So people who want to integrate your code into their toolchain could silence your output or select how much output they want.

Path separators

Path separators shouldn't be hardcoded into your scripts. Although it works most of the times, there are modules like PathLib2 that take away all the hassle with python path separators. E.g. if you are on windows, you have a folder that starts with U and you manipulate those strings (like adding a filename to the end of the path), python will not like that newly created string (see here: https://docs.python.org/3/howto/unicode.html for the example I gave).

Add a qt5 usage example (may require e.g. `pre_compile` script)

Since external tools, such as moc, uic, rcc may need to be called, it may be necessary to introduce additional steps, such as pre_compile, post_compile and post_link, which will call a user-provided python script (if present).
The example would then contain a QT MWE and such an additional script which calls the required tools manually.

If this example were successful, it would prove the necessary versatility and applicability for this tool.

Add toml parser and basic external target parsing

Should be able to parse a basic example such as

[myexe]
[myexe.sources]
include_directories = ["myexe/include"]
source_directories = ["myexe/src"]
[myexe.link]
dependencies = ["eigen"]
flags = ["-DEIGEN_NO_FAST_MATH"]

[eigen]
external = true
url = "https://github.com/eigenteam/eigen-git-mirror"

Build folder paths: reflect version numbers and build types?

The question is what should happen if no version is known or the build type is Default...

That aside, it would make sense to have paths such as build/release/targetname/1.0.0/ (with subdirectories for external_sources, obj/source_subfolder, lib, bin, include and possibly others).

For unknown versions, instead of a folder like v1.0.0 maybe vAny?

Rethink build folder structure

After resolving #14, it may be sensible to put files in folders called lib, bin and include (and, for that matter, copy export headers, analogous to CMake install).
The question is if all target build results should go into the same folder or, instead, folders lib/target1 etc. should be created.

Define a way to build only a desired set of targets

For example: a project may define various library targets, which should only be built if actually used (e.g. either a c-interface or a c++-interface).
One may also not want to always build all test executables.

Allow explicit setting of sources without glob

Either via the rule that if sources are specified directly, directories will not be globbed, or via a switch, e.g. search_directories.
Some stupid project layouts have a lot of stuff mixed in a single folder and it may be better to allow this. Unit test .cpp files are generally gathered in a single folder, too.

Rebuild only modified

When traveling through the dependency graph, the build process should only be started when a target with updated sources or config is encountered.

As a first implementation, it should be enough to poll the modification dates and compare it to those in a cache file: https://stackoverflow.com/a/182259/4069571
E.g. a touch.toml could be written into the root of the builddir.

Enable parallel build

For this, a command line option -j should be added, so that e.g. -j4 could be used to create up to 4 parallel threads of build execution.
To make it work, it's merely necessary to spawn an appropriate number of threads when a targets compile/link has completed and the parents are called. Probably this should be done with a thread pool/queue, where it's continuously checked how many threads are currently running, threads are started if possible and the others kept waiting.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.