Add support for python code generators (from Oryol):

• a python file in a source code directory is executed to generate .h/.cc file during cmake run
• python generators are imported from external projects through a fips.yml entry
• top-level generator.py file in fips, imports 'plugin' generators from imported projects

The new detection macro (FIPS_MSVC et al) seems to not work with some combinations - in my case, CMake 3.1.1 and MSVC 2013 Community Edition. The compiler define isn't set, and this warning is written to the console:

  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "MSVC" will no longer be dereferenced when the policy
  is set to NEW.  Since the policy is not set the OLD behavior will be used.

The fix is to change this to the following, though I'm not sure how compatible this is with other OS'es and/or older CMake versions:

    # detect compiler
    if (${CMAKE_CXX_COMPILER_ID} STREQUAL Clang)
        set(FIPS_CLANG 1)
        message("C++ Compiler: Clang (FIPS_CLANG)")
    elseif (${CMAKE_CXX_COMPILER_ID} STREQUAL GNU)
        set(FIPS_GCC 1)
        message("C++ Compiler: GCC (FIPS_GCC)")
    elseif (${CMAKE_CXX_COMPILER_ID} STREQUAL MSVC)
        set(FIPS_MSVC 1)
        message("C++ Compiler: VStudio (FIPS_MSVC)")
    else()
        message("C++ Compiler: Unknown")
    endif()

It should be cool if you could get ccache support out of the box and not fiddle around with env vars.

there should be 2 new functions fips list apps and fips list libs which list executable and lib targets
in a project (useful for 'fips make' and 'fips run')

emcc does not run out of the box on a new Mac, since /usr/bin/python2 doesn't exist. Probably something for emscripten or the emscripten SDK to fix.

fips init shouldn't do a git clone, but instead simply take a project-directory name.

[ERROR] don't know how to open a 'Unix Makefiles' project

even though i configured cmake for eclipse - is there an option to change that to also get support for eclipse?

The generated main.cc file for a UnitTest has Oryol-specific calls in it: https://github.com/floooh/fips/blob/master/cmake/fips_unittests.cmake#L44, this should be replaced with a project-specific template file (and if the project doesn't provide a template file, use a generic file).

Probably use cmake's configure_file for this: http://www.cmake.org/cmake/help/v3.1/command/configure_file.html

The shader code generator currently not only needs a .shd file with the actual shader source code, but an additional .py file which triggers the code generation. It would be better to have a separate cmake macro which allows to have a cleaner project structure, e.g.:

fips_generate(type args...)

For the shader code generator only the .shd file would be needed in this case, and the shader.py file could be removed:

fips_generate(Shader shader.shd)

at least the unix makefile project could make use of this - maybe as an env var, or cmake set, or ....

See #57, currently a dependency added with fips_deps() must have been declared before so that the second-level dependencies can be resolved.

Not sure whether it's ok to require a specific order (in this case, at least a cmake run should fail with a descriptive error message), or whether the better solution is to move dependency resolution into fips_finish() (may be by just going over all targets in a second pass, and resolve their dependencies).

Add a fips_namespace() function which sets a global variable before importing targets:

fips_namespace(proj1)
fips_import(/dir/to/import)
fips_import(/dir/to/import)
...
don't forget this at the end to clear the namespace variable:
fips_reset_namespace()

A fips_begin_module()/fips_end_module() puts the name namespace name in front of the target name:

proj1_target

When defining dependencies, it is allowed to set targets with a namespace name:

proj1::target (the :: is immediately substituted with a _)

When the project is built locally, the namespace global variable is empty, and thus local dependencies will be resolved correctly.

Maybe I did something wrong, but for me it looks like generating a file from a python script relies on the header file to already exist.

File "/home/mgerhardy/dev/fips/generators/genutil.py", line 65, in isDirty
if fileVersionDirty(dstSrcPath, version) or fileVersionDirty(dstHdrPath, version) :
File "/home/mgerhardy/dev/fips/generators/genutil.py", line 44, in fileVersionDirty
f = open(filePath, 'r')

Getting an IOError telling me that my shaders.h file doesn't exist - which imo is fine, because the generator should just try to create it. I don't want to check that file (the header that should be generated) into version control.

OSX has placeholder java program which opens a dialog box which points to the Oracle Java page.

This confuses the current 'fips diag tools' check.

Do the same git remote check like for project dependencies and warn the user if fips is not uptodate.

This just popped into my mind:

A build target might include a generated header from another module without depending on the other module. Meaning it is not guaranteed that the header is generated or updated before the (unrelated) target which includes header.

One solution might be that modules always setup their fips_deps() as cmake dependency, this would hopefully result in the correct build order when code generation is involved (without code generation the build order of modules and libs doesn't matter).

Is there a way to guarantee, or at least validate correct target dependencies when code generation is involved? We could inspect cmake the cmake target list and dependencies and either setup dependencies automatically or at least dump errors.

Imported projects can extend fips with their own 'verb' plugins to add new functionality to fips. For instance, oryol should implement a 'build-webpage' verb to compile the samples webpage.

'./fips make clean' should work like 'make clean' for all supported build tools

I suggest to add Clang static analyzer to the list of fips targets. It will help to keep project in a good shape. I just run it right now with Oryol project and it shows 0 issues. Cool!!!

Diff:

diff --git a/cmake/fips_private.cmake b/cmake/fips_private.cmake
index 752ca87..5dd9f7f 100644
--- a/cmake/fips_private.cmake
+++ b/cmake/fips_private.cmake
@@ -177,6 +177,8 @@ macro(fips_choose_config)
             set (FIPS_CONFIG "linux-make-release")
         elseif (${CMAKE_BUILD_TYPE} AND ${CMAKE_BUILD_TYPE} STREQUAL "Profiling")
             set (FIPS_CONFIG "linux-make-profiling")
+        elseif (${CMAKE_BUILD_TYPE} AND ${CMAKE_BUILD_TYPE} STREQUAL "ScanBuild")
+            set (FIPS_CONFIG "linux-make-scan")
         else()
             set (FIPS_CONFIG "linux-make-debug")
         endif()
diff --git a/mod/config.py b/mod/config.py
index 80503cb..1db545e 100644
--- a/mod/config.py
+++ b/mod/config.py
@@ -5,7 +5,7 @@ import glob
 import yaml
 from collections import OrderedDict
 from mod import log, util
-from mod.tools import cmake, make, ninja, xcodebuild
+from mod.tools import cmake, make, ninja, xcodebuild, scan
 from mod import emscripten, nacl, android

 platforms = [
@@ -67,13 +67,15 @@ build_tools = [
     'make',
     'ninja',
     'xcodebuild',
+    'scan-build',
     'cmake'
 ]

 build_types = [
     'Release',
     'Debug',
-    'Profiling'
+    'Profiling',
+    'ScanBuild'
 ]

 default_config = {
@@ -226,6 +228,8 @@ def check_build_tool(fips_dir, tool_name) :
         return ninja.check_exists(fips_dir)
     elif tool_name == 'xcodebuild' :
         return xcodebuild.check_exists(fips_dir)
+    elif tool_name == 'scan-build' :
+        return scan.check_exists(fips_dir)
     else :
         return False;

diff --git a/mod/project.py b/mod/project.py
index 8359ae5..cae9162 100644
--- a/mod/project.py
+++ b/mod/project.py
@@ -6,7 +6,7 @@ import subprocess
 import yaml

 from mod import log, util, config, dep, template, settings
-from mod.tools import git, cmake, make, ninja, xcodebuild, ccmake, cmake_gui
+from mod.tools import git, cmake, make, ninja, xcodebuild, scan, ccmake, cmake_gui

 #-------------------------------------------------------------------------------
 def init(fips_dir, proj_name) :
@@ -181,6 +181,8 @@ def make_clean(fips_dir, proj_dir, cfg_name) :
                     result = ninja.run_clean(fips_dir, build_dir)
                 elif cfg['build_tool'] == xcodebuild.name :
                     result = xcodebuild.run_clean(fips_dir, build_dir)
+                elif cfg['build_tool'] == scan.name :
+                    result = scan.run_clean(fips_dir, build_dir)
                 else :
                     result = cmake.run_clean(fips_dir, build_dir)

@@ -240,6 +242,8 @@ def build(fips_dir, proj_dir, cfg_name, target=None) :
                     result = ninja.run_build(fips_dir, target, build_dir, num_jobs)
                 elif cfg['build_tool'] == xcodebuild.name :
                     result = xcodebuild.run_build(fips_dir, target, cfg['build_type'], build_dir, num_jobs)
+                elif cfg['build_tool'] == scan.name :
+                    result = scan.run_build(fips_dir, target, build_dir, num_jobs)
                 else :
                     result = cmake.run_build(fips_dir, target, cfg['build_type'], build_dir)

configs/linux-make-scan.yml:

---
platform: linux 
generator: Unix Makefiles
build_tool: scan-build
build_type: ScanBuild

mod/tools/scan.py:

"""
wrapper for scan-build tool 
"""
import subprocess

name = 'scan-build'
platforms = ['linux'] #, 'osx', 'win']
optional = True
not_found = "required for generating Clang static analyze report"

#-------------------------------------------------------------------------------
def check_exists(fips_dir) :
    """test if make tool is in the path

    :returns: True if make tool is in path
    """
    try:
        out = subprocess.check_output('scan-build -h', shell=True)
        return True
    except (OSError, subprocess.CalledProcessError):
        return False;

#-------------------------------------------------------------------------------
def run_build(fips_dir, target, build_dir, num_jobs=1) :
    """make a build target

    :param target:      name of build target, or None
    :param build_dir:   directory where Makefile is located
    :param num_jobs:    number of jobs, default is 1
    :returns:           True if build was successful
    """
    cmdLine = 'scan-build --use-analyzer /usr/local/bin/clang -o {}/oryol-scan make -j{}'.format(build_dir, num_jobs)
    if target is not None :
        cmdLine += ' ' + target;
    print(cmdLine)
    res = subprocess.call(cmdLine, cwd=build_dir, shell=True)
    return res == 0

#-------------------------------------------------------------------------------
def run_clean(fips_dir, build_dir) :
    """run the special 'clean' target

    :param build_dir:   directory where Makefile is located
    :returns:           True if make returned with success
    """
    return True

The Windows shell interprets the whole command line as the name of the command in subprocess.call.

There's some juggling required with shell=True and not building the command as string but as array of strings.

But this fails on other platforms (for instance 'ninja clean' didn't work).

For emscripten/android/pnacl:

2 ways to use sdks:

• use an existing, externally provided sdk: "fips set [emsdk|androidsdk|androidndk|pnacksdk] [path-to-sdk]"
• setup local sdk (in .fips-sdks) with 'fips setup sdk [emscripten|android|pnacl]"

fips should allow more control over the SDK compared to oryol:

• Android NDK platform version (19, 20, ...)
• PNaCl SDK version (pepper_canary, ...)

At the moment an xxx.py generator script is expected to generate one xxx.cc and one xxx.h file. It isn't possible to generate a .c file, or use different file extensions for C++ files (like .hpp, .cpp, ...). This is a bit tricky because cmake needs to know the extension beforehand since it must create empty placeholder files. One idea would to put the file extension (at least the .cc file extension) into the .py file, e.g.:

xxx.cpp.py -> generates xxx.cpp and xxx.h file
xxx.cpp.hpp.py -> generates a xxx.cpp and xxx.hpp file
xxx.py -> default, generates xxx.cc and xxx.h file

To test the new run-with-args feature, the fips-hello-world 'hello' exe should accept and print an arg, and .travis.ci should do one a 'hello' run with args

It would be nice if you could support me with https://github.com/mgerhardy/fips-flatbuffers

The problem is that flatc can't specify the filename - but only the path - also flatc isn't found properly.

The .gitignore must have a section which is managed by fips:

### fips begin
.fips-settings
### fips end

There should not be a central registry, instead git URLs should be more flexible:

With the short form 'floooh/oryol', fips should assume that the project is on github, otherwise the URL is expected to be a full git URL.

the clone url ends on fips.get - should most likely be fips.git ;)

It should be possible to clone a fips-project without yet having fips itself. The first fips run in the project should then detect that fips is not there and clone it automatically.

Basically it should be possible to build the default-config of any fips-project with a single command:

> ./fips build

looks like node is coming with the emscripten SDK, the check whether it exists can be removed
from 'fips diag tools'

http://mxe.cc/

setting is up is already quite easy - but it would be very cool to get it set up with the fips setup features: http://mxe.cc/#tutorial

At the moment, only directories are defined and source files are globbed with pattern matching. This is convenient but doesn't match cmake's philosophy (the problem is that after adding or removing sources, a manual cmake run must be performed (fips gen). Instead mark the old macro as deprecated and create a new macro which takes a list of files:

fips_files(
  file0.cc file0.h
  file1.cc file1.h
)

Files can contain a directory component, if present, fips will put these into a separate IDE group (similar to now).

This has the advantage that cmake runs automatically after files have been added or removed, because the CMakeLists.txt files must be updated accordingly, which automatically triggers a cmake run at the next build.

• switch on ccache: './fips set ccache on'
• set a ninja or make config: './fips set config osx-ninja-release'
• clean: './fips clean all'
• build: './fips build'

Compile errors because the generator target hasn't been rerun.

Is this because the generated sources are outside the source tree? Didn't this work before but didn't fail because the generated sources were existed (but in the previous version)?

Problem seems to be that the ALL_GENERATE target isn't run before the actual targets which contain code generation files.

Only a minor annoyance, but I don't have Git added to the system path on my Windows system. This is easily avoidable by using Git Bash instead of cmd.exe, but maybe you want to add a check for this.

=== tools:
git: NOT FOUND (git not found in path, can't happen(?))

=== fips:
[...]
File "D:\projects\noise-fips/fips\mod\tools\git.py", line 50, in get_branches
output = subprocess.check_output(['git', 'branch', '-vv'], cwd=proj_dir)
File "C:\Python27\lib\subprocess.py", line 566, in check_output
process = Popen(stdout=PIPE, _popenargs, *_kwargs)
[...]
WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden

E.g.

git clone git://foo/fancyname

CMakeLists.txt creates fips project called fancyname

mv fancyname nameisfancy
./fips run fancyname

^^ doesn't work anymore.

Is that intended? Imo it makes my own preferred directory structure a little bit inflexible. Building and so on still works as intended and puts the binary into the proper location.

fips make
fips make [target]
fips make [target] [config]

target can be special name 'all'

fips should do the following additional checks for configs, and show appropriate diagnostics:

• target platform cross-compiling SDK not found
• build tool not found
• whether config can be built at all on current host platform

...otherwise confusing errors during build happen, e.g.:

Linking CXX shared library android/libs/armeabi-v7a/libClear.so
Buildfile: build.xml does not exist!
Build failed
make[2]: *** [code/Samples/Gfx/Clear/android/libs/armeabi-v7a/libClear.so] Error 1
make[1]: *** [code/Samples/Gfx/Clear/CMakeFiles/Clear.dir/all] Error 2
make: *** [all] Error 2

Just as header and lib search paths, imported projects can define preprocessor defines in their fips.yml file, which are then added to the generated .fips-imports.cmake file with cmake add_definitions() statements.

some systems need to link the atomic lib in order to use std::atomic functionality. I did some sort of detection for simpleai at mgerhardy/simpleai@f811dd5

You might want to consider adding this to fips, too - as you are using c++11 and threads for it anyway.

Instead of setting up new, local SDKs it should be possible to point fips to an existing SDK install for emscripten, PNaCl and Android. Fips diag should then make sure that the SDKs have everything that fips expects.

Probably through a './fips set emscripten-location bla'...

in fips.yml, proj-imports can have an optional branch name, separate with a '#' from the proj-name or git-url:

imports:
    - proj-name-1#branch-name
    - proj-name-2
    - proj-name-3

If a branch name is present, the git clone is performed with "git clone [url] --branch [branch-name]".

E.g. via fips_begin_app({name} shared).

Building a small sample using fips-glfw. While osx-xcode-* works fine, osx-make-* fails with:

Building CXX object fips-glfw_glfw3/CMakeFiles/glfw3.dir/glfw/src/cocoa_monitor.m.o
[ 42%] error: invalid argument '-std=c++11' not allowed with 'C/ObjC'

Somehow CMake tries to compile those files as C++ / with C++ flags.

this should clean all cmake build files and deploy files for one, multiple, all configs

Looks like building an xcode project on the command line currently only the first target (which is not always ALL_BUILD). Need to explicitely build all targets when only './fips build' is called!

The global warning level for compiling should be configurable (3 or 4 warning levels).

I've been trying to specify link libraries with "fips_libs()" for an external library, e.g. in this particular example, some Windows system libraries.

As far as I can see ${CurLinkLibs} is only evaluated for modules and apps, and reset after every library, module and app. Only libs set for an app seem to be passed to the linker.

For now I'm working with fips_libs() with all required libraries inside fips_begin_app()/fips_end_app().

Not sure if you know these tool already:

• https://github.com/toeb/cmakepp
• https://github.com/ruslo/hunter
• https://github.com/iauns/cpm
• http://www.biicode.com

All of them have implemented some sort of external dependencies for cmake. It might be useful to try to model fips around the meta packages of some other tools to reuse the packages from there.

Not sure if this is feasable, but it might be worth a look.

when run from within project directory:

• check for missing imports
• match exports against imports
• check for import name collissions

Generated source file must be in the source tree (because of header search paths in 'upstream' dependent projects), but they shouldn't be under version control. Generate a local .gitignore for generated sources?