framework project build system about flux-core HOT 6 CLOSED

On the more mundane task of just getting ngrm building again, I added a new pkg-config file called flux-core-srctree in pull request #74 that is not installed, and can be accessed from the GNU Makefile of a project checked out beside it like this:

topdir := $(dir $(lastword $(MAKEFILE_LIST)))
abs_topdir := $(abspath $(topdir))
flux_top_builddir := $(abspath $(abs_topdir)/../flux-core)
flux_top_srcdir := $(abspath $(abs_topdir)/../flux-core)

FLUX_CFLAGS := $(shell PKG_CONFIG_PATH=$(flux_top_builddir)/etc \
        pkg-config --cflags flux-core-srctree)
FLUX_LIBS := $(shell PKG_CONFIG_PATH=$(flux_top_builddir)/etc \
        pkg-config --libs flux-core-srctree)

This provides a relatively simple way to export the information needed to find lua, json-c, zmq, czmq, and to link against -lflux-core and #include <flux/core.h> in a manner such that code will not need to be changed to build against installed vs source tree flux-core.

The utility code duplication problem could be solved by breaking liblsd out to its own installed package and maybe adding optparse and nodeset to it. LSD code would need to be built without the exit-on-error handlers compiled in, and nodset etc would need to not call xzmalloc() and oom() internally. Alternatively, this code could be compiled as a git subpackage in its own repo and shared between flux-core, ngrm, and whatever else needs it. Posponing this decision, I created an ngrm branch ngrm-builds-against-flux-core which simply pulls in any utility bits it needs with vpath, thus it only builds against the checked out source tree, e.g.

# Borrow some source from flux-core source tree that was not
# intended to be exported (FIXME)
#
vpath %.c $(flux_top_srcdir)/src/bindings/lua
vpath %.c $(flux_top_srcdir)/src/common/liblsd
vpath %.c $(flux_top_srcdir)/src/common/libutil

from flux-core.

We changed the approach in ngrm-builds-against-core slightly. IInstead of the flux-core-srctree pkg-config file, now we generate a Makefile.inc in flux-core that can be included in ngrm. In ngrm, run ./config ../flux-core (relative path to flux-core's build directory). This allows more info, such as the srctree locaiton, to be exported from flux-core to other projects, and eliminates the need to edit ngrm's Makefile.inc, which could be a hassle.

from flux-core.

Just for an example, I created a quick CMake based build setup for the flux-barrier repo here: https://github.com/trws/flux-barrier

If we were to go that way, I would probably suggest including a FindFluxCore.cmake or similar in the main project that can populate all of the necessary stuff in any branch project. That way people are free to use a flux-like tree or not as they like. To get this to work now, all it needs is cmake -D FLUX_CORE_PATH=<path to installed flux core> where it will pull in the pkg-config from the current setup and use that to build the xbarrier libraries and module.

It's pretty minimal at the moment, but it should at least give an idea of what the full system would look like.

from flux-core.

@trws: sorry, I have not had time to really focus on this. Would you be willing to submit a pull request to flux-core to add the needed cmake bits?

from flux-core.

Here is a nice comparison of some of the options - albeit one promoting a commercial solution.
http://www.perforce.com/blog/121025/managing-projects-across-git-repositories

from flux-core.

I think we've reached a fairly stable arrangement at least with regard to flux-core and flux-sched. Any further issues can be opened separately.

from flux-core.