Giter VIP home page Giter VIP logo

Comments (24)

joaoleal avatar joaoleal commented on June 4, 2024 1

I'm not sure how your code is structured but if you create the object of the class LinuxDynamicLib inside a function and use it to create the GenericModel object and only return the GenericModel from the function then the LinuxDynamicLib will be deleted when you leave the function (unless you save it somewhere else).

You also have to return the LinuxDynamicLib from the function or save them somewhere else.
You could return both the library and model or just the library.

As a side note, a LinuxDynamicLib can have multiple GenericModels.

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

The folder structure might be different.
Which google test version do you have?

Can you try turning ON the following CMake option:
GOOGLETEST_GIT

It will get the version 1.8.1 from the google test git repository.

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

See my question at the end of this message:
in the build directory, running the command

cmake -D GOOGLETEST_GIT=ON ..

Results in

-- The CXX compiler identification is GNU 9.2.1
-- The C compiler identification is GNU 9.2.1
... snip ...
-- Valgrind found
CMake Warning at test/cppad/cg/dae_index_reduction/CMakeLists.txt:31 (MESSAGE):
     'Eigen3notfound:Dummyderivativestestsdisabled!'
-- Configuring done
-- Generating done
-- Build files have been written to: /home/bradbell/repo/cppad_cg.git/build

What make commands are now available ? For example, the following do not seem to do anything

make example
cd example
make
make all

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

These are the main targets:

make examples
make build_tests
make test
make doc
make install

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

Trying

make test

results in

Running tests...
Test project /home/bradbell/repo/cppad_cg.git/build
        Start   1: array_view
Could not find executable /home/bradbell/repo/cppad_cg.git/build/test/cppad/cg/array_view
Looked in the following places:
/home/bradbell/repo/cppad_cg.git/build/test/cppad/cg/array_view
...

Executing the following comamnds

cd test/cppad/cg
make array_view

generates the output

Scanning dependencies of target create_tmp_folder_cg
[  0%] Creating tmp folder
[  0%] Built target create_tmp_folder_cg
Scanning dependencies of target googletest
[  0%] Creating directories for 'googletest'
[  0%] Performing download step (git clone) for 'googletest'
... snip ...
home/bradbell/prefix/cppad/include/cppad/utility/sparse2eigen.hpp:97:11: 
fatal error: Eigen/Sparse: No such file or directory
97 | # include <Eigen/Sparse>
   |           ^~~~~~~~~~~~~~
compilation terminated.

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

Work related to this is being done in the branch cppad-2020.

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

The tests are now passing in Travis with the new version of CppAD.
Thank you!

from cppadcodegen.

cauachagas avatar cauachagas commented on June 4, 2024

I believe it is a problem to locate Eigen.

In general, the solution for Manjaro and other Linux distributions is

sudo ln -s /usr/include/eigen3/Eigen /usr/include/

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

The file FindEigen3.cmake was taken from Eigen itself.
I would expect it to find eigen even if it was in /usr/include/eigen3/Eigen:

  find_path(EIGEN3_INCLUDE_DIR NAMES signature_of_eigen3_matrix_library
      HINTS
      $ENV{EIGEN_HOME}
      ${CMAKE_INSTALL_PREFIX}/include
      ${KDE4_INCLUDE_DIR}
      PATH_SUFFIXES eigen3 eigen
    )

The error:

home/bradbell/prefix/cppad/include/cppad/utility/sparse2eigen.hpp:97:11: 
fatal error: Eigen/Sparse: No such file or directory
97 | # include <Eigen/Sparse>
   |           ^~~~~~~~~~~~~~
compilation terminated.

appears related to a file in CppAD (cppad/utility/sparse2eigen.hpp).

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

Below is a bash script that I used as a get_started example.
I have included colpack because my version of CppAD has colpack_prefix specified.

#! /bin/bash -e
web_page='https://github.com/joaoleal/CppADCodeGen.git'
cppad_install_prefix="$HOME/prefix/cppad"
colpack_install_prefix="$HOME/prefix/colpack"
eigen_include_dir="$HOME/prefix/eigen/include"
# -----------------------------------------------------------------------------
# bash function that echos and executes a command
echo_eval() {
    echo $*
    eval $*
}
# -----------------------------------------------------------------------------
libdir='none'
for dir in lib lib64
do
    if [ -e "$cppad_install_prefix/$dir" ]
    then
        libdir="$dir"
    fi
done
if [ "$libdir" == 'none' ]
then
    echo 'Cannot find lib or lib64 below cppad_install_prefix:'
    echo "$cppad_install_prefix"
    exit 1
fi
# -----------------------------------------------------------------------------
# Get the source code
echo_eval cd $HOME/install
if [ ! -e cppadcg.git ]
then
    echo_eval git clone "$web_page" cppadcg.git 
fi
echo_eval cd cppadcg.git
echo_eval git checkout master
echo_eval git pull
echo_eval git reset --hard
# -----------------------------------------------------------------------------
# remove old install
if [ -e $HOME/prefix/cppadcg ]
then
    echo_eval rm -r $HOME/prefix/cppadcg
fi
# -----------------------------------------------------------------------------
# Configure and install cppadcg
if [ ! -e build ]
then
    echo_eval mkdir build
fi
cd build
if [ -e CMakeCache.txt ]
then
    rm CMakeCache.txt
fi
cmake \
    -D CREATE_DOXYGEN_DOC=ON \
    -D GOOGLETEST_GIT=ON \
    -D CMAKE_INSTALL_PREFIX="$cppad_install_prefix" \
    -D EIGEN3_INCLUDE_DIR="$eigen_include_dir" \
    ..
make install
#
# -----------------------------------------------------------------------------
# Create get_started.cpp
if [ ! -e get_started ]
then
    echo_eval mkdir get_started
fi
echo_eval cd get_started
cat << EOF > get_started.cpp
# include <iosfwd>
# include <cppad/cg/cppadcg.hpp>
int main(void)
{   typedef CppAD::cg::CG<double>    c_double;
    typedef CppAD::AD<c_double>      ac_double;
    typedef CppAD::vector<c_double>  c_vector;
    typedef CppAD::vector<ac_double> ac_vector;

    // declare independent variables for f(x)
    size_t nx = 2;
    ac_vector ac_x(nx);
    ac_x[0] = 2.0;
    ac_x[1] = 3.0;
    CppAD::Independent(ac_x);

    // create dependent variables and values for f(x)
    size_t nz = 1;
    ac_vector ac_z(nz);
    ac_double ac_diff = ac_x[0] - ac_x[1];
    ac_double ac_sum  = ac_x[0] + ac_x[1];
    ac_z[0] = ac_diff * ac_sum / 2.0;

    // create AD function mapping independent to dependent variables
    // f(x) = (x[0] - x[1]) * (x[0] + x[1]) / 2.0
    CppAD::ADFun<c_double> c_f(ac_x, ac_z);

    // create the source code generator for function g(x) = d/dx f(x)
    // g(x)_0 = ( +(x[0] + x[1]) + (x[0] - x[1]) ) / 2.0 = + x[0]
    // g(x)_1 = ( -(x[0] + x[1]) + (x[0] - x[1]) ) / 2.0 = - x[1]
    CppAD::cg::CodeHandler<double> code_handler;

    // declare the independent variables for g(x)
    c_vector  c_x(nx);
    code_handler.makeVariables(c_x);

    // Compute the dependent variables and values for g(x)
    size_t ny = nz * nx;
    c_vector c_y(ny);
    c_y = c_f.Jacobian(c_x);

    // Mapping from variables in this program to variables in source_code
    // independent variable = x
    // dependent variable   = y
    // temporary variable   = v
    CppAD::cg::LanguageC<double> langC("double");
    CppAD::cg::LangCDefaultVariableNameGenerator<double> nameGen;

    // generate the source code
    std::ostringstream source_code;
    code_handler.generateCode(source_code, langC, c_y, nameGen);

    // number of temporary variables
    size_t nv = code_handler.getTemporaryVariableCount();

    // wrap the string generated by code_handler into a function y = g(x)
    std::string source_str = 
        "namespace {\\n"
        "\\ttemplate <class Vector>\\n"
        "\\tvoid g(const Vector& x, Vector& y)\\n"
        "\\t{\\n"
    ;
    source_str += "\\t\\tVector v(" + CppAD::to_string(nv) + ");\\n"; 
    source_str += "// Begin code generated by CppADCodeGen\\n";
    source_str += source_code.str();
    source_str += 
        "// End code generated by CppADCodeGen\\n"
        "\\t}\\n"
        "} // end namespace\\n"
    ;
    // C souce code corresponding to y = g(x)
    std::cout << source_str;
    
    return 0;
}
EOF
# -----------------------------------------------------------------------------
# Compile get_started.cpp
echo_eval g++ \
    -g \
    -std=c++11 \
    -I $cppad_install_prefix/include \
    -I $eigen_include_dir \
    get_started.cpp -o get_started \
    -L $cppad_install_prefix/$libdir -lcppad_lib \
    -L $colpack_install_prefix/$libdir -lColPack
# -----------------------------------------------------------------------------
# Test y = g(x)
./get_started > test.cpp
cat << EOF >> test.cpp
# include <vector>
# include <limits>
# include <cmath>
int main(void)
{   // initialize flag
    bool   ok   = true;
    // numerical precision for tests
    double eps99 = 99.0 * std::numeric_limits<double>::epsilon();
    // number of components in vectors
    size_t nx    = 2;
    size_t nz    = 1;
    size_t ny    = nz * nx;
    //
    // compute y = g(x)
    std::vector<double> x(nx), y(ny);
    x[0] = 2.0;
    x[1] = 3.0;
    g(x, y);
    //
    // check results
    ok  &= std::fabs(y[0] - x[0] ) < eps99;
    ok  &= std::fabs(y[1] + x[1] ) < eps99;
    if( ! ok ) 
        return 1;
    return 0;  
}
EOF
#
# Compile test
echo_eval g++ \
    -g \
    -std=c++11 \
    -I $cppad_install_prefix/include \
    -I $eigen_include_dir \
    test.cpp -o test \
    -L $cppad_install_prefix/$libdir -lcppad_lib \
    -L $colpack_install_prefix/$libdir -lColPack
# -----------------------------------------------------------------------------
if ! ./test
then
    file="$HOME/install/cppadcg/build/get_started/test.cpp"
    echo "install_cppadcg.sh: Error in $file"
    exit 1
fi
#
echo 'install_cppadcg.sh: OK'
exit 0

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

I've updated the example of creating a dynamic library so that it would be easier to understand how to create the library and then reuse it later on:
dynamic_linux.cpp

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

Given the base2ad feature in CppAD; see the heading 09-19 on
https://coin-or.github.io/CppAD/doc/whats_new_18.htm#09-19
one can create and ADFun that tapes any combination of derivative calculations.

If I can create a model that evaluates the zero order forward mode for an ADFun< CG >,
I think I can re-write my get started example to use that to compute a Jacobian, or any other CppAD calculation.

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

CppADCodeGen already computes Jacobian, Hessian, forward/reverse modes (up to 2nd order).
There is no need to generate multiple ADFuns.
All you need is to turn the calculation you need on or off.

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

I've updated the example of creating a dynamic library so that it would be easier to understand how to create the library and then reuse it later on:
dynamic_linux.cpp

I tried commenting out the following lines in dynamic_linux.cpp:

// save to files (not really required)
// SaveFilesModelLibraryProcessor<double> p2(libcgen);
// p2.saveSources();

While the source code does not get generated, I still get the file model_library.so ?

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

Those lines are only to store the source code in the filesystem for human inspection.
You can remove those and the dynamic library will still get created.
By the way, you can also generate static libraries or use LLVM for JIT compilation.

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

Here is my version of your getting started example. It gives one the flexibility to uses any method to compute the derivatives and combine derivatives in a function:

#include <vector>
#include <cppad/cg.hpp>


const std::string global_library_file_name = "./gradient_norm_squared";

void create_library() {
    // types used for source generation
    typedef CppAD::cg::CG<double>  c_double;
    typedef CppAD::AD<c_double>    ac_double;

    // Start a CppAD recording with ac_x as the independent variable vector
    size_t nx = 2;
    std::vector<ac_double> ac_x(nx);
    CppAD::Independent(ac_x);

    // Dependent variable vector
    size_t ny = 1;
    std::vector<ac_double> ac_y(ny);
    ac_y[0] = ( ac_x[0] * ac_x[0]  + ac_x[1] * ac_x[1] );

    // Stop recording and created c_double verson of
    // f(x) = ( x_0^2 + x_1^2 ) = Euclidean norm of x squared.
    CppAD::ADFun<c_double> c_f(ac_x, ac_y);

    // Create an ac_double version of f(x); i.e.,
    // ac_double is the type for operations by this function and
    // c_double is the Base type when this function was recorded.
    CppAD::ADFun< ac_double, c_double > ac_f;
    ac_f = c_f.base2ad();

    // Start a CppAD recording with ac_x as the independent variable vector
    CppAD::Independent(ac_x);

    // Compute gradient of f(x) using any CppAD operations you like
    ac_f.Forward(0, ac_x);
    std::vector<ac_double> ac_w(ny);
    ac_w[0] = 1.0;
    std::vector<ac_double> ac_z = ac_f.Reverse(1, ac_w); 

    // Stop recrding and create c_double version of g(x) = f'(x)
    CppAD::ADFun<c_double> c_g(ac_x, ac_z);

    // Generate source code
    CppAD::cg::ModelCSourceGen<double> cgen(c_g, "model");
    CppAD::cg::ModelLibraryCSourceGen<double> libcgen(cgen);

    // Compile source code and generator a dynamic library
    CppAD::cg::DynamicModelLibraryProcessor<double> 
        p(libcgen, global_library_file_name);

    CppAD::cg::GccCompiler<double> compiler;
    bool loadLib = false;
    p.createDynamicLibrary(compiler, loadLib);

    // Save to library and source to a file (not required)
    // CppAD::cg::SaveFilesModelLibraryProcessor<double> p2(libcgen);
    // p2.saveSources();
}

bool check_library() {
    bool ok = true;

    // add extension used for dynamic libraries on this system
    const std::string name_plus_ext = global_library_file_name + 
        CppAD::cg::system::SystemInfo<>::DYNAMIC_LIB_EXTENSION;

    // load the library
    CppAD::cg::LinuxDynamicLib<double> dynamicLib(name_plus_ext);
    std::unique_ptr< CppAD::cg::GenericModel<double> > model = 
        dynamicLib.model("model");

    // evaluate f(x)
    std::vector<double> x{2.5, 3.5};
    std::vector<double> z = model->ForwardZero(x);

    // check the result
    ok &= z.size() == 2;
    ok &= z[0] == 2.0 * x[0];
    ok &= z[1] == 2.0 * x[1];

    // return result of check
    return ok;
}

int main() {

    // Add extension used for dynamic libraries on this system
    const std::string name_plus_ext = global_library_file_name + 
        CppAD::cg::system::SystemInfo<>::DYNAMIC_LIB_EXTENSION;

    if (! CppAD::cg::system::isFile(name_plus_ext) ) {
        std::cout << "Creating a new library" << std::endl;
        create_library();
    } else {
        std::cout << "Using existing library" << std::endl;
    }

    bool ok = check_library();
    if( ok )
        return 0;
    return 1;
}

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

In the example
https://github.com/joaoleal/CppADCodeGen/blob/develop/example/dynamic_linux.cpp
If you change the useLibrary function to:

# define BRACE 1
void useLibrary() {
    std::unique_ptr<GenericModel<double>> model;
# if BRACE
    {
# endif
        LinuxDynamicLib<double> dynamicLib(LIBRARY_NAME_EXT);
        model = dynamicLib.model("model");
# if BRACE
    }
# endif
    std::vector<double> xv{2.5, 3.5};
    std::vector<double> jac = model->Jacobian(xv);
    std::cout << jac[0] << " " << jac[1] << std::endl;
}

you will get an error with BRACE equal to 1 and no error with BRACE equal to 0.
This is because model is no longer valid when dynamicLib goes out of scope. This was not obvious to me and took a significant amount of time to track down using the debugger.

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

How did you compile the example?
If you compile it without NDEBUG, then CppAD::ErrorHandler is called with the message "Model library is not ready (possibly closed)".

from cppadcodegen.

bradbell avatar bradbell commented on June 4, 2024

Yes, I got the message below. It was not clear to me when and why _isLibraryReady was changed outside of the class it resides in. I do not know the possible causes as well as you but perhaps it would help say something like this:
"Library corresponding to this model is not ready.
Perhaps it was not loaded, or has dropped out of scope."

cppad-20200131 error from a known source:
Model library is not ready (possibly closed)
Error detected by false result for
    _isLibraryReady
at line 364 in the file 
    /home/bradbell/prefix/cppad/include/cppad/cg/model/functor_generic_model.hpp

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

I'll improve the error message so that it is clearer.

from cppadcodegen.

mohamdev avatar mohamdev commented on June 4, 2024

Hello,

I'm actually getting this error when I instantiate my models at the beginning of my code then try to use them later in different functions. Actually, I would like to instantiate my models only once, then to use them several times in different functions. I stored my different models in a vector std::vector<std::unique_ptr<GenericModel<double>>> models(number_of_models), this way :

    std::vector<std::unique_ptr<GenericModel<double>>> models(3);
    LinuxDynamicLib<Scalar> fkine_ang_IMU1("./fkine_angular_IMU1.so");
    LinuxDynamicLib<Scalar> fkine_ang_IMU2("./fkine_angular_IMU2.so");
    LinuxDynamicLib<Scalar> fkine_ang_IMU3("./fkine_angular_IMU3.so");
    models[0] = fkine_ang_IMU1.model("fkine_angular_IMU1");
    models[1] = fkine_ang_IMU2.model("fkine_angular_IMU2");
    models[2] = fkine_ang_IMU3.model("fkine_angular_IMU3");

Then, for instance, I try to use my models this way : Y = models[i]->ForwardOne(X), where i is the index of the model I want to use. And I get this error : The model library is not ready. The model library that provided this model might have been closed or deleted. Is there a way to solve this error ? Am I misusing GenericModel class ? Or is it just a limitation of the lib ?

from cppadcodegen.

joaoleal avatar joaoleal commented on June 4, 2024

It is likely that, when you use the models from vector, the library containing those models has already been deleted.
You will get that error if the following happens:

std::vector<std::unique_ptr<GenericModel<double>>> models(1);
{
    LinuxDynamicLib<Scalar> fkine_ang_IMU1("./fkine_angular_IMU1.so");
    models[0] = fkine_ang_IMU1.model("fkine_angular_IMU1");
}
Y = models[0]->ForwardOne(x); // in this scope the object fkine_ang_IMU1 has already been deleted

You also have to save the model library objects.

from cppadcodegen.

mohamdev avatar mohamdev commented on June 4, 2024

Thank you for your feedback. What do you mean by "you also have to save the model library objects" ? How do I save them ?

Actually, I use a function called loadModels() that returns a vector std::vector<std::unique_ptr<GenericModel<double>>>. Do you mean that I shouldn't use it this way and just load them at the beggining of my int main() using those 2 lines for each model :

LinuxDynamicLib fkine_ang_IMU1("./fkine_angular_IMU1.so"); models[0] = fkine_ang_IMU1.model("fkine_angular_IMU1");

Thanks for your help

from cppadcodegen.

mohamdev avatar mohamdev commented on June 4, 2024

Great ! I didn't really get the difference between LinuxDynamicLib and GenericModel, I perfectly understand now, thank you very much for your help :)

from cppadcodegen.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.