open-simulation-platform / proxy-fmu Goto Github PK

A tool-independent solution for running FMUs compatible with FMI for Co-simulation version 1.x & 2.x in distributed fashion.

License: MIT License

CMake 15.74% Thrift 4.73% C++ 77.63% Python 1.91%

fmi-standard

proxy-fmu's People

Contributors

Stargazers

Watchers

Forkers

ecos-platform arav-jp wingver xmirabel junyang0412 hlyang1992

proxy-fmu's Issues

logging that proxies start

proxyfmu reports instances shutting down. Would be nice to also log when they start. As it takes some seconds before simulation starts due to starting the proxies, it would be good to see what's going on.

Here's an example when used in cosim-cli:

❯ .\cosim.exe run C:\path\to\conf\ -v
info: Using cache directory: "C:\\Users\\user\\AppData\\Local\\cosim"
info: 10% complete, t=0.100000
info: 20% complete, t=0.200000
info: 30% complete, t=0.300000
info: 40% complete, t=0.400000
info: 50% complete, t=0.500000
info: 60% complete, t=0.600000
info: 70% complete, t=0.700000
info: 80% complete, t=0.800000
info: 90% complete, t=0.900000
info: 100% complete, t=1.000000
[proxyfmu] Shutting down proxy for 'Spring::spring' done..
[proxyfmu] External proxy process for instance 'spring' returned with status 0
[proxyfmu] Shutting down proxy for 'Damper::damper' done..
[proxyfmu] External proxy process for instance 'damper' returned with status 0
[proxyfmu] Shutting down proxy for 'Mass::mass' done..
[proxyfmu] External proxy process for instance 'mass' returned with status 0

Automatic spawning of new processes

Enable FMU logging to stdout

Fixed in #56

Compilation error: fmicontext.hpp

A compilation error occurs in src/proxyfmu/fmi/fmicontext.hpp

It is needed to add the following headers:
#include <cstdlib>
#include <cstdio>
#include <array>

Consider using thread::detach

Might solve potential issues with FMUs that won't exit gracefully.

Populate README

Exceptions are not always propagated to client code

It seems that proxy-fmu spawns one or more worker threads to perform some of its operations, and exceptions that are thrown within these threads are not handled properly.

I'm not familiar enough with the proxy-fmu API to provide a minimal test case. I discovered this when running the cosim CLI. Here is one way to reproduce it:

Make sure that cosim CLI is available and compiled with proxy-fmu support.
Delete/rename/move the proxyfmu executable so that the program can't find it.
Run cosim run path/to/OspSystemStructure.xml, where the system structure file is one that uses proxyfmu (for example this one).

Observe the following output (on Linux):

terminate called after throwing an instance of 'std::runtime_error'
  what():  [proxyfmu] No proxyfmu executable found. [path redacted]/bin/proxyfmu does not exist!
Aborted (core dumped)  cosim run OspSystemStructure.xm

Using GDB, I've observed that the exception originates in a worker thread and is never caught before it propagates to the thread entry point and causes the C++ runtime to call std::terminate().

Instead, there should be some mechanism to capture the exception, pass it to the main thread, and rethrow it there, enabling the whole program to exit gracefully.

Link dependencies statically

Link fmi4cpp statically
Link thrift statically
Link libzip statically

proxyfmu fails when there are white space in path

Proxy fails loading FMU when there are white spaces in the path.

Examples below shows running the same configuration with and without white space in the path.

c:\temp\cosim-v0.5.0-win64\bin>cosim.exe run "c:\osp\debugConf\damper test"
[proxyfmu] Found proxyfmu executable: "proxyfmu.exe"
[proxyfmu] Booting FMU instance 'spring'..
[proxyfmu] External proxy process for instance 'spring' returned with status 1
[proxyfmu] Unable to bind to external proxy process!

c:\temp\cosim-v0.5.0-win64\bin>cosim.exe run "c:\osp\debugConf\damper"
[proxyfmu] Found proxyfmu executable: "proxyfmu.exe"
[proxyfmu] Booting FMU instance 'spring'..
[proxyfmu] FMU instance 'spring' instantiated using port 57460
[proxyfmu] Found proxyfmu executable: "proxyfmu.exe"
[proxyfmu] Booting FMU instance 'damper'..
[proxyfmu] FMU instance 'damper' instantiated using port 60564
[proxyfmu] Found proxyfmu executable: "proxyfmu.exe"
[proxyfmu] Booting FMU instance 'mass'..
[proxyfmu] FMU instance 'mass' instantiated using port 62537
[proxyfmu] Shutting down proxy for 'Spring::spring' done..
[proxyfmu] External proxy process for instance 'spring' returned with status 0
[proxyfmu] Shutting down proxy for 'Damper::damper' done..
[proxyfmu] External proxy process for instance 'damper' returned with status 0
[proxyfmu] Shutting down proxy for 'Mass::mass' done..
[proxyfmu] External proxy process for instance 'mass' returned with status 0

Fix CTest on windows CI

Can't run CTest on windows as we need proxyfmu.exe on PATH. I'm unable to find a solution for that in the context of GitHub workflow...

proxyfmu seems to not retry new ports

After #48 got merged I've never seen proxyfmu retry binding to ports. Something that happened all the time previously.

Prior to #48 the errors related to binding to ports resulted in retries:

[proxyfmu] Booting FMU instance 'vessel_model'..
[proxyfmu] Could not bind: errno = 10013. Failed to bind to port 64724. Retrying with another one. Attempt 1 of 10..
[proxyfmu] FMU instance 'vessel_model' instantiated using port 57651

After #48 the process returns with status code 2:

[proxyfmu] Booting FMU instance 'vessel_model'..
[proxyfmu] External proxy process for instance 'vessel_model' returned with status 2

Introduce proper logging

conan2 (from branch) + visual studio

Version from branch conan2 doesn't compile.
Somehow takes 'Visual Studio' as compiler not 'msvc'
even if it's in default profile:

======== Input profiles ========
[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=14
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.version=192
os=Windows

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=14
compiler.runtime=dynamic
compiler.runtime_type=Release
compiler.version=192
os=Windows

Please see the error message:

cmake ..
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19044.
-- The C compiler identification is MSVC 19.28.29337.0
-- The CXX compiler identification is MSVC 19.28.29337.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Current proxyfmu version: 0.3.2

-- Conan: checking conan executable
-- Conan: Found program C:/Soft/Conan/conan/conan.exe
-- Conan: Version found Conan version 2.0.5

-- Conan: Detected VS runtime: MD
-- Conan: checking conan executable
-- Conan: Found program C:/Soft/Conan/conan/conan.exe
-- Conan: Version found Conan version 2.0.5

-- Conan: Adding osp remote repository (https://osp.jfrog.io/artifactory/api/conan/conan-local)
WARN: Remote 'osp' already exists in remotes
-- Conan: Automatic detection of conan settings from cmake
-- Conan: Detected VS runtime: MD
-- Conan: arch was added as an argument. Not using the autodetected one.
-- Conan: build_type was added as an argument. Not using the autodetected one.
-- Conan: compiler was added as an argument. Not using the autodetected one.
-- Conan: compiler.version was added as an argument. Not using the autodetected one.
-- Conan: compiler.runtime was added as an argument. Not using the autodetected one.
-- Conan: Settings= -s;arch=x86_64;-s;build_type=Release;-s;compiler=Visual Studio;-s;compiler.version=16;-s;compiler.runtime=MD
-- Conan: checking conan executable
-- Conan: Found program C:/Soft/Conan/conan/conan.exe
-- Conan: Version found Conan version 2.0.5

-- Conan executing: C:/Soft/Conan/conan/conan.exe install C:/Users/kosekj/repos/IPSi/fmi/OpenSimulationPlatform/proxy-fmu/conanfile.py -s arch=x86_64 -s build_type=Release -s compiler=Visual Studio -s compiler.version=16 -s compiler.runtime=MD -g=cmake --build=missing
ERROR: Invalid setting 'Visual Studio' is not a valid 'settings.compiler' value.
Possible values are ['sun-cc', 'gcc', 'msvc', 'clang', 'apple-clang', 'intel-cc', 'qcc', 'mcst-lcc']
Read "http://docs.conan.io/2/knowledge/faq.html#error-invalid-setting"
CMake Error at cmake/conan.cmake:524 (message):
  Conan install failed='1'
Call Stack (most recent call first):
  cmake/conan.cmake:761 (old_conan_cmake_install)
  CMakeLists.txt:73 (conan_cmake_run)


-- Configuring incomplete, errors occurred!
See also "C:/Users/kosekj/repos/IPSi/fmi/OpenSimulationPlatform/proxy-fmu/build/CMakeFiles/CMakeOutput.log".

proxyfmu process fails to terminate when running specific FMU

Proxyfmu process fails to terminate (terminates after 2 min) when running the gunnerus-palfinger-crane case.
It seems to be the VesselFmu that causes this. It can be reproduced with the following configuration:

OspSystemStructure.xml:

<?xml version="1.0" encoding="utf-8"?>
<OspSystemStructure xmlns="http://opensimulationplatform.com/MSMI/OSPSystemStructure" version="0.1">
	<StartTime>0.0</StartTime>
	<BaseStepSize>1e-3</BaseStepSize>
	<Algorithm>fixedStep</Algorithm>
	<Simulators>
		<Simulator name="VesselFmu" source="proxyfmu://localhost?file=../fmus/VesselFmu.fmu" stepSize="0.05">
			<InitialValues>
				<InitialValue variable="vesselZipFile">
					<String value="absolute\path\to\ShipModel.zip"/>
				</InitialValue>
			</InitialValues>
		</Simulator>
	</Simulators>
</OspSystemStructure>

Using cosim-cli v0.5.0: cosim run \path\to\configuration\above\OspSystemStructure.xml -v the result is:

info: Using cache directory: "C:\\Users\\user\\AppData\\Local\\cosim"
[proxyfmu] Found proxyfmu executable: "proxyfmu.exe"
[proxyfmu] Booting FMU instance 'VesselFmu'..
[proxyfmu] FMU instance 'VesselFmu' instantiated using port 63248
info: 10% complete, t=0.100000
info: 20% complete, t=0.200000
info: 30% complete, t=0.300000
info: 40% complete, t=0.400000
info: 50% complete, t=0.500000
info: 60% complete, t=0.600000
info: 70% complete, t=0.700000
info: 80% complete, t=0.800000
info: 90% complete, t=0.900000
info: 100% complete, t=1.000000
<stuck here for 2 minutes>

Note 1 that this specific FMU requires vesselZipFile variable to be set to the absolute path of the ShipModel.zip as described here

Note 2 I've run the client test (test_controlled_temp) with the VesselFmu without any crash.

std::vector<value_ref> vr{2};
std::vector<std::string> stringVal{"C:\\dev\\osp\\demo-cases\\gunnerus-palfinger-crane\\fmus\\resources\\ShipModel.zip"};
slave->set_string(vr, stringVal);

Push receipt to artifactory

Thrift 1.14.x eventually throws an exception

This issue affects long(ish) running simulations.
Tried different servers and protocols. Same result.
1.13.0 works..

Thrift exception thrown at specific simulation time

Running simulations with proxyfmu seems to crash after some hours (simulation time). For a given OspSystemStructure.xml the exception is thrown at the same simulation time every time.

Here is an example using the Damper.fmu

<?xml version="1.0" encoding="UTF-8"?>
<OspSystemStructure xmlns="http://opensimulationplatform.com/MSMI/OSPSystemStructure"
                    version="0.1">
    <BaseStepSize>0.01</BaseStepSize>
    <Simulators>
        <Simulator name="damper" source="proxyfmu://localhost?file=Damper.fmu" />
    </Simulators>   
</OspSystemStructure>

The case can easily be replicated by using cosim-cli. You can download the cosim executable that includes proxyfmu here.

Run the following command:

.\cosim.exe run C:\path\to\configuration\above\ -d 30000 --mr-progress 1000 --log-level trace

The following will be printed to console after 21840 seconds:

@progress 728 21840.000000 21840.000000
Thrift: Tue Aug 31 08:48:48 2021 TSocket::read() recv() <Host: localhost Port: 60788>: An established connection was aborted by the software in your host machine.

Observations:

If simulating without logging results to csv (--output-config none), the simulation will run for approximately 43200 seconds before crashing. However, this is also happening on the same time for every simulation.
The proxyfmu process is still alive after cosim-cli (libcosim) has crashed, but its state is unknown.
The problem occurs for both FMI 1.0 and 2.0 FMUs
Due to not getting any output from the proxyfmu process it is still not confirmed if it is the proxyfmu or the libcosim side where the problem occurs.

Simulation time for crash with single fmu configurations:
DPController.fmu runs for ~41940 seconds
Damper.fmu runs for ~21840 seconds

Remote connection attempts has no timeout

When connecting to a remote there is no specified timeout duration. If the connection for some reason is unavailable when trying to connect, the current retry loop will throw an error almost immediately as is fails to open a connection to the remote. It is probably better to handle this with a timeout using a retry loop.

Print proxyfmu version on usage

Can't find thrift executable

The thrift recipe has been updated upstream, and now we are unable to locate thrift..

Fails when using proxyfmu_booter

Starting proxyfmu_booter

.\proxyfmu_booter.exe --port 9090

Press any key to quit...
[proxyfmu] Found proxyfmu executable: "proxyfmu.exe"
[proxyfmu] Booting FMU instance 'OSOM'..
[proxyfmu] FMU instance 'OSOM' instantiated using port 65510
[proxyfmu] External proxy process for instance 'OSOM' returned with status -1073741819

Using OspSystemStructure.xml:

<OspSystemStructure xmlns="http://opensimulationplatform.com/MSMI/OSPSystemStructure" version="0.1">
        <StartTime>0.0</StartTime>
        <BaseStepSize>0.05</BaseStepSize>
        <Algorithm>fixedStep</Algorithm>
        <Simulators>
                <Simulator name="OSOM" source="proxyfmu://localhost:9090?file=OSOM.fmu" stepSize="0.05"/>
        </Simulators>
</OspSystemStructure>

Fails when running cosim

 .\cosim.exe run .\OspSystemStructure.xml
error: No more data to read.

It works fine when not using the booter by altering to the following:
<Simulator name="OSOM" source="proxyfmu://localhost?file=OSOM.fmu" stepSize="0.05"/>

error: No resolvers available to handle URI: proxyfmu:xxxxx

Very appreciate your excellent work!

After start cosim cli in terminal, I got error like this: error: No resolvers available to handle URI: proxyfmu://127.0.0.1:9090?file=../DPController.fmu.
When I use distribued Co-simulation part, I can't find where to start the proxyfmu_booter. I can't find this executable file in proxy-fmu/cosim-cli/cosim app demo or anywhere, can anyone help me solve this problem?

Linker issue VS2019 Release

Unable to reproduce. Only happens with release build during conan create on CI server.

Conanify

Integrate Conan with CMake

Remote loading of FMUs

Autogenerate thrift sources

Incorrectly assumes proxyfmu.exe in working directory

Currently the working directory must be equal to location of proxyfmu.exe. Running libcosim should know where to find proxyfmu.

Works:
C:\osp\cosim\bin> .\cosim.exe run C:\path\to\config\

Fails:
C:\osp\cosim> .\bin\cosim.exe run C:\path\to\config\

Missing boost package configuration files when building on ubuntu 20.04

I get CMake errors complaining about missing package configuration files for boost_program_options and boost_unit_test_frameworks when building proxy-fmu on ubuntu 20.04. The files are found if I install the dev version of the missing packages on my system.

Shouldn't this be handled by conan?

`CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package):
Could not find a package configuration file provided by
"boost_unit_test_framework" (requested version 1.71.0) with any of the
following names:

boost_unit_test_frameworkConfig.cmake
boost_unit_test_framework-config.cmake

Add the installation prefix of "boost_unit_test_framework" to
CMAKE_PREFIX_PATH or set "boost_unit_test_framework_DIR" to a directory
containing one of the above files. If "boost_unit_test_framework" provides
a separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
/usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component)
/usr/share/cmake-3.16/Modules/FindBoost.cmake:443 (find_package)
CMakeLists.txt:63 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/thomas/Repositories/OSP/proxy-fmu/cmake-build-debug/CMakeFiles/CMakeOutput.log".`

Implement proper port selection

Can't handle FMUs with parantheses in name on Linux/WSL

Setup:
In OspSystemStructure.xml:

<Simulator name="PropellerAndRudder" source="proxyfmu://localhost?file=path/to/PropellerandFlapRudderwithOWC(BETA).fmu" stepSize="0.05">

Action:

cosim run /path/to/OspSystemStructure.xml

Result:

[proxyfmu] Found proxyfmu executable: "/mnt/c/cosim-v0.7.1-linux64-gcc9/bin/proxyfmu" version: v0.3.1
[proxyfmu] Booting FMU instance 'PropellerAndRudder'..
[proxyfmu] FMU instance 'PropellerAndRudder' instantiated using port 58696
terminate called without an active exception
Aborted

cosim run-single PropellerandFlapRudderwithOWC(BETA).fmu works fine.