open-simulation-platform / cosim-cli Goto Github PK

Over the years, we've run into multiple cases of run-time dependency conflicts between Cosim (or its precursor Coral) and FMUs. The conflicts come in two different varieties:

1. Both Cosim and the FMU depend on the same shared library, but in different, incompatible variants/versions. Cosim's dependency will be loaded first and the FMU will be dynamically linked to it, causing crashes and other problems. (Some libraries, like zlib, are popular enough that this risk is actually pretty high.)
2. The FMU calls a function from one of its shared library dependencies, but one of cosim's dependencies have a function with the same name. Since cosim's dependency was loaded first, it will be selected by the dynamic linker, and we're in UB territory. (We recently came across this issue with libzip's zip_close(), when an FMU tried to call a function also called zip_close() from a completely unrelated library.)

These problems can be worked around by spawning off a new process for the FMU using proxyfmu, but that is not always a desirable solution.

I think the most sensible thing to do would be to make static compilation the default for cosim. That way, all symbols are resolved at compile time and won't interfere with any symbols dynamically loaded by the FMU. Since we don't have any control over what FMUs will be run with cosim, this seems like a low-effort way to reduce the risk of conflicts.

Note that if we want to distribute statically linked binaries, then we also need to make sure that the licences of all dependencies are compatible with ours.

As a user, I would like to configure the logging variables by a given file template so that not all the outputs are logged during the simulation run by cli.

If a LogConfig.xml file is present in the system structure directory, this should be used to configure cse::file_observer.

To avoid having to unzip the same FMUs over and over again, CSE CLI should use cse::fmi::importer with a persistent cache directory. A clean-cache subcommand should also be added.

I propose to use $HOME/.cse/fmu-cache on Linux and %LOCALAPPDATA%\CSE\fmu-cache on Windows. (These could be shared by other CSE applications too.)

Hi,

I would like to have the following additional information at the inspect mode so that I can save the information in the database automatically.

• Preferred communication step size (defined under DefaultExperiment)
• FMI version

This might belong in cse-core but I'm seeing it when using cse-cli, so reporting it here. It would be nice if the error message below could be a little bit more specific about exactly which connection or initial value is having problems.
error: conversion of data to type "double" failed

The v0.7.1 release is missing two assets:

• cosim-v0.7.1-win64.zip
• cosim-v0.7.1-linux64-gcc9.zip

I'm assigning this to you @davidhjp01 as you performed the release. Maybe the missing assets can be dug out from Actions or the OSP Conan repo?

cosim run OspSystemStructure.xml will set user defined initial values before cosim::simulator::setup() is called.

cosim run-single MyFmu.fmu foo=bar will set user defined initial values after cosim::simulator::setup() is called, see run_single.cpp:275.

cosim::simulator::setup() eventually calls the FMI initialize() method which means, in the latter case, that the FMU will be initialized before the desired initial values have been set.

I have a number of FMUs which are vitally dependent on a variable having a certain value before fmi::initialize() is called, the prime example being the path of a resource file not known at FMU generation time. Being able to use cosim run-single for debugging would be great. Until now I've had to create a simple OspSystemStructure.xml and use cosim run for this purpose.

Configure Conan to install dll's.
Configure CMake install.
Make a bundle zip.
Archive artifacts on Jenkins.

I would like to have the step size information of an fmu from the cli inspect mode so that I can provide the information to a user in case he/she would like to change the value for a simulator.

Workflows should use https://osp.jfrog.io/artifactory/conan-local repository instead of the virtual repository https://osp.jfrog.io/artifactory/conan-public

The current build from master branch requires an absolute path to FMU file. It only succeeds in the form
C:\>cosim.exe inspect C:\PATH\Spring.fmu
instead of
C:\>cosim.exe inspect .\Spring,fmu
or
C:\>cosim.exe inspect Spring.fmu
Same for release cosim v0.5.0.
Tested on W10.0.17763, command.exe

When creating external tools it would be nice to have some mean of detecting progress.
This progress reporting should be easily parse-able by a computer.
Suggestion:
@progress 1 10
Where 1 is the current step or progress and 10 is the total number of steps

• Introduce a switch to control the output of the CSE logging mechanism: --log-level=<level>.
• Use the same logging mechanism for non-essential CSE CLI output (e.g. the progress of a simulation).

open-simulation-platform/libcosim#110 needs to be fixed first.

The CSE CLI version number is specified in two places: The project() command in CMakeLists.txt and in src/main.cpp. It should only be given in one place, and propagate automatically to any other place where it's needed.

Hi, please update the version number for cosim-cli as well:
Windows10
cosim-v0.7.0-win64.zip
cmd cosim.exe --version
cosim 0.6.0
Using libcosim 0.9.0
instead of cosim 0.7.0
Best regards

CSE CLI (or possibly CSE Core) does not handle (non-absolute) rooted paths correctly on Windows. It's easiest to explain with an example:

C:\foo\bar> cse inspect \path\to\model.fmu
error: ZIP file error: Unzipping of file: 'C:\foo\bar\\path\to\model.fmu' failed with error: No such file

Of course, a correct path resolution should have yielded C:\path\to\model.fmu.

The same problem occurs with run-single, but not with run.

The current release cosim-cli/v0.5.0 does fail in the form
C:\>cosim.exe run-single C:\PATH\OspSystemStructure.xml
with error message
error: No resolvers available to handle URI: file:///C:/PATH/OspSystemStructure.xml
Specifying relative paths ends up in error message
error: internal: cosim::path_to_file_uri: Input requirement not satisfied: path.empty() || path.has_root_directory()
Tested on W10.0.17763, command.exe

As VS 2019 is requiring a default destructor, please dump also one to cli_application.hpp classes cli_subcommand and cli_option_set as it has been done in libcosim.

When I set up this repo, I just copied the .clang-format file from cse-core. However, this treats headers in cse/ differently from other headers, like boost/. To cse-cli, cse-core is just another library, and should not receive special treatment.

Let's fix this once all the current PRs are merged, so we don't have to reformat each and every PR.

Add support for the new format introduced in open-simulation-platform/libcosim#333.

Building, installing and running the Debug version of cosim.exe from the dist folder now causes Windows to complain that yaml-cppd.dll is missing.

Apparently, the yaml-cpp package has added a "d" suffix to the DLL filename when build_type=Debug, and we're not importing that file to dist/ – only yaml-cpp.dll. The problem was probably introduced with the recent upgrade of libcosim's yaml-cpp dependency to 0.7 (open-simulation-platform/libcosim#705).

I'm not sure what is the best way to fix it within the current conanfile.txt. Is there any way to conditionalise on the build type in the [imports] section, or do we have to switch to conanfile.py?

Workarounds: Copy the file from the Conan package directory to dist manually, change the DLL filename in conanfile.txt manually, or compile in Release mode.

The run-single command must check that the step size given with --step-size is smaller than or equal to the total simulation duration, or weird things happen.

Hi,

It will be nice if the cse inspect command returns the model information as xml or json text. It makes it very easy for the external software to retrieve the model information.

Kevin

After commit #91, I was unable to build from sources because of missing src/cosim/fs_portability.hpp
Neither libcosim nor cosim-cli has such file.
If it is my fault, can you pls tell me what to source to make it run?
thanks!

Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Checking Build System
Building Custom Rule C:/Users/XXX/Documents/SystemSimulation/OSP/cosim-cli/CMakeLists.txt
cache.cpp
C:\Users\XXX\Documents\SystemSimulation\OSP\cosim-cli\src\cache.cpp(9,10): fatal error C1083: Cannot open include file: 'cosim/fs_portability.hpp': No such file or directory [C:\Users\XXX\Documents\SystemSimulation\OSP\cosim-cli\b
uild\cosim.vcxproj]
clean_cache.cpp
cli_application.cpp
console_utils.cpp
inspect.cpp
...
Generating Code...

With the provided path, there will be 14 files uploaded
Total file count: 14 ---- Processed file #13 (92.8%)
##[error]Unexpected response. Unable to upload chunk to https://pipelines.actions.***usercontent.com/RyW51QkV5vHA9gNgaq95nxv6TuSvljHTjJzOd8ucSNzlh0NzRZ/_apis/resources/Containers/220006?itemPath=cosim-Linux%2Flib%2Flibcosim.so

Begin Diagnostic HTTP information

Status Code: 500
Status Message: Internal Server Error
Header Information: {
"cache-control": "no-store,no-cache",
"pragma": "no-cache",
"content-length": "333",
"content-type": "application/json; charset=utf-8",
"strict-transport-security": "max-age=2592000",
"x-tfs-processid": "e380fa9c-7de9-4621-957c-63915793422b",
"activityid": "8b05f603-79ed-4c66-b7ee-b4518e93f0b1",
"x-tfs-session": "8b05f603-79ed-4c66-b7ee-b4518e93f0b1",
"x-vss-e2eid": "8b05f603-79ed-4c66-b7ee-b4518e93f0b1",
"x-vss-senderdeploymentid": "13a19993-c6bc-326c-afb4-32c5519f46f0",
"x-frame-options": "SAMEORIGIN",
"x-msedge-ref": "Ref A: 14C8DB5E7A29423B91539F717BB3A23F Ref B: BN3EDGE0417 Ref C: 2020-05-19T07:36:43Z",
"date": "Tue, 19 May 2020 07:36:43 GMT"
}

End Diagnostic HTTP information

##[warning]Aborting upload for /home/runner/work/cse-cli/cse-cli/build/dist/lib/libcosim.so due to failure
##[error]aborting artifact upload
Total size of all the files uploaded is 31228631 bytes
Finished uploading artifact cosim-Linux. Reported size is 31228631 bytes. There were 1 items that failed to upload
##[error]An error was encountered when uploading cosim-Linux. There were 1 items that failed to upload.

The following should be possible to do without Conan:

• Locate dependencies
• Build code
• Install artifacts, optionally bundled with dependencies.

The conan repository url has changed and needs to be updated in Jenkinsfile.

On Windows, the run-single command interprets a path/URI argument like C:\foo.fmu as an URI with scheme C rather than a filesystem path, leading to the following error:

> cse run-single C:\foo.fmu
error: No resolvers available to handle URI: C:\foo.fmu

The same issue was pointed out by @ljamt for the inspect command in PR #2 and fixed there, so it should only be a matter of applying the same fix here.

Currently, cse help simply displays options in the order they are added. Users would most likely prefer them to be sorted alphabetically.

It should be possible to (cleanly) abort running simulations by pressing e.g. a key.

Besides the fact that it may be nice to let users decide for themselves where (possibly quite large amounts of) data are stored, I have a very concrete use case for this: Running misbehaving FMUs that treat their own unpack/resource directories as writable. I've come across a few of those.

An environment variable (COSIM_CACHE_PATH) might be a good way to do this.

Due to fixes in cse-core, it is no longer required.

This would increase performance for lightweight models. This requires some changes to libcosim so that the internal class fmu_model (in orchestration.cpp) has an option to use make_pseudo_async() instead of make_background_thread_slave() to instantiate the model.

When trying to retrieve the logging messages using check_output from Python, it returns an empty byte string.

checkout('cosim run xxx')

Using Popen where it is possible to get stdout and stderr separately, all the info and error messages are found in stderr and stdout is empty.

with Popen(args=['cosim', 'run', 'xxx'], shell=True, stdout=PIPE, stderr=PIPE) as proc:
    log = proc.stdout.read()  # Always empty
    err = proc.stderr.read()  # Contains both normal logging and error message

This is a bit confusing when integrating cosim as a command line tool.