FMIFlux.jl is a free-to-use software library for the Julia programming language, which offers the ability to place FMUs (fmi-standard.org) everywhere inside of your ML topologies and still keep the resulting model trainable with a standard (or custom) FluxML training process.
License: MIT License
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
I am attaching a screenshot-
see Discourse ...
Hello,
I tried the MS-NeuralFMU example and got the following error while trying to add the FMIFlux.jl package:
pkg> add FMIFlux
Resolving package versions...
ERROR: Unsatisfiable requirements detected for package FMIFlux [fabad875]:
FMIFlux [fabad875] log:
├─possible versions are: 0.1.1-0.6.2 or uninstalled
├─restricted to versions * by an explicit requirement, leaving only versions 0.1.1-0.6.2
├─restricted by compatibility requirements with FMI [14a09403] to versions: 0.3.0-0.6.2 or uninstalled, leaving only versions: 0.3.0-0.6.2
│ └─FMI [14a09403] log:
│ ├─possible versions are: 0.9.0 or uninstalled
│ └─FMI [14a09403] is fixed to version 0.9.0
├─restricted by compatibility requirements with DifferentialEquations [0c46a032] to versions: [0.1.1-0.1.4, 0.4.0-0.6.2] or uninstalled, leaving only versions: 0.4.0-0.6.2
│ └─DifferentialEquations [0c46a032] log:
│ ├─possible versions are: 5.0.0-7.2.0 or uninstalled
│ └─restricted to versions 7.1.0-7 by FMI [14a09403], leaving only versions 7.1.0-7.2.0
│ └─FMI [14a09403] log: see above
└─restricted by compatibility requirements with FMIImport [9fcbc62e] to versions: 0.1.1-0.2.2 or uninstalled — no versions left
└─FMIImport [9fcbc62e] log:
├─possible versions are: 0.5.0-0.10.0 or uninstalled
└─restricted to versions 0.10 by FMI [14a09403], leaving only versions 0.10.0
└─FMI [14a09403] log: see abovePlatform: Julia v1.6.5 on Windows
Any idea what the reason could be?
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!
In src/batch.jl in function batchDataSolution it is written:
if plot
fig = FMIFlux.plot(batch[i-1]; solverKwargs...)
display(fig)
end
instead it should be
if plot
fig = FMIFlux.plot(batch[i]; solverKwargs...)
display(fig)
end
If you call batchDataSolution with option plot=true, you immediately get an error, because there is no batch[0]. Furthermore, I don't see any reason why you want to plot any other batch than the one you just simulated.
Below is a reduced variant of the juliacon_2023.ipynb, I know it is not minimal, but the error is obvious, any more reduction of the script to generate the error is not worth the effort.
# Loading the required libraries
using FMI # import FMUs into Julia
using FMIFlux # for NeuralFMUs
using FMIZoo # a collection of demo models, including the VLDM
using FMIFlux.Flux # Machine Learning in Julia
import JLD2 # data format for saving/loading parameters
import Random # for fixing the random seed
using Plots # plotting results
# a helper file with some predefined functions to make "things look nicer", but are not really relevant to the topic
include(joinpath(@__DIR__, "juliacon_2023_helpers.jl"));
showProgress=false;
dt = 0.1
data = VLDM(:train, dt=dt)
############################################################
############################################################
#->PREPARATION OF SIMULATION DATA (for NN) ->manipulatedDerVals
# start (`tStart`) and stop time (`tStop`) for simulation, saving time points for ODE solver (`tSave`)
tStart = data.consumption_t[1]
tStop = data.consumption_t[end]
tSave = data.consumption_t
fmu = fmiLoad("VLDM", "Dymola", "2020x"; type=:ME)
resultFMU = fmiSimulate(fmu, # the loaded FMU of the VLDM
(tStart, tStop); # the simulation time range
parameters=data.params, # the parameters for the VLDM
showProgress=showProgress, # show (or don't) the progres bar
recordValues=:derivatives, # record all state derivatives
saveat=tSave) # save solution points at `tSave`
# variable we want to manipulate - why we are picking exactly these three is shown a few lines later ;-)
manipulatedDerVars = ["der(dynamics.accelerationCalculation.integrator.y)",
"der(dynamics.accelerationCalculation.limIntegrator.y)",
"der(result.integrator.y)"]
manipulatedDerVals = fmiGetSolutionValue(resultFMU, manipulatedDerVars)
# unload FMU
fmiUnload(fmu)
#############################################################################
###############################################################
# function that builds the considered NeuralFMU on basis of a given FMU (FMI-Version 2.0) `f`
function build_NFMU(f::FMU2)
# pre- and post-processing
preProcess = ShiftScale(manipulatedDerVals) # we put in the derivatives recorded above, FMIFlux shift and scales so we have a data mean of 0 and a standard deivation of 1
preProcess.scale[:] *= 0.25 # add some additional "buffer"
postProcess = ScaleShift(preProcess; indices=2:3) # initialize the postPrcess as inverse of the preProcess, but only take indices 2 and 3 (we don't need 1, the vehcile velocity)
# cache
cache = CacheLayer() # allocate a cache layer
cacheRetrieve = CacheRetrieveLayer(cache) # allocate a cache retrieve layer, link it to the cache layer
# we have two signals (acceleration, consumption) and two sources (ANN, FMU), so four gates:
# (1) acceleration from FMU (gate=1.0 | open)
# (2) consumption from FMU (gate=1.0 | open)
# (3) acceleration from ANN (gate=0.0 | closed)
# (4) consumption from ANN (gate=0.0 | closed)
# the acelerations [1,3] and consumptions [2,4] are paired
gates = ScaleSum([1.0, 1.0, 0.0, 0.0], [[1,3], [2,4]]) # gates with sum
# setup the NeuralFMU topology
model = Chain(x -> f(; x=x, dx_refs=:all), # take `x`, put it into the FMU, retrieve all derivatives `dx`
dx -> cache(dx), # cache `dx`
dx -> dx[4:6], # forward only dx[4, 5, 6]
preProcess, # pre-process `dx`
Dense(3, 32, tanh), # Dense Layer 3 -> 32 with `tanh` activation
Dense(32, 2, tanh), # Dense Layer 32 -> 2 with `tanh` activation
postProcess, # post process `dx`
dx -> cacheRetrieve(5:6, dx), # dynamics FMU | dynamics ANN
gates, # compute resulting dx from ANN + FMU
dx -> cacheRetrieve(1:4, dx)) # stack together: dx[1,2,3,4] from cache + dx[5,6] from gates
# new NeuralFMU
neuralFMU = ME_NeuralFMU(f, # the FMU used in the NeuralFMU
model, # the model we specified above
(tStart, tStop), # a default start ad stop time for solving the NeuralFMU
saveat=tSave) # the time points to save the solution at
neuralFMU.modifiedState = false # speed optimization (NeuralFMU state equals FMU state)
return neuralFMU
end
fmu = fmiLoad("VLDM", "Dymola", "2020x"; type=:ME)
# built the NeuralFMU on basis of the loaded FMU `fmu`
neuralFMU = build_NFMU(fmu)
# a more efficient execution mode
x0 = FMIZoo.getStateVector(data,data.consumption_t[1])
fmiSingleInstanceMode(fmu, true)
train_t = data.consumption_t
train_data = collect([d] for d in data.cumconsumption_val)
BATCHDUR = 4.5
# batch the data (time, targets), train only on model output index 6, plot batch elements
batch = batchDataSolution(neuralFMU, # our NeuralFMU model
t -> FMIZoo.getStateVector(data, t), # a function returning a start state for a given time point `t`, to determine start states for batch elements
train_t, # data time points
train_data; # data cumulative consumption
batchDuration=BATCHDUR, # duration of one batch element
indicesModel=6:6, # model indices to train on (6 equals the state `cumulative consumption`)
plot=true, # don't show intermediate plots (try this outside of Jupyter)
parameters=data.params, # use the parameters (map file paths) from *FMIZoo.jl*
showProgress=showProgress)
Using rrule from ChainRules would be a lot more general and allow using other AD frameworks than Zygote.
Problem Description:
I want to know if FMIFlux can currently chain multiple FMUs and NNs together. For example, I have constructed a circuit system in OpenModelica that includes a voltage source and a resistor. Using OpenModelica, I split this system into three subsystems (voltage source, resistor, and ground), and exported each as an independent FMU. I then tried importing these three subsystem FMUs into OpenModelica and connecting their inputs/outputs for simulation. Unsurprisingly, the simulation results were the same as the original circuit system.
Diagram as follow, from left to right, it sequentially shows the Voltage Source FMU, Resistor FMU, and Ground FMU:
Now, I want to know two things:
Current Attempts:
I found that FMIFlux currently supports using Parallel to chain multiple FMUs and NNs (via Chain). However, after looking into it in detail, I discovered that the Parallel function allows multiple FMUs to run concurrently and produce results. We can then concatenate the results or perform other operations like summation. However, this is different from sequentially chaining multiple subsystem FMUs as I intend.
If there is any misunderstanding on my part, please correct me. Thank you.
You can define a ME-NeuralFMU via
Line 87 in 77471fb
You can call the created nfmu
Line 1044 in 77471fb
Variables which exist as member values but are not overwritten:
PS: haven't checked CS-NeuralFMU.
If you have a ME_NeuralFMU defined with recordValues, which you use in a batchDataSolution, from which you create a FMIFlux.Losses.loss(neuralFMU, batch; ...), then the ForwardDiff.gradient cannot be calculated. You receive a
MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{var"#128#129", Float64}, Float64, 22}).
The "MWE" is a reduced variant of the juliacon2023.ipynb:
using FMI # import FMUs into Julia
using FMIFlux # for NeuralFMUs
using FMIZoo # a collection of demo models, including the VLDM
using FMIFlux.Flux # Machine Learning in Julia
import Random # for fixing the random seed
using Plots # plotting results
include(joinpath(@__DIR__, "juliacon_2023_helpers.jl"));
showProgress=false;
dt = 0.1
data = VLDM(:train, dt=dt)
data_validation = VLDM(:validate, dt=dt)
n = 81#length(data.consumption_t)
tStart = data.consumption_t[1]
tStop = data.consumption_t[n]
tSave = data.consumption_t[1:n]
x0 = FMIZoo.getStateVector(data,tStart)
function build_net(f::FMU2)
# pre- and post-processing
preProcess = ShiftScale{Float64}([-5.363728491534626, -6.257465235165946e-6, -2444.0838108753733], [0.13987191723940867, 0.9502609504008034, 0.00013605664860124656])
preProcess.scale[:] *= 0.25 # add some additional "buffer"
postProcess = ScaleShift(preProcess; indices=2:3) # initialize the postPrcess as inverse of the preProcess, but only take indices 2 and 3 (we don't need 1, the vehcile velocity)
# cache
cache = CacheLayer() # allocate a cache layer
cacheRetrieve = CacheRetrieveLayer(cache) # allocate a cache retrieve layer, link it to the cache layer
gates = ScaleSum([1.0, 1.0, 0.0, 0.0], [[1,3], [2,4]]) # gates with sum
# setup the NeuralFMU topology
model = Chain(x -> f(; x=x, dx_refs=:all), # take `x`, put it into the FMU, retrieve all derivatives `dx`
dx -> cache(dx), # cache `dx`
dx -> dx[4:6], # forward only dx[4, 5, 6]
preProcess, # pre-process `dx`
Dense(3, 2, tanh), # Dense Layer 32 -> 2 with `tanh` activation
postProcess, # post process `dx`
dx -> cacheRetrieve(5:6, dx), # dynamics FMU | dynamics ANN
gates, # compute resulting dx from ANN + FMU
dx -> cacheRetrieve(1:4, dx)) # stack together: dx[1,2,3,4] from cache + dx[5,6] from gates
return model
end
# prepare training data
train_t = data.consumption_t[1:n]
train_data = collect([d] for d in data.cumconsumption_val[1:n])
function _lossFct(solution::FMU2Solution, data::VLDM_Data, LOSS::Symbol, LASTWEIGHT::Real=1.0/length(data.consumption_t) )
# determine the start/end indices `ts` and `te` in the data array (sampled with 10Hz)
ts = dataIndexForTime(solution.states.t[1])
te = dataIndexForTime(solution.states.t[end])
# retrieve the data from NeuralODE ("where we are") and data from measurements ("where we want to be") and an allowed deviation ("we are unsure about")
nfmu_cumconsumption = fmiGetSolutionState(solution, 6; isIndex=true)
cumconsumption = data.cumconsumption_val[ts:te]
cumconsumption_dev = data.cumconsumption_dev[ts:te]
Δcumconsumption = FMIFlux.Losses.mse_last_element_rel_dev(nfmu_cumconsumption, cumconsumption, cumconsumption_dev, LASTWEIGHT)
return Δcumconsumption
end
hyper_params = [0.0001, 0.9, 0.999, 4.0, 0.7, :MSE]
ressource = 8.0#tStop-tStart
TRAINDUR = ressource
ETA, BETA1, BETA2, BATCHDUR, LASTWEIGHT, LOSS = hyper_params
steps = max(round(Int, TRAINDUR/BATCHDUR), 1)
# load our FMU (we take one from the FMIZoo.jl, exported with Dymola 2020x)
fmu = fmiLoad("VLDM", "Dymola", "2020x"; type=:ME)
fmiSingleInstanceMode(fmu, true)
# built the NeuralFMU on basis of the loaded FMU `fmu`
net = build_net(fmu)
neuralFMU = ME_NeuralFMU(fmu, net, (tStart, tStop), saveat=tSave ,recordValues=:derivatives)
neuralFMU.modifiedState = false
params = FMIFlux.params(neuralFMU)
batch = batchDataSolution(neuralFMU, # our NeuralFMU model
t -> FMIZoo.getStateVector(data, t), # a function returning a start state for a given time point `t`, to determine start states for batch elements
train_t, # data time points
train_data; # data cumulative consumption
batchDuration=BATCHDUR, # duration of one batch element
indicesModel=6:6, # model indices to train on (6 equals the state `cumulative consumption`)
plot=true, # don't show intermediate plots (try this outside of Jupyter)
parameters=data.params, # use the parameters (map file paths) from *FMIZoo.jl*
showProgress=showProgress) # show or don't show progess bar, as specified at the very beginning
solverKwargsTrain = Dict{Symbol, Any}(:maxiters => round(Int, 1000*BATCHDUR))
# a smaller dispatch for our custom loss function, only taking the solution object
lossFct = (solution::FMU2Solution) -> _lossFct(solution, data, LOSS, LASTWEIGHT)
scheduler = RandomScheduler(neuralFMU, batch; applyStep=1, plotStep=0)
# initialize the scheduler, keywords are passed to the NeuralFMU
initialize!(scheduler; parameters=data.params, p=params[1], showProgress=showProgress)
# loss for training, do a simulation run on a batch element taken from the scheduler
loss = p -> FMIFlux.Losses.loss(neuralFMU, # the NeuralFMU to simulate
batch; # the batch to take an element from
p=p, # the NeuralFMU training parameters (given as input)
parameters=data.params, # the FMU paraemters
lossFct=lossFct, # our custom loss function
batchIndex=scheduler.elementIndex, # the index of the batch element to take, determined by the choosen scheduler
logLoss=true, # log losses after every evaluation
showProgress=showProgress, # show progress bar (or don't)
solverKwargsTrain...) # the solver kwargs defined above
loss(params[1])
###############################################################
#->the interesting last bit:
grads = zeros(Float64, length(params[1]))
neuralFMU = ME_NeuralFMU(fmu, net, (tStart, tStop),saveat=tSave,recordValues=:derivatives)
FMIFlux.computeGradient!(grads, loss, params[1], :ForwardDiff, :auto_fmiflux, false)
#<-MethodError
################################################################
#redefine neuralFMU without recordValues:
neuralFMU = ME_NeuralFMU(fmu, net, (tStart, tStop),saveat=tSave)
FMIFlux.computeGradient!(grads, loss, params[1], :ForwardDiff, :auto_fmiflux, false)
#<-no error
###############################################################
From what I tested you don't have this problem as long as you don't try batching.
In my current toy problem (of course not the one above ;-)) I only need the values of an FMU-state, which are returned anyway, so that's fine. However, I am not sure how to record the derivatives or a specific output of the FMU, if not via recordValues, which I will definitively need later.
Please provide an example (and presumably missing functionality) to include an input-function into the FMU-layer of an FMIFlux.Chain.
This is crucial when you work with FMUs that only create reasonable output with nonzero input.
Remark:
I know there are several options to define an input-function to an FMU:
inputFunction!(t::T, u::AbstractArray{<:T})
inputFunction!(comp::FMU2Component, t::T, u::AbstractArray{<:T})
inputFunction!(comp::FMU2Component, x::Union{AbstractArray{<:T,1}, Nothing}, u::AbstractArray{<:T})
inputFunction!(x::Union{AbstractArray{<:T,1}, Nothing}, t::T, u::AbstractArray{<:T})
inputFunction!(comp::FMU2Component, x::Union{AbstractArray{<:T,1}, Nothing}, t::T, u::AbstractArray{<:T})
However I am not sure, if support of the more complex function types (i.e. those which do not only depend on t) are actually relevant for many users.
I would be happy, if a purely time-dependent input function was supported.
When extending functionality, please keep in mind that in batched training one might want to use different inputs on different batches. (..as we discussed for the batch-wise parameters. This might be worth a second ticket, once this one is finished).
In Julia, String literals are encoded using the UTF-8 encoding. UTF-8 is a variable-width encoding, meaning that not all characters are encoded in the same number of bytes ("code units"). [...]String indices in Julia refer to code units (= bytes for UTF-8), the fixed-width building blocks that are used to encode arbitrary characters (code points). This means that not every index into a String is necessarily a valid index for a character. If you index into a string at such an invalid byte index, an error is thrown. julia docs
So what's the problem?
In _tryrun, you try to shorten a string to max_msg_len. If max_msg_len hits an invalite byte index, the above error is thrown.
How can it be avoided?
Replace msg[1:max_msg_len] with join(collect(msg)[1:max_msg_len]).
MVE to test? (I can't provide one which uses _tryrun, because I can't share my fmu).
s = "∀ x,a,b,c ∃ y"
io = IOBuffer();
for i in 1:length(s) #<you can also try lastindex(s) here which runs up to the number of counted bytes and is larger than length(s)
try
t = s[1:i]
println("s[1:$i]: $t")
catch e
print("Error with s[1:$i]:t ")
showerror(io, e)
error_msg = String(take!(io))
println("$error_msg")
end
try
t = join(collect(s)[1:i])
println("join(collect(s)[1:$i]): $t")
catch e
print("Error with join(collect(s)[1:i]): ")
showerror(io, e)
error_msg = String(take!(io))
println("$error_msg")
end
end
You should observe that..
The ScaleSum layer is a helpful way to add, e.g. NN-output with FMU-output, however it only works for a Vector of the same length as the scale which has been used in the definition of the ScaleSum layer (in the provided examples, it is 2).
ScaleSum should be able to handle this for more than a Vector. My proposal would be to prepare it for an n-Vector of m-Vectors (where m is the length of the scale vector when defining the layer, typically m=2) and n is the number of inputs that are to be "scale-summed".
Here's a minimal example (including a proposed solution):
using FMIFlux
scale = [0.7 ,0.3]
gates = ScaleSum(scale)
#input as n-Vector of 2-vectors
b1=[1.0,2.0] #<-this is the variant for which ScaleSum works already
b3=[[1.0,2.0],[1.0,2.0],[1.0,2.0]] #<-this is for what it should work
b1Vec=[[1.0,2.0]] #<-this Vector of Vector variant for 1 value pair
#proposed Variant for the ScaleSum-function:
function scsum(x)
if length(x[1])==1#<-input is not Vector of Vector,but only Vector (with the lenght equal to scale)
x_proc = [sum(x.*scale)]
else
x_proc = similar(x[1],length(x))
for i in eachindex(x)
x_proc[i] = sum(x[i].* scale)
end
end
return x_proc
end
#test
using BenchmarkTools
@btime scsum(b1)
#vs.
@btime gates(b1)
#<-same time, same resources, same result
@btime scsum(b1Vec)
#<-same resources, same result as b1, even though it goes through the else branch
@btime scsum(b3)
While running this package's tests on PkgEval (testing the latest Julia master build), we've been seeing several segmentation faults during a type intersection of:
using LinearAlgebra, SparseArrays
x = Tuple{typeof(Base.hcat), Vararg{Union{Number, Array{T, 1} where T, Array{T, 2} where T, LinearAlgebra.AbstractTriangular{T, A} where A<:(Array{T, 2} where T) where T, LinearAlgebra.AbstractTriangular{T, A} where A<:Union{LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv} where T, LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(Array{T, 1} where T) where var"#s968", LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Hermitian{T, A} where A<:(Array{T, 2} where T) where T, LinearAlgebra.Hermitian{T, A} where A<:Union{LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Symmetric{T, A} where A<:(Array{T, 2} where T) where T, LinearAlgebra.Symmetric{T, A} where A<:Union{LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(Array{T, 1} where T) where var"#s966", LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv}, 354}}
y = Tuple{typeof(Base.hcat), Vararg{SparseArrays.SparseVector{Tv, Ti}}} where Ti<:Integer where Tv
typeintersect(x, y)
As noted in JuliaLang/julia#47609, Julia generally isn't expected to handle types of that size.
https://juliadiff.org/ChainRulesCore.jl/dev/api.html#ChainRulesCore.ignore_derivatives
Thanks for the hint @rejuvyesh ! Will be done soon!
Hi,
This is not a bug report. Merely a question (and perhaps clarification of a my misunderstanding :-))
I would like to specify the FMU inputs when calling fmiEvaluteME() and I can see that the fmiEvaluate() has arguments setValueReferences and setValues. But is it possible to set an input function (like for fmiSimulateME) that is evaluated with the input time of the ODE solver. So e.g. one could have something like sin(t) being the input function for the FMU.
The attached script is a reduced and modified variant of simple_hybrid_ME.ipynb.
Removed has been anything unnecessary to reproduce the error.
Modifications are:
recordEigenvaluesSensitivity=:ForwardDiff, recordEigenvalues = true in the loss function (lossSum)gradient =:ForwardDiff# imports
using FMI
using FMIFlux
using FMIFlux.Flux
using FMIZoo
using DifferentialEquations: Tsit5
import Plots
# set seed
import Random
Random.seed!(42);
tStart = 0.0
tStep = 0.01
tStop = 5.0
tSave = collect(tStart:tStep:tStop)
realFMU = fmiLoad("SpringFrictionPendulum1D", "Dymola", "2022x")
fmiInfo(realFMU)
initStates = ["s0", "v0"]
x₀ = [0.5, 0.0]
params = Dict(zip(initStates, x₀))
vrs = ["mass.s", "mass.v", "mass.a", "mass.f"]
realSimData = fmiSimulate(realFMU, (tStart, tStop); parameters=params, recordValues=vrs, saveat=tSave)
posReal = fmi2GetSolutionValue(realSimData, "mass.s")
fmiUnload(realFMU)
simpleFMU = fmiLoad("SpringPendulum1D", "Dymola", "2022x")
# loss function for training
function lossSum(p)
global posReal
solution = neuralFMU(x₀; p=p,recordEigenvaluesSensitivity=:ForwardDiff, recordEigenvalues = true)
posNet = fmi2GetSolutionState(solution, 1; isIndex=true)
FMIFlux.Losses.mse(posReal, posNet)
end
# NeuralFMU setup
numStates = fmiGetNumberOfStates(simpleFMU)
net = Chain(x -> simpleFMU(x=x, dx_refs=:all),
Dense(numStates, 16, tanh),
Dense(16, 16, tanh),
Dense(16, numStates))
neuralFMU = ME_NeuralFMU(simpleFMU, net, (tStart, tStop), Tsit5(); saveat=tSave);
# train
paramsNet = FMIFlux.params(neuralFMU)
optim = Adam()
FMIFlux._train!(lossSum, paramsNet, Iterators.repeated((), 1), optim; gradient =:ForwardDiff)
MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{typeof(lossSum), Float64}, Float64, 32})
Closest candidates are:
(::Type{T})(::Real, ::RoundingMode) where T<:AbstractFloat
@ Base rounding.jl:207
(::Type{T})(::T) where T<:Number
@ Core boot.jl:792
Float64(::IrrationalConstants.Fourπ)
@ IrrationalConstants C:\Users\JUR.julia\packages\IrrationalConstants\vp5v4\src\macro.jl:112
...Stacktrace:
[1] convert(::Type{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(lossSum), Float64}, Float64, 32})
@ Base .\number.jl:7
[2] setindex!(A::Vector{Float64}, x::ForwardDiff.Dual{ForwardDiff.Tag{typeof(lossSum), Float64}, Float64, 32}, i1::Int64)
@ Base .\array.jl:1021
[3] _generic_matvecmul!(C::Vector{…}, tA::Char, A::Matrix{…}, B::Vector{…}, _add::LinearAlgebra.MulAddMul{…})
@ LinearAlgebra C:\Users\JUR\AppData\Local\Programs\julia-1.10.0\share\julia\stdlib\v1.10\LinearAlgebra\src\matmul.jl:743
[4] generic_matvecmul!
@ LinearAlgebra C:\Users\JUR\AppData\Local\Programs\julia-1.10.0\share\julia\stdlib\v1.10\LinearAlgebra\src\matmul.jl:687 [inlined]
[5] mul!
@ LinearAlgebra C:\Users\JUR\AppData\Local\Programs\julia-1.10.0\share\julia\stdlib\v1.10\LinearAlgebra\src\matmul.jl:66 [inlined]
[6] mul!
@ LinearAlgebra C:\Users\JUR\AppData\Local\Programs\julia-1.10.0\share\julia\stdlib\v1.10\LinearAlgebra\src\matmul.jl:237 [inlined]
[7] jvp!(jac::FMISensitivity.FMU2Jacobian{…}, x::Vector{…}, v::Vector{…})
@ FMISensitivity C:\Users\JUR.julia\packages\FMISensitivity\Yt2rV\src\FMI2.jl:1323
[...]
recordEigenvaluesSensitivity=:none in the lossSum.:ReverseDiff (in the lossSum and in _train!), and end up with another error. So that doesn't work either.:none (in lossSum) and :ReverseDiff (in _train!) works, however, if one wants to include the eigenvalues in the senstitivity calculation, this is not an option, is it?*this is actually a bug that this is not updated, but _train! is probably not the preferred resolution, rather train! with the neuralFMU instead of params as the second argument
juliacon_2023_distributedhyperopt.jl, line 17 wants to include workshop_module.jl, however this file does not exist in the repo.
I wonder if this is the *.jl variant of the juliacon_2023.ipynb, with all the code inside the module NODE_Training, but then this somehow doesn't make sense, since it starts a training on all workers with the same parameters immediately.
mutable struct CS_MultiNeuralFMU
model
fmus::Vector{FMU}
tspan
saveat
valueStack
CS_MultiNeuralFMU() = new()
end
function FMIFlux.NeuralFMUInputLayer(fmus::Vector, inputs)
t = inputs[1]
x = inputs[2:end]
for fmu in fmus
NeuralFMUCacheTime(fmu, t)
end
return x
end
function FMIFlux.NeuralFMUOutputLayerCS(fmus::Vector, inputs)
out = inputs
t = fmus[1].t
for fmu in fmus
@assert t == fmu.t
end
vcat([t], out)
end
function CS_MultiNeuralFMU(fmus, model, tspan; saveat=[], addTopLayer=true, addBottomLayer=true, recordValues=[])
nfmu = CS_MultiNeuralFMU()
nfmu.fmus = fmus
if addTopLayer && addBottomLayer
nfmu.model = Chain(inputs -> NeuralFMUInputLayer(nfmu.fmus, inputs),
model.layers...,
inputs -> FMIFlux.NeuralFMUOutputLayerCS(nfmu.fmus, inputs))
elseif addTopLayer
nfmu.model = Chain(inputs -> NeuralFMUInputLayer(nfmu.fmus, inputs),
model.layers...)
elseif addBottomLayer
nfmu.model = Chain(model.layers...,
inputs -> NeuralFMUOutputLayerCS(nfmu.fmus, inputs))
else
end
nfmu.tspan = tspan
nfmu.saveat = saveat
return nfmu
end
function (nfmu::CS_MultiNeuralFMU)(t_step, t_start=nfmu.tspan[1], t_stop=nfmu.tspan[end]; inputs=[], reset::Bool=true)
if reset
for fmu in nfmu.fmus
while fmiReset(fmu) !=0
end
while fmiSetupExperiment(fmu, t_start) !=0
end
while fmiEnterInitializationMode(fmu) !=0
end
while fmiExitInitializationMode(fmu) !=0
end
end
end
ts = t_start:t_step:t_stop
model_input = collect.(eachrow(hcat(ts, inputs)))
nfmu.valueStack = nfmu.model.(model_input)
return nfmu.valueStack
end
function Flux.params(nfmu::CS_MultiNeuralFMU)
Flux.params(nfmu.model)
end
Usage:
fmus = [cartpole_fmu, wind_fmu]
total_fmu_outdim = sum(map(x->length(x.modelDescription.outputValueReferences), fmus))
net = Chain(
Parallel(
vcat,
inputs -> fmi2InputDoStepCSOutput(fmus[1], t_step, inputs[end-1:end]),
inputs -> fmi2InputDoStepCSOutput(fmus[2], t_step, inputs[2:2])
),
Dense(total_fmu_outdim, 16, tanh),
Dense(16, 16, tanh),
Dense(16, length(cartpole_fmu.modelDescription.outputValueReferences)),
)
problem = CS_MultiNeuralFMU(fmus, net, (t_start, t_stop); saveat=t_data)Not exactly DRY, but allows running multiple FMUs in parallel in a Flux.Chain. I can make a pull request if of interest.
Will be fixed in a few hours ...
Dense(10, 20, tanh) works.
Dense(ones(20,10), zeros(20), tanh) fails when grabbing parameters.
This is a feature request.
For FMUs which are not able to get/set their state, the proposed call sequence to batch the training (as in juliacon_2023.ipynb) does not work.
The first instance where it fails is when calling batchDataSolution. Then one receives the warning that the respective FMU is not able to set/get the state. Later, the function run! tries to set this state anyway, and this leads to the crash of the program.
My proposal is to:
run!(neuralFMU::ME_NeuralFMU, batchElement::FMU2SolutionBatchElement; lastBatchElement=nothing, kwargs...)batchDataSolutionfmi2CanGetSetState(neuralFMU.fmu) from batchDataSolution to run!,if fmi2CanGetSetState(neuralFMU.fmu) is true.Due to IP I can't share an MWE here, but @ThummeTo, I can share one with you, and of course discuss.
All examples currently train on a single trajectory. Ideally with NNs it would be great if we could do minibatch training. My guess is that it requires running as many FMU as minibatch size in parallel. It's unclear to me what is required to enable that.
Wrapping a custom walk function as an AnonymousWalk. Future versions will only support custom walks that explicitly subtyle AbstractWalk.
When you call an ME_NeuralFMU
Line 1043 in 177045b
This is since in neural.jl, line 1290 the definition of the sensealg is based on the definition of the solver.
MWE is the script of #120 up to batchDataSolution (which fails).
If you simply leave the sensealg in such cases as nothing (e.g. modify line 1290 to if !isnothing(solver) && isimplicit(solver)), batchDataSolution runs through smoothly. However, I am not sure this is a globally valid solution (which would mean that both the solver and the sensealg are determined by heuristics of other packages!?).
The example script juliacon_2023_distributedhyperopt.jl uses DistributedHyperOpt.jl, which is not included in the Project.toml, and can neither be added via Pkg.add(DistributedHyperOpt.jl).
The script or Project.toml or DistributedHyperOpt.jl should be modified such that adding and using this package is possible. Otherwise the user has a hard time finding this package and can only guess it is one of the ThummeTo-repos.
... see testing scripts. The time for gradient determination seems to increase with every evaluation. This needs to be investigated.
Thanks you for the nice repository! I tried to run the examples and get the following error:
realFMU = fmiLoad(realFMUPath)
[ Info: fmi2Unzip(...): Successfully unzipped 28 files at `/tmp/fmijl_Gr71rS/SpringFrictionPendulum1D`.
ERROR: LoadError: AssertionError: Target platform is Linux, but can't find valid FMU binary at `` for path `/tmp/fmijl_Gr71rS/SpringFrictionPendulum1D`.
I think this is because I work in linux and the FMU only works in windows. Is that possible? Are there also Linux-FMUs for the examples?
When trying to initialize a Dense layer by passing the weight matrix as an argument, an error is thrown.
Examples:
FMIFlux.Dense(ones(2,2))
FMIFlux.Dense(ones(2,2), zeros(2))
The error message is " got unsupported keyword argument "init" " and is caused in flux_overload.jl .
The issue is that Flux.Dense doesnt take an argument init when passing the weight matrix (and bias), however FMIFlux.Dense passes an init argument anyway. These are the valid calls:
Flux.Dense(ones(2,2))
Flux.Dense(ones(2,2), zeros(2))
For the extended ScaleSum (cf. #100) it is helpful to have a matching Cache-Retrieve layer, which prepares value pairs (2-Vectors) of the entered "dx"-Vector and the values of the cache at the specified indices. The current version creates a vector with cached values only before or after the entered "dx"-vector, which even with a resphape cannot be transformed into the expected (and handy) format of a Vector of 2-Vectors.
I propose to define a new Cache-Retrieve-Layer-Variant, as follows:
cache = CacheLayer()
v7 = [1.0,2.0,3.0,4.0,5.0,6.0,7.0]
cv7 = cache(v7)
cRL = CacheRetrieveLayer(cache)
idxs = vcat(1,3,5:6)#
#->I would like to have:
pairVec = [[v7[1],cRL(1)],[v7[3],cRL(3)],[v7[5],cRL(5)],[v7[6],cRL(6)]]
#->proposed new PairWithCacheRetrieve layer to simplify this line:
struct PairWithCacheRetrieve
cacheLayer::CacheLayer
function PairWithCacheRetrieve(cacheLayer::CacheLayer)
inst = new(cacheLayer)
return inst
end
end
export PairWithCacheRetrieve
function (l::PairWithCacheRetrieve)(idx,x)
#for all idx writes cache[idx],x[idx] <-i.e. length of x must be equal to lenght of idx
tid = Threads.threadid()
retVal = typeof(x)[]
sizehint!(retVal, length(idx))
for i in 1:lastindex(idx)
push!(retVal, [l.cacheLayer.cache[tid][idx[i]], x[i]])
end
return retVal
end
#->usage:
cRPairL = PairWithCacheRetrieve(cache)
y = cRPairL(idxs, v7[idxs])
y == pairVec
#->and then this output can be used with the "vectorized" scalesum ;-)
scsum(y)
I think the sizehint and push could be replaced by the correct allocation of the retVal Vector of 2-Vectors which is more performant(!?), but I haven't figured out how to allocate it correctly, yet.
If the chain only contains the fmu layer, the execution fails:
chain = FMIFlux.Chain(x -> fmu(;x=x)[2] )
neuralFMU = ME_NeuralFMU(fmu, chain, (0.0, 1.0), Tsit5())
solution = neuralFMU(x) # Throws BoundsError: attempt to access 0-element Vector{Bool} at index [1] whereas with a layer with parameters, e.g.
chain = FMIFlux.Chain(x -> fmu(;x=x)[2], x->Dense(2 => 2) )the code works.
The error is caused in neural.jl in line 1089: p_len = length(p[1]) as p is empty.
@JuliaRegistrator register
Hi,
I am trying to run the neuralFMUsimple CS example, and have hit a wall when adding the packages.
I am using Ubuntu 22.04 as a VM, and julia 1.8.
I get an error when adding FMIFlux add FMIFlux, as shown here
Any ideas on what is going wrong here?
Calling trainStep with printStep=true leads to an error, since the variable grad is not defined in this function.
Line 1196 in fae8ac4
Creating an MWE is 1000 times more work than the actual fix (of deleting grad). Hence I refrain from providing one.
The parameter printStep is handed over from the train function to trainStep, so running train with printStep=true should show the error.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.