When planning restructuring of CausalityTools.jl into multiple smaller sub-packages, I have the following undetermined question:
Where do measures that build on the various entropies belong?
There are a plethora of such entropy-based methods in the literature (cross-entropy, joint entropy, conditional entropy, relative entropy or KL divergence, etc.). Although these exist scattered around in other packages such as StatsBase.jl and InformationMeasures.jl, it would be nice if we included them as part of our ecosystem, so that it is possible to compute these quantities using any of our probability estimators.
Currently, mutualinfo
lives in TransferEntropy.jl, but "hiding" mutualinfo
in TransferEntropy.jl seems a bit weird. I think it should be part of a lower level-package. Entropies.jl is a natural candidate package, but it is also conceivable that it belongs elsewhere.
I much more like the idea of having a separate dedicated package to all the "intermediate" measures, in the sense that they build on various entropies. Something like EntropyMeasures.jl
/EntropicMeasures.jl
? Such a package would contain the "basic" entropy-based measures, while more complicated/established-as-its-own-methodology like transfer entropy and friends can exist in their own top-level packages.
This package hierarchy is similar to what you mentioned that you imagine for DynamicalSystems.jl for the multi-repo approach, @Datseris, when discussing moving AbstractDataset
and related functionality to a separate lower-level package.
On the other hand, these measures also naturally belong together with entropy, so we could include them here. Any thoughts on the matter, @Datseris?