renatogeh / gospn Goto Github PK
View Code? Open in Web Editor NEWA free, open-source inference and learning library for Sum-Product Networks (SPN)
License: BSD 3-Clause "New" or "Revised" License
A free, open-source inference and learning library for Sum-Product Networks (SPN)
License: BSD 3-Clause "New" or "Revised" License
Description: DeriveWeights
should compute the derivative dS/dW
, where W
is the multiset of weights in S
. It should allow the user to choose a data structure to perform the graph search (e.g. DFS with a stack, BFS with a queue). It also should use Storer
as a DP table.
File: learn/derive.go
Description: Implement the Dennis-Ventura structure learning algorithm from the paper Learning the Architecture of Sum-Product Networks Using Clustering on Variables NIPS 25.
File: learn/dv.go
References:
Description: Implement the Poon-Domingos SPN structure for local dependency as described in [1] and [2]. Rename learn/poon.go
to model/poon.go
, since it is not a learning algorithm, but a modeling for images and any local dependency dataset.
File: model/poon.go
References:
Description: Implement the Vergari-diMauro structure learning algorithm from the paper Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning. ECMLPKDD 2015.
File: learn/vergari.go
References:
Description: A mixture of k gaussians can be represented as a sum node with k gaussians as children. Learning the weights would then be done with EM.
File: spn/gausmix.go
Fisher's exact test is a contingency table variable independence test ideal for small sized datasets. GoSPN currently implements Pearson's Chi-Square test and the G-test, both of which provide approximations that are quite bad on smaller sample sizes. Since LearnSPN recursively splits data at each step downsizing samples potentially in an exponential rate, an exact test is preferred.
There are, as of yet (and as far as I know), no implementation of the Fisher exact test for the general m
xn
contingency table in Go, C or C++. I have found implementations in R and Python, though I'm not sure the latter is correctly implemented (we should definitely implement unit tests here).
The test itself should be placed at utils/indep/fisher.go
, and its unit test under utils/indep/fisher_test.go
.
hi @RenatoGeh
I tried run go main.go
I got the following error (sample):
...
Creating new leaf...
Sample size: 321, scope size: 1
Creating new leaf...
Sample size: 321, scope size: 1
Creating new leaf...
Testing instance 0. Should be classified as 8.
Pr(X=0|E) = antilog(NaN) = NaN
Pr(X=1|E) = antilog(NaN) = NaN
Pr(X=2|E) = antilog(NaN) = NaN
Pr(X=3|E) = antilog(NaN) = NaN
Pr(X=4|E) = antilog(NaN) = NaN
Pr(X=5|E) = antilog(NaN) = NaN
Pr(X=6|E) = antilog(NaN) = NaN
Pr(X=7|E) = antilog(NaN) = NaN
Pr(X=8|E) = antilog(NaN) = NaN
Pr(X=9|E) = antilog(NaN) = NaN
Pr(X=10|E) = antilog(NaN) = NaN
Pr(X=11|E) = antilog(NaN) = NaN
Pr(X=12|E) = antilog(NaN) = NaN
Pr(X=13|E) = antilog(NaN) = NaN
Pr(X=14|E) = antilog(NaN) = NaN
...
Is it ok got NaN value?
THanks
hcr
I believe that the project should have its own website, preferably a simple one built with github pages. I think it's a good way to register activities, improvements and research on the field.
I would be happy to make this happen if approved :)
We seek to integrate TensorFlow with the goal of implementing RAT-SPNs as proposed in [Peharz et al 2018]. However, as of yet, only Python is covered by the API stability guarantees, meaning it's the only language with the API to build models. Having said that, the Go API is still capable of training the model and running inference.
This means that if we were to add TF integration, we would have to add a Python layer for model creation. Once the model has been created, the user would then be able to export it to GoSPN for learning, running inference or whatever else GoSPN may offer.
This is obviously not ideal, but until Google provides a full API for C++, C or Go, this is the best we can do. Suggestions on how to deal with this are welcome.
References
[Peharz et al 2018] - https://arxiv.org/pdf/1806.01910.pdf
Description: Add continuous variable support for both dataset parsing and Gens-Domingos.
Files: io/arff.go
, io/csv.go
, learn/gens.go
Description: GoSPN only supports a custom .data
and .arff
dataset formats. We plan on adding .csv
support.
File: io/csv.go
Description: Implement generative parameter learning.
File: learn/generative.go
References:
Tasks:
Description: DeriveSPN
should compute the derivative dS/dS_i
of each node in an SPN. It should allow the user to choose a data structure to perform the graph search (e.g. DFS with a stack, BFS with a queue). It also should use Storer
as a DP table.
File: learn/derive.go
Description: My Gens-Domingos complexity analysis is wrong. It currently only takes into account the current iteration of clustering-independencies. It omits the recursive calls to each child it creates.
File: doc/analysis/analysis.tex
Description: Implement discriminative parameter learning.
File: learn/discriminative.go
References:
Description::** Our custom dataset digits
has no proper documentation. We should follow add more information regarding the dataset, as shown in [1]. Another idea is to move the dataset to a completely distinct repository, referencing said repository in the GoSPN docs. If this be the case, we should also remove all other datasets (i.e. caltech
and olivetti
) from this repository.
File: data/digits/README.md
References:
OPTICS (Ordering Points To Identify the Clustering Structure) is a clustering algorithm similar to DBSCAN. DBSCAN's major weakness is density tuning. OPTICS attempts to address this issue by ordering points and choosing the best epsilon.
We currently have an incomplete OPTICS implementation at utils/cluster/optics.go. LearnSPN relies heavily on both clustering and variable independence, and having OPTICS should increase its performance.
This isn't a priority though, as plenty other more interesting structure learning algorithms have sprung up recently.
Description: Our Gens
implementation currently clusters instances with the DBSCAN
and alternatively k-means
clustering algorithms. We wish to add the EM clustering as cited in [1]. Additionally, we intend on implementing both hard and soft EM, as explained in the following quote extracted from [1]:
For soft EM, where instances can be fractionally assigned to clusters, T needs to be extended with a weight for each instance, and each instance is passed to each cluster it has nonzero weight in. However, this is considerably less efficient than hard EM, where each instance is wholly assigned to its most probable cluster, and we will use the latter method.
DBSCAN
and k-means
function like hard EM, assigning instances to the most problable cluster.
Files: learn/gens.go
, utils/cluster/em.go
References:
Description: Implement a language modeling SPN according to the paper Language Modeling with Sum-Product Networks (Cheng et al) INTERSPEECH 2014.
File: models/language.go
References:
GoSPN currently has little to no coverage of modules. Go has a native unit test library, which we should use.
Though marked as good first issue
, we obviously do not expect a single PR with all changes. Instead, please send PRs addressing one checkbox at a time.
The following is our goal coverage list:
app
module:
app/image.go
)app/image.go
)common
module:
common/color.go
)common/equal.go
)common/queue.go
)
common/stack.go
)
conc
module:
conc/queue.go
)data
module:
data/manipulate.go
)
io
module:
io/arff.go
)
io/http.go
)
io/images.go
)
io/input.go
)
io/npy.go
)
io/output.go
)
io/spn.go
)
learn
module:
learn/derive.go
)
learn/discriminative.go
)
learn/generative.go
)
learn/variable.go
)
score
module:
spn
module:
spn/gaussian.go
)spn/product.go
)spn/sum.go
)spn/indicator.go
)spn/multinom.go
)spn/search.go
)spn/serial.go
)spn/storer.go
)spn/topo.go
)spn/utils.go
)
spn/utils.go
)
spn/utils.go
)spn/utils.go
)spn/utils.go
)spn/utils.go
)spn/utils.go
)sys
module:
sys/rand.go
)utils
module:
utils/log.go
)
utils/stats.go
)
utils/unionfind.go
)utils/cluster.go
)Your work is really awesome!
EM is used in LearnSPN, but DBSCAN is used in this project. I wonder Why? Does it have better performance?
Thank you.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.