abess-team / skscope Goto Github PK
View Code? Open in Web Editor NEWskscope: Sparse-Constrained OPtimization via itErative-solvers
Home Page: https://skscope.readthedocs.io
License: MIT License
skscope: Sparse-Constrained OPtimization via itErative-solvers
Home Page: https://skscope.readthedocs.io
License: MIT License
Related literature about Fast Iterative Shrinkage-Thresholding Algorithm (FISTA): https://www.ceremade.dauphine.fr/~carlier/FISTA
It would be interesting to give users a easy-to-use API for the isotonic regression. The regression implemented skscope
is already provided in example gallery: https://skscope.readthedocs.io/en/latest/gallery/LinearModelAndVariants/isotonic-regression.html. So, this can be easily solved by wrapping the necessary code.
Reference:
[1] Isotonic regression. https://en.wikipedia.org/wiki/Isotonic_regression
[2] Stylianou, Mario, and Nancy Flournoy. “Dose finding using the biased coin up‐and‐down design and isotonic regression.” Biometrics 58.1 (2002): 171-177.
@chenpnn The current implementation is here:
https://skscope.readthedocs.io/en/latest/userguide/examples/SurvivalModels/multivariate-failure-time-model.html, just wrap it.
The scikit-survival
has provided an interface for censored-response, you may borrow related ideas from this library to design your API.
Conduct the command:
make html
but the following error occurs:
/bin/sh: sphinx-build: command not found
make: *** [html] Error 127
But, actually, sphinx
has been installed.
Details about Environment:
Currently, we cannot install skscope
use in python 3.8 via pip
? Can we support it on Python 3.8 on at least some platform such as MacOS or Linux?
Add some badges in the README.md
file (see below). Through them, users will convince our library are well-organized.
[![Python Build](https://github.com/abess-team/abess/actions/workflows/python_test.yml/badge.svg)](https://github.com/abess-team/abess/actions/workflows/python_test.yml)
[![R Build](https://github.com/abess-team/abess/actions/workflows/r_test.yml/badge.svg)](https://github.com/abess-team/abess/actions/workflows/r_test.yml)
[![codecov](https://codecov.io/gh/abess-team/abess/branch/master/graph/badge.svg?token=LK56LHXV00)](https://codecov.io/gh/abess-team/abess)
[![docs](https://readthedocs.org/projects/abess/badge/?version=latest)](https://abess.readthedocs.io/en/latest/?badge=latest)
[![R docs](https://github.com/abess-team/abess/actions/workflows/r_website.yml/badge.svg)](https://abess-team.github.io/abess/)
[![cran](https://img.shields.io/cran/v/abess?logo=R)](https://cran.r-project.org/package=abess)
[![pypi](https://img.shields.io/pypi/v/abess?logo=Pypi)](https://pypi.org/project/abess)
[![Conda version](https://img.shields.io/conda/vn/conda-forge/abess.svg?logo=condaforge)](https://anaconda.org/conda-forge/abess)
[![pyversions](https://img.shields.io/pypi/pyversions/abess)](https://img.shields.io/pypi/pyversions/abess)
[![License](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](http://www.gnu.org/licenses/gpl-3.0)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/3f6e60a3a3e44699a033159633981b76)](https://www.codacy.com/gh/abess-team/abess/dashboard?utm_source=github.com&utm_medium=referral&utm_content=abess-team/abess&utm_campaign=Badge_Grade)
[![CodeFactor](https://www.codefactor.io/repository/github/abess-team/abess/badge)](https://www.codefactor.io/repository/github/abess-team/abess)
[![Platform](https://anaconda.org/conda-forge/abess/badges/platforms.svg)](https://anaconda.org/conda-forge/abess)
[![Downloads](https://pepy.tech/badge/abess)](https://pepy.tech/project/abess)
When I run the example sparse-precision-matrix.ipynb, the following code block
solver = GraspSolver(
dimensionality=int(p * (p + 1) / 2),
sparsity=np.count_nonzero(prec[np.triu_indices(p)]),
always_select=np.where(np.triu_indices(p)[0] == np.triu_indices(p)[1])[0],
convex_solver=convex_solver_cvxpy,
)
raises an error as follows
TypeError: __init__() got an unexpected keyword argument 'convex_solver'
It seems that the new version of scope
is not compatible with the original parameter convex_solver
.
In our examples, we often utilize the re-parametrization trick to absorb the equality constraint into the objective function. I think it would be more convenient if we can wrap common re-parametrizations into functions/classes. The reason is three-fold:
we can avoid re-programming these tricks again and again;
user can easily use re-parametrizations without programming by themselves;
we can improve the maintenance for these re-parametrizations.
@chenpnn I find that the file [utilities.py](https://app.codecov.io/gh/abess-team/skscope/commit/e7031f68f26eed1fd5301b2c511ca1936dbda380/blob/skscope/utilities.py)
is not well covered by python testing.
@bbayukari Shall we our test also cover this utility file?
The notebook uses a Latex environment, which is currently not directly supported by Sphinx
.
So, shall we avoid using this environment? Or, there are some convenience configuration/plugin/library can support this environment? If we consider the latter one, we have to investigate whether it is supported by the readthedocs.
It would be interesting to use skscope
to address the clustering problem.
Reference:
Lashkari, D., & Golland, P. (2007). Convex clustering with exemplar-based models. Advances in neural information processing systems, 20.
Mahdi Soltanolkotabi, Ehsan Elhamifar, Emmanuel J. Candès "Robust subspace clustering," The Annals of Statistics, Ann. Statist. 42(2), 669-699, (April 2014)
Support a new algorithm "Newton Hard-Thresholding Pursuit". See literature: https://spiral.imperial.ac.uk/bitstream/10044/1/89243/4/19-026.pdf
I try to minimize the least square error with simplex constraint via directly optimizing the normalized absolute parameters
w / jnp.sum(w)
and the loss function is shown in the following:
def simplex_solver(X, y, sparsity=None):
n, p = X.shape
def custom_objective(w):
w = jnp.abs(w) / jnp.abs(w).sum()
loss = jnp.mean((X @ w - y) ** 2)
return loss
solver = ScopeSolver(p, sparsity=sparsity)
w = solver.solve(custom_objective)
return w / w.sum()
This fails because all nonzero elements in the output are nan
s, but no warning information raises. Then, I replace the line solver = ScopeSolver(p, sparsity=sparsity)
with solver = ScopeSolver(p, sparsity=sparsity, init_params=np.ones(p)/p)
, it works. This phenomena may be due to the division-by-zero error in w = jnp.abs(w) / jnp.abs(w).sum()
since the default initial params is a all-zeros vector.
Therefore, colud the ScopeSolver
provide more intermediate information when such warning
or error
happens?
Thanks!
It raises the following error when I import this library.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/zhujin/abess-team/scope/scope/__init__.py", line 16, in <module>
from .universal import (ConvexSparseSolver)
File "/Users/zhujin/abess-team/scope/scope/universal.py", line 4, in <module>
from .pybind_cabess import pywrap_Universal, UniversalModel, init_spdlog
ImportError: dlopen(/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so, 0x0002): tried: '/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so' (no such file), '/Users/zhujin/abess-team/scope/scope/pybind_cabess.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
@bbayukari Since skscope
is open-source now, we can submit this package to pypi. I also hope this submission can be automatic once we create a tag
on github.
I notice that the docs of the function convex_solver_nlopt
incorrectly appear in docs/build/html/autoapi/base_solver.html
and docs/build/html/autoapi/solver.html#classes
.
Additionally, the docs of BaseSolver
appears in docs/build/html/autoapi/solver.html
.
I have two suggestions for possible improvements as follows:
skscope
utilize the characteristic of sparse matrix to imporve the efficiency of storage and computation? Some reference may be helpful such as scipy.sparse or its JAX version jax.scipy.sparse.skscope
is mainly based on nlopt
now, and I wonder whether it is possible to support more optimizers. For example, the differentiable optimizer JAXopt may be more suitable for general (unconstrained, constrained and composite) optimization problems .Thanks!
Can our website cooperates with two following useful tool for conducting the examples in gallery:
Just like the website for scikit-learn
https://scikit-learn.org/stable/auto_examples/miscellaneous/plot_kernel_approximation.html#sphx-glr-auto-examples-miscellaneous-plot-kernel-approximation-py
At .../docs,
python -m pip install -r requirements.txt
make html
But there is an error:
ImportError: cannot import name 'environmentfilter' from 'jinja2'
It would be interesting to implement the objective function for solving the square-root-lasso type method.
Reference: A. Belloni and others, Square-root lasso: pivotal recovery of sparse signals via conic programming, Biometrika, Volume 98, Issue 4, December 2011, Pages 791–806, https://doi.org/10.1093/biomet/asr043
[AutoAPI] Reading files... [100%] /Users/zhujin/abess-team/scope/scope/numeric_solver.py
WARNING: Cannot resolve cyclic import: scope.solver, scope, scope.solver
WARNING: Cannot resolve cyclic import: scope.solver, scope, scope.solver
WARNING: Cannot resolve cyclic import: scope.solver, scope, scope.solver
WARNING: Cannot resolve cyclic import: scope.solver, scope, scope.solver
WARNING: Cannot resolve cyclic import: scope.solver, scope, scope.solver
WARNING: Cannot resolve cyclic import: scope.solver, scope, scope.solver
WARNING: Cannot resolve cyclic import: scope.solver, scope, scope.solver
WARNING: Cannot resolve import of scope._scope in scope.solver
WARNING: Cannot resolve import of scope._scope in scope.model
[AutoAPI] Mapping Data... [100%] /Users/zhujin/abess-team/scope/scope/numeric_solver.py
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 57 source files that are out of date
updating environment: [new config] 57 added, 0 changed, 0 removed
docstring of scope.base_solver.BaseSolver:16: WARNING: Unexpected indentation.
docstring of scope.base_solver.BaseSolver:8: WARNING: Inline emphasis start-string without end-string.
docstring of scope.base_solver.BaseSolver:17: WARNING: Block quote ends without a blank line; unexpected unindent.
sphinx-sitemap: No pages generated for sitemap.xml
Exception occurred:
File "/Users/zhujin/miniforge3/envs/convex-solver/lib/python3.9/site-packages/autoapi/directives.py", line 22, in get_items
obj = all_objects[name]
KeyError: 'scope.ScopeSolver'
The full traceback has been saved in /var/folders/tw/stn1nxqn4y77yhy76z8nkpp00000gn/T/sphinx-err-owx0j58n.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [html] Error 2
It would be interesting to implement sparse regression models listed here: vector generalized linear and additive models
@star-dust-ctrl , skscope
is open-source now, and thus, readthedocs
becomes available to us. Please configure related settings, thx!
It seems that the primal-dual active set (PDAS) algorithm is also a (potentially) powerful algorithm for sparsity optimization. It would helpfully enrich our support solvers if it is implemented.
Reference:
Robust variable selection is practically helpful for handling real data because it is insensitive to the outliers that frequently appear in the real world. It is strongly recommended to implement it.
Reference:
seed
with random_state
? The latter is a convention parameter in scikit-learn
.hist
with cov_matrix
? See this: https://pyportfolioopt.readthedocs.io/en/latest/_modules/pypfopt/efficient_frontier/efficient_frontier.html#EfficientFrontier.__init__k
with s
might be better because we use s
in the other places when illustrating sparsity constraint optimization.lambda_
with lambda
?scikit-learn
? For example, you may test whether sklearn.model_selection.GridSearchCV
can be used for selecting lambda
?A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.