jaredleekatzman / deepsurv Goto Github PK
View Code? Open in Web Editor NEWDeepSurv is a deep learning approach to survival analysis.
License: MIT License
DeepSurv is a deep learning approach to survival analysis.
License: MIT License
Followed example given on the homepage for visualizing the training and validation curves, but it gives a 'key error', claiming the log file generated from network.train() does not contain the key 'loss'.
Thanks for the great work @jaredleekatzman, but I have a question about training of DeepSurv. Is there any "batch" split from train set when training or just a whole epoch of example are feed to the network directly?
I was trying to understand why the concordance index was inconsistent when calculating from a loaded model (see this comment here) and discovered this problem: model.X and model.partial_hazard are not correctly saved in save_model.
How to replicate:
model.save_model('bestparams.json', weights_file='bestweights.h5')
model2 = deepsurv.deep_surv.load_model_from_json('bestparams.json', weights_fp='bestweights.h5')
compute_hazards = theano.function(inputs = [model.X],outputs = -model.partial_hazard)
partial_hazards= compute_hazards(x_train)
print(partial_hazards)
model2 = deepsurv.deep_surv.load_model_from_json('bestparams.json', weights_fp='bestweights.h5')
compute_hazards = theano.function(inputs = [model2.X],outputs = -model2.partial_hazard)
partial_hazards2 = compute_hazards(x_train2)
print(partial_hazards2)
This results in an inconsistent concordance index, since you must compute the hazards to get the concordance. I have tried to fix it myself but I don't think I understand Theano tensors well enough. Hope this can be addressed soon.
As I try to run DeepSurv Example.ipynb, I get the following error:
Traceback (most recent call last):
File "D:\Thesis\DeepSurv\deepsurv_eg.py", line 4, in
import deep_surv
ModuleNotFoundError: No module named 'deep_surv'
[Finished in 0.1s with exit code 1]
I have the updated versions of Theano and Lasagne installed.
Is it absolutely necessary to have Theano 0.8.2 to properly run DeepSurv? Would DeepSurv fail to function if the Theano version is different from 0.8.2? Installing DeepSurv rolls back my Theano to an older, outdated version.
My first question:
If I use the following code below, is deepsurv indeed running without any hidden layers?:
hyperparams = {
'n_in': 25,
'learning_rate': 0.0001,
'hidden_layers_sizes': None,
'lr_decay': 0,
'momentum': 0.9,
'L2_reg': 0.0,
'L1_reg': 0,
'activation': 'rectify',
'dropout': 0,
'batch_norm': False,
'standardize': False,
}
network = deepsurv.DeepSurv(**hyperparams)
I can train the model:
python log = network.train(train_data, valid_data, n_epochs = 1300)
This results in a good concordance value. I just want to verify what is being passed. What were your predefined ranges you used for Optunity?
My second question is when I specify hidden_layers_sizes:
When I run the model with hyper-parameters as above, but with hidden_layers_sizes specified,
hyperparams = {
...
'hidden_layers_sizes': [14,2],
...
'activation': 'rectify',
...
}
Then I get the error:
/home/spark/anaconda2/envs/theano-fips/lib/python2.7/site-packages/lasagne/layers/dense.pyc in get_output_for(self, input, **kwargs)
122 if self.b is not None:
123 activation = activation + self.b
--> 124 return self.nonlinearity(activation)
125
126
TypeError: 'str' object is not callable
It's only when I comment out the activation does it model actually attempt to train. But the code should regardless be running rectify, correct?
Hi There:
I just wonder if the load model function is working properly. I can see the weights and updates are saved and reloaded properly, by when calling the saved model to predict on the same data when training the model, the c-index is significantly lower, looks like the the model is not properly specified.
It would be helpful to illustrate the correct way to save and reload a trained model.
Thanks.
Could you please let me know which version of dependencies you used, and the environment for this package.
I run on linux, using Pycharm5.0, python 2.7.11, theano 0.8.2, and got many exceptions of library code, for example "AttributeError: type object 'GCC_compiler' has no attribute 'compile_str' " in lazylinker_c.py.
And here is the error using centos:
Hi All @jaredleekatzman ,
I got an error which I can not figure out why even after reading the hint.... Can anyone help?
In[44]: log = network.train(mypbc_Train,mypbc_Valid,n_epochs=50,validation_frequency=1)
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
log = network.train(mypbc_Train,mypbc_Valid,n_epochs=50,validation_frequency=1)
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 428, in train
loss = train_fn(x_train, e_train)
File "/usr/lib64/python2.7/site-packages/theano/compile/function_module.py", line 871, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/lib64/python2.7/site-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/lib64/python2.7/site-packages/theano/compile/function_module.py", line 859, in call
outputs = self.fn()
File "/usr/lib64/python2.7/site-packages/theano/gof/op.py", line 912, in rval
r = p(n, [x[0] for x in i], o)
File "/usr/lib64/python2.7/site-packages/theano/tensor/basic.py", line 5436, in perform
z[0] = numpy.asarray(numpy.dot(x, y))
ValueError: shapes (234,6) and (10,10) not aligned: 6 (dim 1) != 10 (dim 0)
Apply node that caused the error: dot(x, W)
Toposort index: 8
Inputs types: [TensorType(float32, matrix), TensorType(float64, matrix)]
Inputs shapes: [(234, 6), (10, 10)]
Inputs strides: [(24, 4), (80, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{add,no_inplace}(dot.0, InplaceDimShuffle{x,0}.0), Elemwise{Composite{(i0 * (Abs(i1) + i2 + i3))}}[(0, 2)](TensorConstant{(1, 1) of 0.5}, Elemwise{add,no_inplace}.0, dot.0, InplaceDimShuffle{x,0}.0)]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
File "", line 1, in
log = network.train(mypbc_Train,mypbc_Valid,n_epochs=50,validation_frequency=1)
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 416, in train
update_fn = update_fn, **kwargs
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 254, in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 201, in _get_loss_updates
+ regularize_layer_params(self.network, l2) * L2_reg
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 163, in _negative_log_likelihood
risk = self.risk(deterministic)
File "/usr/lib/python2.7/site-packages/deepsurv/deep_surv.py", line 563, in risk
deterministic = deterministic)
File "/usr/lib/python2.7/site-packages/lasagne/layers/helper.py", line 197, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
File "/usr/lib/python2.7/site-packages/lasagne/layers/dense.py", line 121, in get_output_for
activation = T.dot(input, self.W)
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Can anyone point out why this error happen? Many thanks!
Can deepsurv calculate probabilities of survival at specific time points? When deepsurv recommends Treatment A over Treatment B, I would like to know point estimates with 95% confidence interval at specific time points? It would be easier to understand if deepsurv can produce numbers as below:
Treatment A: 1-year survival 94.5% (95% CI, 90.1-98.9)
Treatment B: 1-year survival 85.3% (95% CI, 80.1-89.5)
If possible, could you provide a code for this?
I encountered several issues when I tried to apply your model to my data.
First, when I imported your package, it immediately reported an error about the “datasets.py”. After a simple change in init.py, it could work.
Second, when I had multiple columns of x in both training and validation data, it reported an error caused by “out of index” in Pandas data frame. I haven’t tried further because I think there might be some other issues. These issues are strange because it seems in your experiments mentioned in your paper, there is not thus issues at all.
Could you please let me know your thoughts about the issue I encountered?
Regards,
Xing
Generally, I believe for a neural network, the loss function consists of comparing the output from the neural network (y_pred) with the already assigned labels/data (y_true). I realize for your lost function, you dod not use y_true, which is the known time events, how can you get the loss updated this way?
Hi,
Thank you for sharing the code. I just have a quick question: Is stochastic gradient descent used for optimization here?
Hi,
thank you very much for this github! I was wondering if you could provide me with a little more detail on how exactly you achieved the c-index reported in the paper. You mention doing a hyperparameter search with 3-fold cross-validation. I would like to benchmark your publication against one that I am soon going to publish.
What did you do once the optimal hyperparameters were obtained? Did you still use a validation set for early stopping, or train until the mean early stopping epoch on the full training set? Or is the prediction an ensemble of the three models from the hyperparameter search?
If I understand your tutorials and code correctly, I cannot just use your docker files since I am modifying the code (adding a couple of line that require another hyperparameter).
Also in your paper, you describe the WHAS data set as consisting of 5 features, but the HDF5 data set when downloaded seems to have 6 features. Am I getting something wrong or is that a typo in the paper?
Also, did you simply use the default values in DeepSurv.train() for n_epochs, patience etc.? If not, where can I get those values from?
Thanks!
Philippe
Hi @jaredleekatzman ,
In your deepsurv front page, you mentioned 'Training DeepSurv can be done in a few lines. First, all you need to do is prepare the datasets to have the following keys: x t and e' How about if my survival data has other features? Will the net model recognize it automatically or should I index them?
Many thanks!
Hi All @jaredleekatzman @vanAmsterdam @dareneiri ,
I am a beginner for using python and really hope to apply this great package. Questions below may seem stupid but I am trying to find a way out...
Much appreciate for any reply!
Xiaozhen
Could you please tag a new version?
The latest stable release, DeepSurv-2.0.0 is not fully compatible with Python 3.
I tried to use the test_deepsurv.py script.
First we created a random data of (5000,9) covariates for training, and (2000,9) for testing. However,when we run the code using 'n_in'=10 in the network hyperparameters it works while it doesn't in the network.predict_risk(self.test['x'])
Would anyone help me to know the meaning of the 10th dimension. I get the following error when I set n_in=9
z[0] = numpy.asarray(numpy.dot(x, y))
ValueError: shapes (5000,10) and (9,10) not aligned: 10 (dim 1) != 9 (dim 0)
Apply node that caused the error: dot(x, W)
Toposort index: 5
Inputs types: [TensorType(float32, matrix), TensorType(float64, matrix)]
Inputs shapes: [(5000, 10), (9, 10)]
Inputs strides: [(40, 4), (80, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{add,no_inplace}(dot.0, InplaceDimShuffle{x,0}.0), Elemwise{Composite{(i0 * (Abs(i1) + i2 + i3))}}[(0, 2)](TensorConstant{(1, 1) of 0.5}, Elemwise{add,no_inplace}.0, dot.0, InplaceDimShuffle{x,0}.0)]]
while if I set it to 10, it throws an exception in the risk prediction as follows:
ValueError: shapes (2000,9) and (10,10) not aligned: 9 (dim 1) != 10 (dim 0)
Apply node that caused the error: dot(x, W)
Toposort index: 2
Inputs types: [TensorType(float32, matrix), TensorType(float64, matrix)]
Inputs shapes: [(2000, 9), (10, 10)]
Inputs strides: [(36, 4), (80, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{(i0 * (Abs((i1 + i2)) + i1 + i2))}}[(0, 1)](TensorConstant{(1, 1) of 0.5}, dot.0, InplaceDimShuffle{x,0}.0)]]
Thank you in advance
Hi Jared,
In your current package you only support datasets with the assumption that there are no tied event times. What do you recommend doing for datasets that have tied event times (say, because time is in discrete intervals). How difficult would it be to implement breslow's method, for example, to handle these tied event times? Or what about jittering the event time so tied events are made unique (if event happened at 300 minutes, and there are duplicates of 300 minutes, then make subseuent event times 300.01, 300.02, etc).
Let's also assume that we need to have all events and cannot just drop the event times that are tied.
Thanks!
Thank you for the wonderful package. The aim of DeepSurv is to recommend a therapy as described in the article. How can I produce survival curves after the calculation of metrics? Could you give me an example of code? I appreciate your help.
https://www.researchgate.net/publication/303812000_Deep_Survival_A_Deep_Cox_Proportional_Hazards_Network
I wanted to use the latest version of DeepSurv based on this repository (primarily to save and load my model), and not on pypi (which was updated June 2016). Can you confirm if the below error is a known issue? While I was able to run my model fine previously, I am no longer able to.
I run:
network = deepsurv.DeepSurv(15,0.0001)
Then:
log = network.train(dataset_TRAIN, dataset_VALID, n_epochs = 300)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-10-aa424000404e> in <module>()
----> 3 log = network.train(dataset_TRAIN, dataset_VALID, n_epochs = 300)
/usr/local/bin/anaconda2/envs/virtual_env/lib/python2.7/site-packages/deepsurv/deep_surv.pyc in train(self, train_data, valid_data, n_epochs, validation_frequency, patience, improvement_threshold, patience_increase, logger, update_fn, **kwargs)
394 learning_rate=lr,
395 momentum = momentum,
--> 396 update_fn = update_fn, **kwargs
397 )
398
/usr/local/bin/anaconda2/envs/virtual_env/lib/python2.7/site-packages/deepsurv/deep_surv.pyc in _get_train_valid_fn(self, L1_reg, L2_reg, learning_rate, **kwargs)
240 loss, updates = self._get_loss_updates(
241 L1_reg, L2_reg, deterministic = False,
--> 242 learning_rate=learning_rate, **kwargs
243 )
244 train_fn = theano.function(
/usr/local/bin/anaconda2/envs/virtual_env/lib/python2.7/site-packages/deepsurv/deep_surv.pyc in _get_loss_updates(self, L1_reg, L2_reg, update_fn, max_norm, deterministic, **kwargs)
206
207 # If the model was loaded from file, reload params
--> 208 if self.restored_update_params:
209 for p, value in zip(updates.keys(), self.restored_update_params):
210 p.set_value(value)
AttributeError: DeepSurv instance has no attribute 'restored_update_params'
Hello,
When i use vix.py to plot log in the code below, I get next error
deepsurv.plot_log(log) network.get_concordance_index(**atest)
`File "C:...\deepsurv\viz.py", line 38, in plot_log
num_epochs = len(log[TRAIN_LOSS])
KeyError: 'loss'
network.get_concordance_index(**atest)
Out[38]: 0.65957446808510634
`
Any idea where this key error comes from ?
Hello,
I get the following error if I run py.test or try to import deepsurv. Any clues? Thanks,
===================================================================== ERRORS =====================================================================
____________________________________________________ ERROR collecting tests/test_deepsurv.py _____________________________________________________
ImportError while importing test module '/run/media/eminozkan/0B46-D280/Cancer/DeepSurv/tests/test_deepsurv.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test_deepsurv.py:4: in <module>
import deepsurv
/home/eminozkan/.conda/envs/life/lib/python3.5/site-packages/deepsurv/__init__.py:1: in <module>
from .deep_surv import DeepSurv
/home/eminozkan/.conda/envs/life/lib/python3.5/site-packages/deepsurv/deep_surv.py:14: in <module>
from deepsurv_logger import DeepSurvLogger
E ImportError: No module named 'deepsurv_logger'
After the training of metrics, can we show the concordance index and negative log likelihood of both training dataset and test dataset at the same time in one figure, respectively? I follow the example of the notebook. Could you provide a code for this?
I have anaconda and created an environment wit python 2.7. Ubuntu 16.04
First, I installed theano 0.8.2 then when i tried pip install Lasagne=0.2.dev1 it couldnt be found.
Then, I went to the docuemntation of Lasagne 0.2 and found that it requires the latest theano which is 0.10.
Here's the link of lassagne because when i use pip install lasagne==0.2.dev1 I get
Could not find a version that satisfies the requirement lasagne==0.2 (from versions: 0.1)
No matching distribution found for lasagne==0.2
These are the packages that I have:
cycler 0.10.0 <pip>
deepsurv 0.1.0 <pip>
functools32 3.2.3.post2 <pip>
Lasagne 0.2.dev1 <pip>
lifelines 0.9.2 <pip>
matplotlib 2.0.2 <pip>
numpy 1.13.1 <pip>
openssl 1.0.2l 0
pandas 0.20.3 <pip>
pip 9.0.1 py27_1
pyparsing 2.2.0 <pip>
python 2.7.13 0
python-dateutil 2.6.1 <pip>
pytz 2017.2 <pip>
readline 6.2 2
scipy 0.19.1 <pip>
setuptools 27.2.0 py27_0
six 1.10.0 <pip>
sqlite 3.13.0 0
subprocess32 3.2.7 <pip>
Theano 0.10.0b1 <pip>
tk 8.5.18 0
wheel 0.29.0 py27_0
zlib 1.2.8 3
When I try to run py.test, I get the following error (I got the same when uninstalled theano and reinstalled the 0.8.2 version):
================================================== test session starts ===================================================
platform linux -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /home/sara/anaconda3/envs/theano_env9, inifile:
collected 0 items / 1 errors
========================================================= ERRORS =========================================================
___________________________________________________ ERROR collecting ____________________________________________________
../../lib/python3.6/site-packages/_pytest/config.py:325: in _getconftestmodules
return self._path2confmods[path]
E KeyError: local('/home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/lasagne')
During handling of the above exception, another exception occurred:
../../lib/python3.6/site-packages/_pytest/config.py:356: in _importconftest
return self._conftestpath2mod[conftestpath]
E KeyError: local('/home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/lasagne/conftest.py')
During handling of the above exception, another exception occurred:
lib/python2.7/site-packages/numpy/core/__init__.py:16: in <module>
from . import multiarray
E ImportError: /home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct
During handling of the above exception, another exception occurred:
lib/python2.7/site-packages/lasagne/__init__.py:12: in <module>
import theano
lib/python2.7/site-packages/theano/__init__.py:42: in <module>
from theano.configdefaults import config
lib/python2.7/site-packages/theano/configdefaults.py:5: in <module>
import numpy
lib/python2.7/site-packages/numpy/__init__.py:142: in <module>
from . import add_newdocs
lib/python2.7/site-packages/numpy/add_newdocs.py:13: in <module>
from numpy.lib import add_newdoc
lib/python2.7/site-packages/numpy/lib/__init__.py:8: in <module>
from .type_check import *
lib/python2.7/site-packages/numpy/lib/type_check.py:11: in <module>
import numpy.core.numeric as _nx
lib/python2.7/site-packages/numpy/core/__init__.py:26: in <module>
raise ImportError(msg)
E ImportError:
E Importing the multiarray numpy extension module failed. Most
E likely you are trying to import a failed build of numpy.
E If you're working with a numpy git repo, try `git clean -xdf` (removes all
E files not under version control). Otherwise reinstall numpy.
E
E Original error was: /home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct
During handling of the above exception, another exception occurred:
../../lib/python3.6/site-packages/_pytest/config.py:362: in _importconftest
mod = conftestpath.pyimport()
../../lib/python3.6/site-packages/py/_path/local.py:662: in pyimport
__import__(modname)
lib/python2.7/site-packages/lasagne/__init__.py:14: in <module>
raise ImportError("Could not import Theano." + install_instr)
E ImportError: Could not import Theano.
E
E Please make sure you install a recent enough version of Theano. Note that a
E simple 'pip install theano' will usually give you a version that is too old
E for Lasagne. See the installation docs for more details:
E http://lasagne.readthedocs.org/en/latest/user/installation.html#theano
During handling of the above exception, another exception occurred:
../../lib/python3.6/site-packages/py/_path/common.py:367: in visit
for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../../lib/python3.6/site-packages/py/_path/common.py:416: in gen
for p in self.gen(subdir):
../../lib/python3.6/site-packages/py/_path/common.py:416: in gen
for p in self.gen(subdir):
../../lib/python3.6/site-packages/py/_path/common.py:416: in gen
for p in self.gen(subdir):
../../lib/python3.6/site-packages/py/_path/common.py:405: in gen
dirs = self.optsort([p for p in entries
../../lib/python3.6/site-packages/py/_path/common.py:406: in <listcomp>
if p.check(dir=1) and (rec is None or rec(p))])
../../lib/python3.6/site-packages/_pytest/main.py:682: in _recurse
ihook = self.gethookproxy(path)
../../lib/python3.6/site-packages/_pytest/main.py:587: in gethookproxy
my_conftestmodules = pm._getconftestmodules(fspath)
../../lib/python3.6/site-packages/_pytest/config.py:339: in _getconftestmodules
mod = self._importconftest(conftestpath)
../../lib/python3.6/site-packages/_pytest/config.py:364: in _importconftest
raise ConftestImportFailure(conftestpath, sys.exc_info())
E _pytest.config.ConftestImportFailure: ImportError("Could not import Theano.\n\nPlease make sure you install a recent enough version of Theano. Note that a\nsimple 'pip install theano' will usually give you a version that is too old\nfor Lasagne. See the installation docs for more details:\nhttp://lasagne.readthedocs.org/en/latest/user/installation.html#theano",)
E File "/home/sara/anaconda3/envs/theano_env9/lib/python2.7/site-packages/lasagne/__init__.py", line 14, in <module>
E raise ImportError("Could not import Theano." + install_instr)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================ 1 error in 0.87 seconds =================================================
Hello and thank you for sharing your great work.
I have trained a network :
network = deepsurv.DeepSurv(10,0.0001) log = network.train(d_train, d_valid, n_epochs=500)
[INFO] Training CoxMLP Finished Training with 500 iterations in 20.05s
It seems it was able to predict a risk vector :
network.predict_risk(d_valid['x'])
array([[ -5.24271558e+00], [ -1.13109659e+03], [ 4.15896825e+00], [ -7.84299117e+00], [ 1.62468140e+01], [ -3.04364481e+00], [ -3.50972074e+00], [ -2.26831663e+02], [ -3.07352921e+00], [ -1.57916871e+00], .....
but when using get_concordance_index() like this :
network.get_concordance_index(d_test['x'],d_test['t'],d_test['e'])
I get this error :
`---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
in ()
----> 1 network.get_concordance_index(d_test['x'],d_test['t'],d_test['e'])
/Library/anaconda/lib/python3.5/site-packages/deepsurv/deep_surv.py in get_concordance_index(self, x, t, e, **kwargs)
268 return concordance_index(t,
269 partial_hazards,
--> 270 e)
271
272 def train(self,
/Library/anaconda/lib/python3.5/site-packages/lifelines/utils/init.py in concordance_index(event_times, predicted_event_times, event_observed)
384 return _concordance_index(event_times,
385 predicted_event_times,
--> 386 event_observed)
387
388
/Library/anaconda/lib/python3.5/site-packages/lifelines/utils/init.py in _concordance_index(event_times, predicted_event_times, event_observed)
857 died_ix = next_ix
858 else:
--> 859 assert not (has_more_died or has_more_censored)
860 break
861
AssertionError:`
Have you any idea of how I could fix it ?
Thank you !
Hello,
I've read your paper and when looking for the negative log likelihood equation and implementation it looks like they are not the same. In the paper there's equation 3 which is like this:
But looking at the code it seems that you are not using the risk set (R(t)) and it behaves like this (notice the absence of R(Ti) in the second summation):
Is there a reason for this?
Hi,
>>> import deepsurv
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/deepsurv/__init__.py", line 1, in <module>
from .deep_surv import DeepSurv
File "/usr/local/lib/python2.7/dist-packages/deepsurv/deep_surv.py", line 1, in <module>
import lasagne
File "/usr/local/lib/python2.7/dist-packages/lasagne/__init__.py", line 19, in <module>
from . import layers
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/__init__.py", line 7, in <module>
from .pool import *
File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/pool.py", line 6, in <module>
from theano.tensor.signal import downsample
ImportError: cannot import name downsample
>>> import theano
>>> theano.__version__
'0.9.0'
Ubuntu 16.04 LTS
I have large amount of data and the training takes quite a lot of time. Therefore I wonder how to save and load the trained network?
On Line 166: uncensored_likelihood = risk.T - log_risk
Why is there a .T
what does that mean?
I am working on the public datasets a this link: https://vincentarelbundock.github.io/Rdatasets/datasets.html
in particular I have tried "aids2" and "flchain".
Nevertheless in both cases log takes the following values:
{'best_params': None,
'best_params_idx': -1,
'best_valid_ci': 0.5,
'best_validation_loss': inf,
'train': [array(28340558372.044426),
array(nan),
array(nan),
array(nan),
array(nan)],
'train_ci': [0.5, 0.5, 0.5, 0.5, 0.5],
'valid': [array(nan), array(nan), array(nan), array(nan), array(nan)],
'valid_ci': [0.5, 0.5, 0.5, 0.5, 0.5]}
what can be done to solve this issue?
Hi,
I'm a beginner here, and I had a couple questions about DeepSurv. When doing "normal" cox proportional hazards survival analysis, we can analyze the impact of each covariate on the survival outcome (with hazard ratios) and calculate p-values. Does DeepSurv have the same univariate capabilities or is it supposed to be used on ALL the covariates at once? If so, how do we know which covariates have a statistically significant effect on the survival?
Edit: When running DeepSurv with a certain number of epochs, running it over and over results in different concordance indices. They seem to vary drastically (0.42 to 0.15 to 0.55, etc.). Am I doing something wrong?
Thanks so much!
I try to tune the hyper parameter for whas dataset and get the following error:
Attaching to hyperparamsearch_hp_search_1
hp_search_1 | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
hp_search_1 | from ._conv import register_converters as _register_converters
hp_search_1 | 2018-01-19 08:11:29,163 - __main__ - DEBUG - Parameters: Namespace(box='/box_constraints.0.json', dataset='whas', logdir='/shared/logs', num_epochs=500, num_evals=100, num_folds=3, update_fn='adam')
hp_search_1 | 2018-01-19 08:11:29,163 - __main__ - DEBUG - Loading dataset: whas
hp_search_1 | Traceback (most recent call last):
hp_search_1 | File "/hyperparam_search.py", line 196, in <module>
hp_search_1 | x, y, strata = load_dataset(args.dataset)
hp_search_1 | File "/hyperparam_search.py", line 105, in load_dataset
hp_search_1 | ds = utils.load_datasets(dataset)['train']
hp_search_1 | File "/deepsurv/utils.py", line 17, in load_datasets
hp_search_1 | with h5py.File(dataset_file, 'r') as fp:
hp_search_1 | File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 271, in __init__
hp_search_1 | fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
hp_search_1 | File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 101, in make_fid
hp_search_1 | fid = h5f.open(name, flags, fapl=fapl)
hp_search_1 | File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-nCYoKW-build/h5py/_objects.c:2840)
hp_search_1 | File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-nCYoKW-build/h5py/_objects.c:2798)
hp_search_1 | File "h5py/h5f.pyx", line 78, in h5py.h5f.open (/tmp/pip-nCYoKW-build/h5py/h5f.c:2117)
hp_search_1 | IOError: Unable to open file (Unable to open file: name = 'whas', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)
hyperparamsearch_hp_search_1 exited with code 1
my docker file is as follows:
FROM floydhub/dl-docker:cpu
ADD . /tmp/pip
RUN pip install /tmp/pip/master.zip
RUN \
echo "h5py==2.7.0\n\
lifelines==0.9.4\n\
logger==1.4\n\
Optunity==1.1.1\n\
tensorboard-logger==0.0.3\n\
matplotlib==2.0.0" > /requirements.txt && \
pip install -U pip --proxy=xxx && \
pip install -U numpy --proxy=xxx && \
pip install -r /requirements.txt --proxy=xxx
COPY . /
CMD [ "python", "-u", "/hyperparam_search.py", \
"/shared/logs", \
"whas", \
"/box_constraints.0.json", \
"100", \
"--update_fn", "adam", \
"--num_epochs", "500", \
"--num_fold", "3" ]
Can anyone tell me why I get the error?
Hello!
I have some question about the data processing. How to deal with the right censoring data which e=0? Just drop them? In experiment,we can only use the data which e=1?
Looking forward to your reply.
Hi there,
I tested run the code but got a 0.48 c-index with the following code. Any idea why it could be the case? When printing out the log plot, the maximum likelihood is flat. Is there something going on with the loss function?
def generate_data(treatment_group = False):
np.random.seed(123)
sd = deepsurv.datasets.SimulatedData(5, num_features = 9,
treatment_group = treatment_group)
train_data = sd.generate_data(950)
valid_data = sd.generate_data(300)
test_data = sd.generate_data(300)
return train_data, valid_data, test_data
train,valid,test = generate_data(True)
hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes': [10,10]
}
network = DeepSurv(**hyperparams)
log = network.train(train,valid,n_epochs=500,validation_frequency = 1)
risk = network.predict_risk(test['x'])
The hyper parameters used for the experiments are already set. May I ask how are the current hyper parameters selected? Are there any tips for selecting hyper parameters to get better experimental results (using the datasets included under experiments folder)?
I am a newbie for survival analysis. I try to set up the docker and run the experiments (linear) and have got the following error
deepsurv_1 | ('Saving model parameters to output file', '/shared/results/modelslinear_model_selu_0.jsone7d7f147-c66a-4478-9375-171ba5c13379.h5')
deepsurv_1 | Traceback (most recent call last):
deepsurv_1 | File "/scripts/deepsurv_run.py", line 165, in <module>
deepsurv_1 | save_model(model, output_file)
deepsurv_1 | File "/scripts/deepsurv_run.py", line 100, in save_model
deepsurv_1 | model.save_weights(output_file)
deepsurv_1 | AttributeError: 'str' object has no attribute 'save_weights'
So what shall I do with this error?
I am still quite new to Python, and am trying to apply my data to deepsurv. Would you mind sharing the exact code you wrote between the two paragraphs below? I appreciate your help.
Training DeepSurv can be done in a few lines. First, all you need to do is prepare the datasets to have the following keys:
{
'x': (n,d) observations (dtype = float32),
't': (n) event times (dtype = float32),
'e': (n) event indicators (dtype = int32)
}
Then prepare a dictionary of hyper-parameters. And it takes only two lines to train a network:
network = deepsurv.DeepSurv(**hyperparams)
log = network.train(train_data, valid_data, n_epochs=500)
I don't get my own data,so I make a data,like: dataset = {'x':np.array([[0.1,0.3],[0.4,0.5],[0.3,0.2],[0.8,0.7],[0.5,0.2]],dtype = np.float32),'e':np.array([0,1,1,0,1],dtype = np.int32),'t':np.array([0.2,0.3,0.4,0.8,0.9],dtype = np.float32)}
can you give me a file about the data??
my train code:
import deepsurv
import pandas as pd
import numpy as np
#dataset = pd.read_csv('/home/mw/mw/work/DeepSurv-master/test.csv')
dataset = {'x':np.array([[0.1,0.3],[0.4,0.5],[0.3,0.2],[0.8,0.7],[0.5,0.2]],dtype = np.float32),'e':np.array([0,1,1,0,1],dtype = np.int32),'t':np.array([0.2,0.3,0.4,0.8,0.9],dtype = np.float32)}
train_data = dataset
valid_data = dataset
network = deepsurv.DeepSurv(n_in =2,learning_rate = 1e-5)
log = network.train(train_data, valid_data, n_epochs=500)
Traceback (most recent call last):
File "test.py", line 14, in
log = network.train(train_data, valid_data, n_epochs=500,logger = None)
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 397, in train
update_fn = update_fn, **kwargs
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 242, in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 208, in _get_loss_updates
if self.restored_update_params:
AttributeError: DeepSurv instance has no attribute 'restored_update_params'
here
Traceback (most recent call last):
File "test.py", line 14, in
log = network.train(train_data, valid_data, n_epochs=500,logger = None)
File "/home/mw/mw/work/DeepSurv-master/deepsurv/deep_surv.py", line 410, in train
logger.logValue('loss', loss, epoch)
AttributeError: 'NoneType' object has no attribute 'logValue'
I don't know whether the cause of my data or other question,Please give me some help ! Thank you very much first !
Would love to see your examples. and could not find py.test - was it part of the package?
your base module for windows does not have it
All the best!
I am trying to run the gaussian experiments.
Before you fix issue #29, deepsurv works just fine and the c-index is similar to the results presented in your paper. However, after updating to the latest version, I run the experiment again for multiple times. I've got a c-index around 0.5 for simulated non-linear dataset.
The results are as follows:
deepsurv_1 | Test metrics: {'mse': 0.35340887, 'mse_bootstrap': {'confidence_interval': (0.3502257244549425, 0.35606869292634413), 'mean': 0.3531472}, 'c_index_bootstrap': {'confidence_interval': (0.5028328594569582, 0.507284618344698), 'mean': 0.5050587389008281}, 'c_index': 0.5044739296688565}
A complete output for the experiment is as follows:
Attaching to experiments_cph_1, experiments_rsf_1, experiments_deepsurv_1
cph_1 | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
cph_1 | from ._conv import register_converters as _register_converters
cph_1 | /usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
cph_1 | "downsample module has been moved to the theano.tensor.signal.pool module.")
cph_1 | ('Arguments:', Namespace(dataset='/shared/data/gaussian_survival_data.h5', experiment='gaussian', plot_error=False, results_dir='/shared/results/', treatment_idx=None))
cph_1 | Loading datasets: /shared/data/gaussian_survival_data.h5
cph_1 | Training CPH Model
cph_1 | /usr/local/lib/python2.7/dist-packages/pandas/core/computation/check.py:17: UserWarning: The installed version of numexpr 2.2.2 is not supported in pandas and will be not be used
cph_1 | The minimum supported version is 2.4.6
cph_1 |
cph_1 | ver=ver, min_ver=_MIN_NUMEXPR_VERSION), UserWarning)
deepsurv_1 | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
deepsurv_1 | from ._conv import register_converters as _register_converters
rsf_1 | /usr/local/lib/python2.7/dist-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
rsf_1 | from ._conv import register_converters as _register_converters
cph_1 | n=4000, number of events=3644
cph_1 |
cph_1 | coef exp(coef) se(coef) z p lower 0.95 upper 0.95
cph_1 | 0 -1.253e-02 9.875e-01 1.493e-02 -8.395e-01 4.012e-01 -4.180e-02 1.673e-02
cph_1 | 1 1.048e-02 1.011e+00 1.495e-02 7.011e-01 4.832e-01 -1.883e-02 3.979e-02
cph_1 | 2 -1.368e-02 9.864e-01 1.666e-02 -8.213e-01 4.115e-01 -4.634e-02 1.898e-02
cph_1 | 3 1.102e-02 1.011e+00 1.655e-02 6.657e-01 5.056e-01 -2.142e-02 4.345e-02
cph_1 | 4 1.062e-02 1.011e+00 1.658e-02 6.409e-01 5.216e-01 -2.187e-02 4.312e-02
cph_1 | 5 1.864e-02 1.019e+00 1.669e-02 1.117e+00 2.641e-01 -1.409e-02 5.137e-02
cph_1 | 6 -1.871e-02 9.815e-01 1.659e-02 -1.128e+00 2.594e-01 -5.124e-02 1.382e-02
cph_1 | 7 -4.042e-02 9.604e-01 1.650e-02 -2.449e+00 1.432e-02 -7.278e-02 -8.068e-03 *
cph_1 | 8 1.483e-02 1.015e+00 1.647e-02 9.003e-01 3.680e-01 -1.746e-02 4.711e-02
cph_1 | 9 -9.909e-03 9.901e-01 1.658e-02 -5.976e-01 5.501e-01 -4.242e-02 2.260e-02
cph_1 | ---
cph_1 | Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
cph_1 |
rsf_1 | /usr/local/lib/python2.7/dist-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
rsf_1 | "downsample module has been moved to the theano.tensor.signal.pool module.")
cph_1 | Concordance = 0.518
cph_1 | Train Likelihood: -27435.781889126756
cph_1 | Valid metrics: {'mse': 1.3591188828245035, 'c_index': 0.48575868575868575}
deepsurv_1 | ('Arguments:', Namespace(dataset='/shared/data/gaussian_survival_data.h5', experiment='gaussian', model='/models/gaussian_model_0.json', num_epochs=1000, plot_error=False, results_dir='/shared/results/', treatment_idx=None, update_fn='sgd', weights=None))
deepsurv_1 | Loading datasets: /shared/data/gaussian_survival_data.h5
deepsurv_1 | Loading json model: {"L2_reg": 4.4249755859375, "dropout": 0.4013037109375, "learning_rate": 0.00031940075434571123, "lr_decay": 0.000317275390625, "momentum": 0.9363432617187499, "batch_norm": true, "standardize": false, "n_in": 10, "hidden_layers_sizes": [17, 17, 17]}
rsf_1 | ('Arguments:', Namespace(dataset='/shared/data/gaussian_survival_data.h5', num_trees=100, results_dir='/shared/results/', treatment_idx=None))
rsf_1 | Loading datasets: /shared/data/gaussian_survival_data.h5
rsf_1 | Training RSF Model
deepsurv_1 | WARNING (theano.tensor.blas): We did not found a dynamic library into the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library.
deepsurv_1 | 2018-01-15 01:05:16,986 - Training step 0/1000 | | - loss: 20.7979 - ci: 0.5023
cph_1 | Test metrics: {'mse': 1.3619913074136019, 'mse_bootstrap': {'confidence_interval': (1.3535952237897249, 1.3733029348865466), 'mean': 1.3634490793381357}, 'c_index_bootstrap': {'confidence_interval': (0.5039806821447451, 0.5086592477851751), 'mean': 0.5063199649649601}, 'c_index': 0.5061738614293155}
cph_1 | Saving Visualizations
experiments_cph_1 exited with code 0
deepsurv_1 | 2018-01-15 01:05:53,062 - Training step 100/1000 |** | - loss: 14.3097 - ci: 0.4924
rsf_1 | ('Train/Valid C-Index:', 0.6317852601385541)
deepsurv_1 | 2018-01-15 01:06:23,748 - Training step 200/1000 |***** | - loss: 11.2494 - ci: 0.4930
rsf_1 | Valid metrics: {'c_index': 0.6398349125621853}
deepsurv_1 | 2018-01-15 01:06:49,638 - Training step 300/1000 |******* | - loss: 9.5788 - ci: 0.4932
deepsurv_1 | 2018-01-15 01:07:15,965 - Training step 400/1000 |********** | - loss: 8.6446 - ci: 0.4932
deepsurv_1 | 2018-01-15 01:07:42,136 - Training step 500/1000 |************ | - loss: 8.1508 - ci: 0.4934
deepsurv_1 | 2018-01-15 01:08:07,600 - Training step 600/1000 |*************** | - loss: 7.8668 - ci: 0.4937
deepsurv_1 | 2018-01-15 01:08:33,813 - Training step 700/1000 |***************** | - loss: 7.7152 - ci: 0.4942
deepsurv_1 | 2018-01-15 01:08:59,912 - Training step 800/1000 |******************** | - loss: 7.6311 - ci: 0.4943
deepsurv_1 | 2018-01-15 01:09:26,828 - Training step 900/1000 |********************** | - loss: 7.5882 - ci: 0.4947
deepsurv_1 | 2018-01-15 01:09:53,080 - Finished Training with 1000 iterations in 276.83s
deepsurv_1 | Training metrics: {'mse': 0.3554571, 'c_index': 0.4953763858239832}
deepsurv_1 | Valid metrics: {'mse': 0.3554571, 'c_index': 0.4953763858239832}
deepsurv_1 | Test metrics: {'mse': 0.35340887, 'mse_bootstrap': {'confidence_interval': (0.3502257244549425, 0.35606869292634413), 'mean': 0.3531472}, 'c_index_bootstrap': {'confidence_interval': (0.5028328594569582, 0.507284618344698), 'mean': 0.5050587389008281}, 'c_index': 0.5044739296688565}
deepsurv_1 | Saving Visualizations
deepsurv_1 | ('Saving model parameters to output file', '/shared/results/modelsgaussian_model_0.jsoned3c1cc6-ec7b-41f8-b926-6b91799b2031.h5')
experiments_deepsurv_1 exited with code 0
rsf_1 | Test metrics: {'c_index_bootstrap': {'confidence_interval': (0.6425199977021452, 0.6463824127128943), 'mean': 0.6444512052075198}, 'c_index': 0.6446193727615209}
experiments_rsf_1 exited with code 0
Could you please help me figure out the issue?
Hi,
thank's for this great package!
I was wondering if there is a way to plot (according to the weights that the net acheived the survival function? (like in Lifelines package )
I generated nonlinear survival data using the SimulatedData class in your package, and only got 0.54 c-index for the test dataset. The negative log likelihood plot showed two separate straight lines for the training data and validation data. I think the loss function did not converge over epochs during my training process. I attached the code I used below. Could you please help me find the reason? Thank you very much!!
def generate_data(treatment_group = False):
np.random.seed(123)
sd = SimulatedData(5, num_features = 9,
treatment_group = treatment_group)
train_data = sd.generate_data(5000)
valid_data = sd.generate_data(2000)
test_data = sd.generate_data(2000)
return train_data, valid_data, test_data
train_data,valid_data,test_data=generate_data(True)
hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes':[10,10]
}
network = DeepSurv(**hyperparams)
log =network.train(train_data, valid_data, n_epochs=500)
network.get_concordance_index(**test_data)
deepsurv.plot_log(log)
I am new to Survival Analysis and I am trying to run the experiment using the dockerfiles but it shows the following error:
Step 5/5 : CMD [ "python", "-u", "/scripts/rsf_run.py", "/shared/data/linear_survival_data.h5", "--results_dir", "/shared/results/", "--num_trees", "100"]
---> Using cache
---> a8f230af9824
Successfully built a8f230af9824
Successfully tagged experiments_rsf:latest
Starting experiments_deepsurv_1 ... done
Starting experiments_cph_1 ... done
Starting experiments_rsf_1 ... done
Attaching to experiments_deepsurv_1, experiments_cph_1, experiments_rsf_1
deepsurv_1 | python: can't open file '/scripts/deepsurv_run.py': [Errno 2] No such file or directory
experiments_deepsurv_1 exited with code 2
rsf_1 | python: can't open file '/scripts/rsf_run.py': [Errno 2] No such file or directory
cph_1 | python: can't open file '/scripts/cph_run.py': [Errno 2] No such file or directory
experiments_rsf_1 exited with code 2
experiments_cph_1 exited with code 2
This happens with all the experiments. I downloaded the zip and my directory is \DeepSurv-master\DeepSurv-master\experiments and the scripts deepsurv_run.py, rsf_run.py and cph_run.py are in the script folder.
Would really appreciate any help. Thanks
I following readme.txt file,when i run py.test ,i meet some problem.How deal with it?There, I copy these problems.
ccf@ccf-Lenovo-Product:~/CCF/DeepSurv-master$ py.test
============================= test session starts ==============================
platform linux2 -- Python 2.7.6, pytest-3.1.2, py-1.4.34, pluggy-0.4.0
rootdir: /home/ccf/CCF/DeepSurv-master, inifile:
collected 8 items
tests/test_deepsurv.py .F.FEEEE
==================================== ERRORS ====================================
________________ ERROR at setup of TestDeepSurvTrain.test_train ________________
self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>
@classmethod
def setup_class(self):
self.train, self.valid, self.test = generate_data(treatment_group=True)
hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes': [10]
}
network = DeepSurv(**hyperparams)
log = network.train(self.train, self.valid,
n_epochs=10,validation_frequency=1)
tests/test_deepsurv.py:63:
deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])
def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
"""
Returns Theano expressions for the network's loss function and parameter
updates.
Parameters:
L1_reg: float for L1 weight regularization coefficient.
L2_reg: float for L2 weight regularization coefficient.
max_norm: If not None, constraints the norm of gradients to be less
than max_norm.
deterministic: True or False. Determines if the output of the network
is calculated determinsitically.
update_fn: lasagne update function.
Default: Stochastic Gradient Descent with Nesterov momentum
**kwargs: additional parameters to provide to update_fn.
For example: momentum
Returns:
loss: Theano expression for a penalized negative log likelihood.
updates: Theano expression to update the parameters using update_fn.
"""
loss = (
self._negative_log_likelihood(self.E, deterministic)
+ regularize_layer_params(self.network,l1) * L1_reg
+ regularize_layer_params(self.network, l2) * L2_reg
)
if max_norm:
grads = T.grad(loss,self.params)
scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
updates = update_fn(
grads, self.params, **kwargs
)
return loss, updates
updates = update_fn(
loss, self.params, **kwargs
)
# If the model was loaded from file, reload params
if self.restored_update_params:
E AttributeError: DeepSurv instance has no attribute 'restored_update_params'
deepsurv/deep_surv.py:209: AttributeError
---------------------------- Captured stdout setup -----------------------------
[ 1. 0. 1. ..., 1. 0. 0.]
[ 1. 1. 1. ..., 1. 1. 1.]
[ 0. 1. 1. ..., 1. 0. 1.]
________ ERROR at setup of TestDeepSurvTrain.test_network_predict_risk _________
self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>
@classmethod
def setup_class(self):
self.train, self.valid, self.test = generate_data(treatment_group=True)
hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes': [10]
}
network = DeepSurv(**hyperparams)
log = network.train(self.train, self.valid,
n_epochs=10,validation_frequency=1)
tests/test_deepsurv.py:63:
deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])
def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
"""
Returns Theano expressions for the network's loss function and parameter
updates.
Parameters:
L1_reg: float for L1 weight regularization coefficient.
L2_reg: float for L2 weight regularization coefficient.
max_norm: If not None, constraints the norm of gradients to be less
than max_norm.
deterministic: True or False. Determines if the output of the network
is calculated determinsitically.
update_fn: lasagne update function.
Default: Stochastic Gradient Descent with Nesterov momentum
**kwargs: additional parameters to provide to update_fn.
For example: momentum
Returns:
loss: Theano expression for a penalized negative log likelihood.
updates: Theano expression to update the parameters using update_fn.
"""
loss = (
self._negative_log_likelihood(self.E, deterministic)
+ regularize_layer_params(self.network,l1) * L1_reg
+ regularize_layer_params(self.network, l2) * L2_reg
)
if max_norm:
grads = T.grad(loss,self.params)
scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
updates = update_fn(
grads, self.params, **kwargs
)
return loss, updates
updates = update_fn(
loss, self.params, **kwargs
)
# If the model was loaded from file, reload params
if self.restored_update_params:
E AttributeError: DeepSurv instance has no attribute 'restored_update_params'
deepsurv/deep_surv.py:209: AttributeError
________ ERROR at setup of TestDeepSurvTrain.test_get_concordance_index ________
self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>
@classmethod
def setup_class(self):
self.train, self.valid, self.test = generate_data(treatment_group=True)
hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes': [10]
}
network = DeepSurv(**hyperparams)
log = network.train(self.train, self.valid,
n_epochs=10,validation_frequency=1)
tests/test_deepsurv.py:63:
deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])
def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
"""
Returns Theano expressions for the network's loss function and parameter
updates.
Parameters:
L1_reg: float for L1 weight regularization coefficient.
L2_reg: float for L2 weight regularization coefficient.
max_norm: If not None, constraints the norm of gradients to be less
than max_norm.
deterministic: True or False. Determines if the output of the network
is calculated determinsitically.
update_fn: lasagne update function.
Default: Stochastic Gradient Descent with Nesterov momentum
**kwargs: additional parameters to provide to update_fn.
For example: momentum
Returns:
loss: Theano expression for a penalized negative log likelihood.
updates: Theano expression to update the parameters using update_fn.
"""
loss = (
self._negative_log_likelihood(self.E, deterministic)
+ regularize_layer_params(self.network,l1) * L1_reg
+ regularize_layer_params(self.network, l2) * L2_reg
)
if max_norm:
grads = T.grad(loss,self.params)
scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
updates = update_fn(
grads, self.params, **kwargs
)
return loss, updates
updates = update_fn(
loss, self.params, **kwargs
)
# If the model was loaded from file, reload params
if self.restored_update_params:
E AttributeError: DeepSurv instance has no attribute 'restored_update_params'
deepsurv/deep_surv.py:209: AttributeError
_________ ERROR at setup of TestDeepSurvTrain.test_recommend_treatment _________
self = <class test_deepsurv.TestDeepSurvTrain at 0x7f9bffc83120>
@classmethod
def setup_class(self):
self.train, self.valid, self.test = generate_data(treatment_group=True)
hyperparams = {
'n_in': 10,
'learning_rate': 1e-5,
'hidden_layers_sizes': [10]
}
network = DeepSurv(**hyperparams)
log = network.train(self.train, self.valid,
n_epochs=10,validation_frequency=1)
tests/test_deepsurv.py:63:
deepsurv/deep_surv.py:397: in train
update_fn = update_fn, **kwargs
deepsurv/deep_surv.py:243: in _get_train_valid_fn
learning_rate=learning_rate, **kwargs
self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3d5b90>, L1_reg = 0.0
L2_reg = 0.0, update_fn = <function nesterov_momentum at 0x7f9c04ebfaa0>
max_norm = None, deterministic = False
kwargs = {'learning_rate': <TensorType(float32, scalar)>, 'momentum': array(0.0, dtype=float32)}
loss = Elemwise{add,no_inplace}.0
updates = OrderedDict([(W, Elemwise{add,no_inplace}.0), (b, Elemwise{add,no_inplace}.0),...b,no_inplace}.0), (<TensorType(float64, vector)>, Elemwise{sub,no_inplace}.0)])
def _get_loss_updates(self,
L1_reg = 0.0, L2_reg = 0.001,
update_fn = lasagne.updates.nesterov_momentum,
max_norm = None, deterministic = False,
**kwargs):
"""
Returns Theano expressions for the network's loss function and parameter
updates.
Parameters:
L1_reg: float for L1 weight regularization coefficient.
L2_reg: float for L2 weight regularization coefficient.
max_norm: If not None, constraints the norm of gradients to be less
than max_norm.
deterministic: True or False. Determines if the output of the network
is calculated determinsitically.
update_fn: lasagne update function.
Default: Stochastic Gradient Descent with Nesterov momentum
**kwargs: additional parameters to provide to update_fn.
For example: momentum
Returns:
loss: Theano expression for a penalized negative log likelihood.
updates: Theano expression to update the parameters using update_fn.
"""
loss = (
self._negative_log_likelihood(self.E, deterministic)
+ regularize_layer_params(self.network,l1) * L1_reg
+ regularize_layer_params(self.network, l2) * L2_reg
)
if max_norm:
grads = T.grad(loss,self.params)
scaled_grads = lasagne.updates.total_norm_constraint(grads, max_norm)
updates = update_fn(
grads, self.params, **kwargs
)
return loss, updates
updates = update_fn(
loss, self.params, **kwargs
)
# If the model was loaded from file, reload params
if self.restored_update_params:
E AttributeError: DeepSurv instance has no attribute 'restored_update_params'
deepsurv/deep_surv.py:209: AttributeError
=================================== FAILURES ===================================
_____________ TestDeepSurvInit.test_deepsurv_initialize_batch_norm _____________
self = <test_deepsurv.TestDeepSurvInit instance at 0x7f9bfa41f488>
def test_deepsurv_initialize_batch_norm(self):
network = DeepSurv(batch_norm = True, **self.hyperparams)
tests/test_deepsurv.py:39:
self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa41fc68>, n_in = 10
learning_rate = 1e-05, hidden_layers_sizes = [10, 10], lr_decay = 0.0
momentum = 0.9, L2_reg = 0.0, L1_reg = 0.0, activation = 'rectify'
dropout = None, batch_norm = True, standardize = False
def __init__(self, n_in,
learning_rate, hidden_layers_sizes = None,
lr_decay = 0.0, momentum = 0.9,
L2_reg = 0.0, L1_reg = 0.0,
activation = "rectify",
dropout = None,
batch_norm = False,
standardize = False,
):
"""
This class implements and trains a DeepSurv model.
Parameters:
n_in: number of input nodes.
learning_rate: learning rate for training.
lr_decay: coefficient for Power learning rate decay.
L2_reg: coefficient for L2 weight decay regularization. Used to help
prevent the model from overfitting.
L1_reg: coefficient for L1 weight decay regularization
momentum: coefficient for momentum. Can be 0 or None to disable.
hidden_layer_sizes: a list of integers to determine the size of
each hidden layer.
activation: a lasagne activation class.
Default: lasagne.nonlinearities.rectify
batch_norm: True or False. Include batch normalization layers.
dropout: if not None or 0, the percentage of dropout to include
after each hidden layer. Default: None
standardize: True or False. Include standardization layer after
input layer.
"""
self.X = T.fmatrix('x') # patients covariates
self.E = T.ivector('e') # the observations vector
# Default Standardization Values: mean = 0, std = 1
self.offset = theano.shared(numpy.zeros(shape = n_in, dtype=numpy.float32))
self.scale = theano.shared(numpy.ones(shape = n_in, dtype=numpy.float32))
network = lasagne.layers.InputLayer(shape=(None,n_in),
input_var = self.X)
if standardize:
network = lasagne.layers.standardize(network,self.offset,
self.scale,
shared_axes = 0)
self.standardize = standardize
if activation == 'rectify':
activation_fn = lasagne.nonlinearities.rectify
else:
raise IllegalArgumentException("Unknown activation function: %s" % activation)
# Construct Neural Network
for n_layer in (hidden_layers_sizes or []):
if activation_fn == lasagne.nonlinearities.rectify:
W_init = lasagne.init.GlorotUniform()
else:
# TODO: implement other initializations
W_init = lasagne.init.GlorotUniform()
network = lasagne.layers.DenseLayer(
network, num_units = n_layer,
nonlinearity = activation_fn,
W = W_init
)
if batch_norm:
network = lasagne.layers.po(network)
E AttributeError: 'module' object has no attribute 'po'
deepsurv/deep_surv.py:86: AttributeError
_________ TestDeepSurvInit.test_deepsurv_initialize_standardize_layer __________
self = <test_deepsurv.TestDeepSurvInit instance at 0x7f9bfa37c9e0>
def test_deepsurv_initialize_standardize_layer(self):
network = DeepSurv(standardize = True, **self.hyperparams)
tests/test_deepsurv.py:47:
self = <deepsurv.deep_surv.DeepSurv instance at 0x7f9bfa3f7b00>, n_in = 10
learning_rate = 1e-05, hidden_layers_sizes = [10, 10], lr_decay = 0.0
momentum = 0.9, L2_reg = 0.0, L1_reg = 0.0, activation = 'rectify'
dropout = None, batch_norm = False, standardize = True
def __init__(self, n_in,
learning_rate, hidden_layers_sizes = None,
lr_decay = 0.0, momentum = 0.9,
L2_reg = 0.0, L1_reg = 0.0,
activation = "rectify",
dropout = None,
batch_norm = False,
standardize = False,
):
"""
This class implements and trains a DeepSurv model.
Parameters:
n_in: number of input nodes.
learning_rate: learning rate for training.
lr_decay: coefficient for Power learning rate decay.
L2_reg: coefficient for L2 weight decay regularization. Used to help
prevent the model from overfitting.
L1_reg: coefficient for L1 weight decay regularization
momentum: coefficient for momentum. Can be 0 or None to disable.
hidden_layer_sizes: a list of integers to determine the size of
each hidden layer.
activation: a lasagne activation class.
Default: lasagne.nonlinearities.rectify
batch_norm: True or False. Include batch normalization layers.
dropout: if not None or 0, the percentage of dropout to include
after each hidden layer. Default: None
standardize: True or False. Include standardization layer after
input layer.
"""
self.X = T.fmatrix('x') # patients covariates
self.E = T.ivector('e') # the observations vector
# Default Standardization Values: mean = 0, std = 1
self.offset = theano.shared(numpy.zeros(shape = n_in, dtype=numpy.float32))
self.scale = theano.shared(numpy.ones(shape = n_in, dtype=numpy.float32))
network = lasagne.layers.InputLayer(shape=(None,n_in),
input_var = self.X)
if standardize:
network = lasagne.layers.standardize(network,self.offset,
self.scale,
shared_axes = 0)
E AttributeError: 'module' object has no attribute 'standardize'
deepsurv/deep_surv.py:60: AttributeError
================= 2 failed, 2 passed, 4 error in 5.42 seconds ==================
pip install .
is success, but py.test has a ModuleNotFoundError. I use the method mentioned in #5 but still throws the error
And when I run the linear experiment by EXPERIMENT=linear docker-compose up --build, it throws ImportError,
my python-dateutil version is 2.7.2, and I tried both python2 & python3, it throws the same error
I'm looking at both lifelines and your method for getting concordance index. In DeepSurv, you have:
return concordance_index(t,
partial_hazards,
e)
where t is time, e is event indicator, and partial_hazards is -self.partial_hazard.
In lifelines utils/init.py:
return _concordance_index(event_times,
predicted_event_times,
event_observed)
Where it differs in using predicted_event_times in place of partial_hazards.
Do you mind explaining the use of partial_hazards in place of predicted_event_times to calculate CI?
Also, is it incorrect to have risk values that are negative? Many of my risk values end up being negative when I do:
self.network.predict_risk(x)
Thanks!
Great module!
Do you have code that runs through the same simulation that you described in your paper?
Hey everyone!
Thanks for the great package first of all! Even with limited ML experience, the DeepSurv network is working just fine. However, I am unable to run the hyperparameter tuning job.
I implement the hyperparam_search on Windows 10 with a CUDA-compatible NVIDIA graphics card on the CPU. I get the following error message (nvcc is part of CUDA, which I installed prior to the hyperparameter search):
hp_search_1 | ERROR (theano.sandbox.cuda): nvcc compiler not found on $PATH. Check your nvcc installation and try again.
I included the nvcc in the path variable (both environment + system variables), but still it is not working. Practically all of the help I find online is concerned with Linux applications, but it's not relevant to the environment I am using and I don't have enough experience to try Linux myself.
Thank you very much in advance for your help. My master thesis & I are eternally grateful for your support!
Tim
Any suggestions for parameters setting. The performance of default setting is very poor
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.