xingyanliu / came Goto Github PK
View Code? Open in Web Editor NEWCell-type Assignment and Module Extraction based on a heterogeneous graph neural network.
Home Page: https://XingyanLiu.github.io/CAME
License: MIT License
Cell-type Assignment and Module Extraction based on a heterogeneous graph neural network.
Home Page: https://XingyanLiu.github.io/CAME
License: MIT License
Hi, after battling through the process of installing and re-installing old versions of dependencies I am hitting up against a new error that hasn't been reported before. I'm using the test data. The epochs run successfully! But then I get this problem with plotting the second figure:
figure has been saved into:
_temp/('Baron_human', 'Baron_mouse')-(08-31 18.53.59)/figs/cluster_index.png
states loaded from: _temp/('Baron_human', 'Baron_mouse')-(08-31 18.53.59)/_models/weights_epoch231.pt
object saved into:
_temp/('Baron_human', 'Baron_mouse')-(08-31 18.53.59)/datapair_init.pickle
WARNING:root:An error occurred when plotting results: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Help would be much appreciated! Really keen to use this tool.
Hi, I am trying to ran pipeline with my own data. I got an error, which is below.
Here is the code I ran:
came_inputs, (adata1, adata2) = pipeline.preprocess_unaligned(
adatas,
key_class=key_class1,
use_scnets=use_scnets,
ntop_deg=ntop_deg,
ntop_deg_nodes=ntop_deg_nodes,
node_source=node_source,
)
This is the error I got:
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/yali/softs/Anaconda3/lib/python3.9/site-packages/came-0.1.9-py3.9.egg/came/pipeline.py", line 938, in preprocess_unaligned adata1 = pp.quick_preprocess(adatas[0], **params_preproc) File "/home/yali/softs/Anaconda3/lib/python3.9/site-packages/came-0.1.9-py3.9.egg/came/utils/preprocess.py", line 1923, in quick_preprocess sc.pp.highly_variable_genes( File "/home/yali/softs/Anaconda3/lib/python3.9/site-packages/scanpy/preprocessing/_highly_variable_genes.py", line 431, in highly_variable_genes df = _highly_variable_genes_single_batch( File "/home/yali/softs/Anaconda3/lib/python3.9/site-packages/scanpy/preprocessing/_highly_variable_genes.py", line 215, in _highly_variable_genes_single_batch df['mean_bin'] = pd.cut(df['means'], bins=n_bins) File "/home/yali/softs/Anaconda3/lib/python3.9/site-packages/pandas/core/reshape/tile.py", line 287, in cut fac, bins = _bins_to_cuts( File "/home/yali/softs/Anaconda3/lib/python3.9/site-packages/pandas/core/reshape/tile.py", line 413, in _bins_to_cuts raise ValueError( ValueError: Bin edges must be unique: array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]). You can drop duplicate edges by setting the 'duplicates' kwarg
The h5ad files were converted from seuratobject by SeuratDisk.
Please let me know any advice on how to resolve this. Thank you!
Hi, when I run the example dataset, step [6] show : AttributeError: module 'dgl.function' has no attribute 'copy_src'.
Hope you can help me! Thanks a lot!
[leiden] Time used: 0.2771 s
already exists:
/home/ld/CAME/came/sample_data/_temp/('Baron_human', 'Baron_mouse')-(02-11 16.32.15)/figs
already exists:
/home/ld/CAME/came/sample_data/_temp/('Baron_human', 'Baron_mouse')-(02-11 16.32.15)
[] Setting dataset names:
0-->Baron_human
1-->Baron_mouse
[] Setting aligned features for observation nodes (self._features)
[*] Setting observation-by-variable adjacent matrices (self._ov_adjs
) for making merged graph adjacent matrix of observation and variable nodes
-------------------- Summary of the DGL-Heterograph --------------------
Graph(num_nodes={'cell': 4028, 'gene': 3446},
num_edges={('cell', 'express', 'gene'): 1677575, ('cell', 'self_loop_cell', 'cell'): 4028, ('cell', 'similar_to', 'cell'): 25760, ('gene', 'expressed_by', 'cell'): 1677575, ('gene', 'self_loop_gene', 'gene'): 3446},
metagraph=[('cell', 'gene', 'express'), ('cell', 'cell', 'self_loop_cell'), ('cell', 'cell', 'similar_to'), ('gene', 'cell', 'expressed_by'), ('gene', 'gene', 'self_loop_gene')])
self-loops for observation-nodes: True
self-loops for variable-nodes: True
AttributeError Traceback (most recent call last)
Cell In[165], line 10
1 came_inputs, (adata1, adata2) = pipeline.preprocess_aligned(
2 adatas,
3 key_class=key_class1,
(...)
7 df_varmap_1v1=df_varmap_1v1, # set as None if NOT cross species
8 )
---> 10 outputs = pipeline.main_for_aligned(
11 **came_inputs,
12 dataset_names=dsnames,
13 key_class1=key_class1,
14 key_class2=key_class2,
15 do_normalize=True,
16 n_epochs=n_epochs,
17 resdir=resdir,
18 n_pass=n_pass,
19 batch_size=batch_size,
20 plot_results=True,
21 )
22 dpair = outputs['dpair']
23 trainer = outputs['trainer']
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/pipeline.py:213, in main_for_aligned(adatas, vars_feat, vars_as_nodes, scnets, dataset_names, key_class1, key_class2, do_normalize, batch_keys, n_epochs, resdir, tag_data, params_model, params_lossfunc, n_pass, batch_size, pred_batch_size, plot_results, norm_target_sum, save_hidden_list, save_dpair)
207 trainer.train_minibatch(
208 n_epochs=n_epochs,
209 params_lossfunc=params_lossfunc,
210 batch_size=batch_size,
211 n_pass=n_pass, device=device)
212 else:
--> 213 trainer.train(n_epochs=n_epochs,
214 params_lossfunc=params_lossfunc,
215 n_pass=n_pass, device=device)
216 trainer.save_model_weights()
217 # ========================== record results ========================
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/train.py:295, in Trainer.train(self, n_epochs, use_class_weights, params_lossfunc, n_pass, eps, cat_class, device, info_stride, backup_stride, **other_inputs)
293 self.optimizer.zero_grad()
294 t0 = time.time()
--> 295 outputs = model(feat_dict, g, **other_inputs)
296 logits = outputs[cat_class]
297 # logits2 = model(feat_dict, g, **other_inputs)[cat_class]
298 # loss = ce_loss_with_rdrop(
299 # logits, logits2, labels=train_labels,
(...)
303 # **params_lossfunc
304 # )
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
1190 # If we don't have any hooks, we want to skip the rest of the logic in
1191 # this function, and just call forward.
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1193 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194 return forward_call(*input, **kwargs)
1195 # Do not call functions when jit is used
1196 full_backward_hooks, non_full_backward_hooks = [], []
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/model/cggc.py:205, in CGGCNet.forward(self, feat_dict, g, **other_inputs)
203 h_dict['gene'] = relu(h_dict0['gene'] + h_dict['gene'])
204 else:
--> 205 h_dict = self.embed_layer(g, feat_dict, )
206 h_dict = self.rgcn.forward(g, h_dict, **other_inputs).copy()
208 h_dict['cell'] = self.cell_classifier.forward(g, h_dict)['cell']
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
1190 # If we don't have any hooks, we want to skip the rest of the logic in
1191 # this function, and just call forward.
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1193 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194 return forward_call(*input, **kwargs)
1195 # Do not call functions when jit is used
1196 full_backward_hooks, non_full_backward_hooks = [], []
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/model/base_layers.py:571, in GeneralRGCLayer.forward(self, g, inputs, etypes, norm, bias, activate, static_wdict)
568 # inputs_src = inputs_dst = inputs
569 inputs = {ntype: self.dropout_feat(feat) for ntype, feat in inputs.items()}
--> 571 hs = self.conv(g, inputs, etypes, mod_kwargs=wdict)
573 def _apply(ntype, h):
575 if self.use_batchnorm and norm:
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
1190 # If we don't have any hooks, we want to skip the rest of the logic in
1191 # this function, and just call forward.
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1193 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194 return forward_call(*input, **kwargs)
1195 # Do not call functions when jit is used
1196 full_backward_hooks, non_full_backward_hooks = [], []
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/model/heteroframe.py:173, in HeteroGraphConv.forward(self, g, inputs, etypes, mod_args, mod_kwargs)
171 if stype not in inputs:
172 continue
--> 173 dstdata = self.mods[etype](
174 rel_graph,
175 inputs[stype],
176 *mod_args.get(etype, ()),
177 **mod_kwargs.get(etype, {}))
178 outputs[dtype].append(dstdata)
179 rsts = {}
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
1190 # If we don't have any hooks, we want to skip the rest of the logic in
1191 # this function, and just call forward.
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1193 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194 return forward_call(*input, **kwargs)
1195 # Do not call functions when jit is used
1196 full_backward_hooks, non_full_backward_hooks = [], []
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/model/base_layers.py:875, in GraphConvLayer.forward(self, g, feat, weight, static_weight)
873 weight = self.weight
874 if static_weight is None:
--> 875 message_func = fn.copy_src(src='h', out='m')
876 else:
877 g.edata['w_static'] = static_weight
AttributeError: module 'dgl.function' has no attribute 'copy_src'
我在使用自己的数据运行came时:
outputs = pipeline.main_for_unaligned(
**came_inputs,
df_varmap=df_varmap,
df_varmap_1v1=df_varmap_1v1,
dataset_names=dsnames,
key_class1=key_class1,
key_class2=key_class2,
do_normalize=True,
keep_non1v1_feats=keep_non1v1_feats,
n_epochs=n_epochs,
resdir=resdir,
n_pass=n_pass,
batch_size=batch_size,
plot_results=True,
)
出现了
already exists:
/work/home/luo_funong/shanhuiquan/came/test/res/figs
already exists:
/work/home/luo_funong/shanhuiquan/came/test/res
trans.shape= (570, 567)
[] Setting dataset names:
0-->liver_cattle
1-->liver_human
[] Setting aligned features for observation nodes (self._features)
[] Setting un-aligned features (self._ov_adjs
) for making links connecting observation and variable nodes
[] Setting adjacent matrix connecting variables from these 2 datasets (self._vv_adj
)
-------------------- Summary of the DGL-Heterograph --------------------
Graph(num_nodes={'cell': 8996, 'gene': 7024},
num_edges={('cell', 'express', 'gene'): 3419993, ('cell', 'self_loop_cell', 'cell'): 8996, ('cell', 'similar_to', 'cell'): 59332, ('gene', 'expressed_by', 'cell'): 3419993, ('gene', 'homolog_with', 'gene'): 12196},
metagraph=[('cell', 'gene', 'express'), ('cell', 'cell', 'self_loop_cell'), ('cell', 'cell', 'similar_to'), ('gene', 'cell', 'expressed_by'), ('gene', 'gene', 'homolog_with')])
second-order connection: False
self-loops for observation-nodes: True
self-loops for variable-nodes: True
DataPair with 8996 obs- and 7024 var-nodes
obs1 x var1 (liver_cattle): 3989 x 3196
obs2 x var2 (liver_human): 5007 x 3828
Dimensions of the obs-node-features: 570
/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/sklearn/utils/validation.py:727: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
warnings.warn(
/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/sklearn/utils/validation.py:727: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
warnings.warn(
Traceback (most recent call last):
File "", line 1, in
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/came/pipeline.py", line 436, in main_for_unaligned
ENV_VARs = prepare4train(dpair, key_class=keys, batch_keys=batch_keys,)
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/train.py", line 107, in prepare4train
labels, classes = dpair.get_obs_labels(
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/came/datapair/unaligned.py", line 385, in get_obs_labels
labels_cat = pd.Categorical(list(labels_12[0]) + list(labels_12[1]),
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/pandas/core/arrays/categorical.py", line 378, in init
dtype = CategoricalDtype._from_values_or_dtype(
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/pandas/core/dtypes/dtypes.py", line 299, in _from_values_or_dtype
dtype = CategoricalDtype(categories, ordered)
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/pandas/core/dtypes/dtypes.py", line 186, in init
self._finalize(categories, ordered, fastpath=False)
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/pandas/core/dtypes/dtypes.py", line 340, in _finalize
categories = self.validate_categories(categories, fastpath=fastpath)
File "/work/home/luo_funong/miniconda3/envs/env_came/lib/python3.8/site-packages/pandas/core/dtypes/dtypes.py", line 534, in validate_categories
raise ValueError("Categorical categories cannot be null")
ValueError: Categorical categories cannot be null
请问怎么解决
Hi developer,
I installed CAME with guide and try test pipeline like this
import came
if name == 'main':
came.test1(6, batch_size=2048)
came.test2(6, batch_size=None)
but get this message:
[leiden] Time used: 1.1408 s
a new directory made:
_temp('Baron_human', 'Baron_mouse')-(12-23 23.38.12)\figs
already exists:
_temp('Baron_human', 'Baron_mouse')-(12-23 23.38.12)
[] Setting dataset names:
0-->Baron_human
1-->Baron_mouse
[] Setting aligned features for observation nodes (self._features)
[*] Setting observation-by-variable adjacent matrices (self._ov_adjs
) for making merged graph adjacent matrix of observation and variable nodes
-------------------- Summary of the DGL-Heterograph --------------------
Graph(num_nodes={'cell': 4028, 'gene': 3446},
num_edges={('cell', 'express', 'gene'): 1677575, ('cell', 'self_loop_cell', 'cell'): 4028, ('cell', 'similar_to', 'cell'): 25760, ('gene', 'expressed_by', 'cell'): 1677575, ('gene', 'self_loop_gene', 'gene'): 3446},
metagraph=[('cell', 'gene', 'express'), ('cell', 'cell', 'self_loop_cell'), ('cell', 'cell', 'similar_to'), ('gene', 'cell', 'expressed_by'), ('gene', 'gene', 'self_loop_gene')])
self-loops for observation-nodes: True
self-loops for variable-nodes: True
AlignedDataPair with 4028 obs- and 3446 var-nodes
n_obs1 (Baron_human): 2142
n_obs2 (Baron_mouse): 1886
Dimensions of the obs-node-features: 701
main directory: _temp('Baron_human', 'Baron_mouse')-(12-23 23.38.12)
model directory: _temp('Baron_human', 'Baron_mouse')-(12-23 23.38.12)_models
============== start training (device='cpu') ===============
0%| | 0/2 [00:00<?, ?it/s]e:\Anaconda\envs\env_came\lib\site-packages\dgl\dataloading\dataloader.py:859: DGLWarning: Dataloader CPU affinity opt is not enabled, consider switching it on (see enable_cpu_affinity() or CPU best practices for DGL [https://docs.dgl.ai/tutorials/cpu/cpu_best_practises.html])
dgl_warning(f'Dataloader CPU affinity opt is not enabled, consider switching it on '
100%|██████████| 2/2 [00:00<00:00, 3.98it/s]
100%|██████████| 1/1 [00:00<00:00, 6.55it/s]
Epoch 0000 | Train Acc: 0.0042 | Test: 0.0154 (max=0.0154) | AMI=0.0225 | Time: 0.6749
0%| | 0/2 [00:00<?, ?it/s]e:\Anaconda\envs\env_came\lib\site-packages\dgl\dataloading\dataloader.py:859: DGLWarning: Dataloader CPU affinity opt is not enabled, consider switching it on (see enable_cpu_affinity() or CPU best practices for DGL [https://docs.dgl.ai/tutorials/cpu/cpu_best_practises.html])
dgl_warning(f'Dataloader CPU affinity opt is not enabled, consider switching it on '
100%|██████████| 2/2 [00:00<00:00, 3.99it/s]
100%|██████████| 1/1 [00:00<00:00, 6.31it/s]
0%| | 0/2 [00:00<?, ?it/s]e:\Anaconda\envs\env_came\lib\site-packages\dgl\dataloading\dataloader.py:859: DGLWarning: Dataloader CPU affinity opt is not enabled, consider switching it on (see enable_cpu_affinity() or CPU best practices for DGL [https://docs.dgl.ai/tutorials/cpu/cpu_best_practises.html])
dgl_warning(f'Dataloader CPU affinity opt is not enabled, consider switching it on '
100%|██████████| 2/2 [00:00<00:00, 3.96it/s]
100%|██████████| 1/1 [00:00<00:00, 6.31it/s]
0%| | 0/2 [00:00<?, ?it/s]e:\Anaconda\envs\env_came\lib\site-packages\dgl\dataloading\dataloader.py:859: DGLWarning: Dataloader CPU affinity opt is not enabled, consider switching it on (see enable_cpu_affinity() or CPU best practices for DGL [https://docs.dgl.ai/tutorials/cpu/cpu_best_practises.html])
dgl_warning(f'Dataloader CPU affinity opt is not enabled, consider switching it on '
100%|██████████| 2/2 [00:00<00:00, 4.03it/s]
100%|██████████| 1/1 [00:00<00:00, 6.19it/s]
Epoch 0003 | Train Acc: 0.1349 | Test: 0.4894 (max=0.4894) | AMI=0.1204 | Time: 0.6767
0%| | 0/2 [00:00<?, ?it/s]e:\Anaconda\envs\env_came\lib\site-packages\dgl\dataloading\dataloader.py:859: DGLWarning: Dataloader CPU affinity opt is not enabled, consider switching it on (see enable_cpu_affinity() or CPU best practices for DGL [https://docs.dgl.ai/tutorials/cpu/cpu_best_practises.html])
dgl_warning(f'Dataloader CPU affinity opt is not enabled, consider switching it on '
100%|██████████| 2/2 [00:00<00:00, 4.00it/s]
100%|██████████| 1/1 [00:00<00:00, 6.47it/s]
0%| | 0/2 [00:00<?, ?it/s]e:\Anaconda\envs\env_came\lib\site-packages\dgl\dataloading\dataloader.py:859: DGLWarning: Dataloader CPU affinity opt is not enabled, consider switching it on (see enable_cpu_affinity() or CPU best practices for DGL [https://docs.dgl.ai/tutorials/cpu/cpu_best_practises.html])
dgl_warning(f'Dataloader CPU affinity opt is not enabled, consider switching it on '
100%|██████████| 2/2 [00:00<00:00, 3.92it/s]
100%|██████████| 1/1 [00:00<00:00, 5.75it/s]
Epoch 0005 | Train Acc: 0.3366 | Test: 0.5175 (max=0.5175) | AMI=0.1017 | Time: 0.6805
figure has been saved into:
_temp('Baron_human', 'Baron_mouse')-(12-23 23.38.12)\figs\cluster_index.png
states loaded from: _temp('Baron_human', 'Baron_mouse')-(12-23 23.38.12)_models\weights_epoch5.pt
0%| | 0/4 [00:00<?, ?it/s]e:\Anaconda\envs\env_came\lib\site-packages\dgl\dataloading\dataloader.py:859: DGLWarning: Dataloader CPU affinity opt is not enabled, consider switching it on (see enable_cpu_affinity() or CPU best practices for DGL [https://docs.dgl.ai/tutorials/cpu/cpu_best_practises.html])
dgl_warning(f'Dataloader CPU affinity opt is not enabled, consider switching it on '
100%|██████████| 4/4 [00:00<00:00, 7.45it/s]
WARNING:root:An error occurred when plotting results: Cannot cast ufunc 'divide' output from dtype('float64') to dtype('int32') with casting rule 'same_kind'
object saved into:
_temp('Baron_human', 'Baron_mouse')-(12-23 23.38.12)\datapair_init.pickle
Test passed for ALIGNED!
[leiden] Time used: 0.2084 s
a new directory made:
_temp('Baron_human', 'Baron_mouse')-(12-23 23.38.29)\figs
already exists:
_temp('Baron_human', 'Baron_mouse')-(12-23 23.38.29)
trans.shape= (533, 532)
[] Setting dataset names:
0-->Baron_human
1-->Baron_mouse
[] Setting aligned features for observation nodes (self._features)
[] Setting un-aligned features (self._ov_adjs
) for making links connecting observation and variable nodes
[] Setting adjacent matrix connecting variables from these 2 datasets (self._vv_adj
)
-------------------- Summary of the DGL-Heterograph --------------------
Graph(num_nodes={'cell': 4028, 'gene': 6449},
num_edges={('cell', 'express', 'gene'): 1434761, ('cell', 'self_loop_cell', 'cell'): 4028, ('cell', 'similar_to', 'cell'): 26222, ('gene', 'expressed_by', 'cell'): 1434761, ('gene', 'homolog_with', 'gene'): 12259},
metagraph=[('cell', 'gene', 'express'), ('cell', 'cell', 'self_loop_cell'), ('cell', 'cell', 'similar_to'), ('gene', 'cell', 'expressed_by'), ('gene', 'gene', 'homolog_with')])
second-order connection: False
self-loops for observation-nodes: True
self-loops for variable-nodes: True
TypeError Traceback (most recent call last)
Cell In[3], line 5
3 if name == 'main':
4 came.test1(6, batch_size=2048)
----> 5 came.test2(6, batch_size=None)
File e:\Anaconda\envs\env_came\lib\site-packages\came\pipeline.py:1112, in test2(n_epochs, batch_size, reverse)
1103 resdir = Path('_temp') / f'{dsnames}-{time_tag}'
1105 came_inputs, (adata1, adata2) = preprocess_unaligned(
1106 adatas,
1107 key_class=key_class,
1108 node_source='deg,hvg',
1109 ntop_deg=50,
1110 )
-> 1112 _ = main_for_unaligned(
1113 **came_inputs,
1114 df_varmap=df_varmap,
1115 df_varmap_1v1=df_varmap_1v1,
1116 dataset_names=dsnames,
1117 key_class1=key_class,
1118 key_class2=key_class,
1119 do_normalize=True,
1120 keep_non1v1_feats=True,
1121 non1v1_trans_to=1,
...
743 xp, is_array_api = get_namespace(array)
745 # store reference to original array to check if copy is needed when
746 # function returns
TypeError: np.matrix is not supported. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
can you help me ? Thanks.
Just running the tutorial and when I got the 2.4 Abstracted graph step, I got the following error below:
The Code as in the tutorial:
norm_ov = ['max', 'zs', None][1]
cut_ov = 0.
groupby_var = 'module'
obs_labels1, obs_labels2 = adt.obs['celltype'][dpair.obs_ids1], \
adt.obs['celltype'][dpair.obs_ids2]
var_labels1, var_labels2 = gadt1.obs[groupby_var], gadt2.obs[groupby_var]
sp1, sp2 = 'human', 'mouse'
g = came.make_abstracted_graph(
obs_labels1, obs_labels2,
var_labels1, var_labels2,
avg_expr1, avg_expr2,
df_var_links,
tags_obs=(f'{sp1} ', f'{sp2} '),
tags_var=(f'{sp1} module ', f'{sp2} module '),
cut_ov=cut_ov,
norm_mtd_ov=norm_ov,
)
The Error:
> ---------------------------------------------------------------------------
> IndexError Traceback (most recent call last)
> Cell In[30], line 10
> 7 var_labels1, var_labels2 = gadt1.obs[groupby_var], gadt2.obs[groupby_var]
> 9 sp1, sp2 = 'human', 'mouse'
> ---> 10 g = came.make_abstracted_graph(
> 11 obs_labels1, obs_labels2,
> 12 var_labels1, var_labels2,
> 13 avg_expr1, avg_expr2,
> 14 df_var_links,
> 15 tags_obs=(f'{sp1} ', f'{sp2} '),
> 16 tags_var=(f'{sp1} module ', f'{sp2} module '),
> 17 cut_ov=cut_ov,
> 18 norm_mtd_ov=norm_ov,
> 19 )
>
> File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/analyze.py:1141, in make_abstracted_graph(obs_labels1, obs_labels2, var_labels1, var_labels2, avg_expr1, avg_expr2, df_var_links, tags_obs, tags_var, key_weight, cut_ov, norm_mtd_ov, ov_norm_first, global_adjust_ov, global_adjust_vv, vargroup_filtered, **kwds)
> 1131 var_labels1, var_labels2, avg_expr1, avg_expr2, df_var_links = \
> 1132 _filter_for_abstract(
> 1133 var_labels1, var_labels2, avg_expr1, avg_expr2, df_var_links,
> 1134 name=vargroup_filtered)
> 1135 # obs_group_order1 = _unique_cats(obs_labels1, obs_group_order1)
> 1136 # obs_group_order2 = _unique_cats(obs_labels2, obs_group_order2)
> 1137 # var_group_order1 = _unique_cats(var_labels1, var_group_order1)
> 1138 # var_group_order2 = _unique_cats(var_labels2, var_group_order2)
> 1139 # print('--->', var_group_order1)
> 1140 # obs-var edge abstraction #
> -> 1141 edges_ov1, avg_vo1 = abstract_ov_edges(
> 1142 avg_expr1, var_labels1,
> 1143 norm_method=norm_mtd_ov,
> 1144 cut=cut_ov,
> 1145 tag_var=tag_var1, tag_obs=tag_obs1,
> 1146 norm_first=ov_norm_first,
> 1147 global_adjust=global_adjust_ov,
> 1148 return_full_adj=True)
> 1149 edges_ov2, avg_vo2 = abstract_ov_edges(
> 1150 avg_expr2, var_labels2,
> 1151 norm_method=norm_mtd_ov,
> (...)
> 1155 global_adjust=global_adjust_ov,
> 1156 return_full_adj=True)
> 1157 # print('---> avg_vo1\n', avg_vo1)
> 1158 # var-weights abstraction #
>
> File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/analyze.py:1398, in abstract_ov_edges(avg_expr, var_labels, norm_method, norm_axis, tag_var, tag_obs, cut, norm_first, global_adjust, return_full_adj)
> 1396 # averaged by varible-groups
> 1397 groupby_var = '__temp_labels__'
> -> 1398 df[groupby_var] = var_labels
> 1399 df_avg = df.groupby(groupby_var).mean()
> 1400 df_avg.dropna(inplace=True)
>
> IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
Anyone else got the same/similar issue?
我在用sample data运行时
y_true = obs['celltype'][obs_ids2].values
y_pred = obs['predicted'][obs_ids2].values
UFuncTypeError Traceback (most recent call last)
Cell In[23], line 5
2 y_true = obs['celltype'][obs_ids2].values
3 y_pred = obs['predicted'][obs_ids2].values
----> 5 ax, contmat = pl.plot_contingency_mat(
6 y_true, y_pred, norm_axis=1,
7 order_rows=False, order_cols=False,
8 )
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/plot.py:253, in plot_contingency_mat(y_true, y_pred, norm_axis, arrange, order_rows, order_cols, ax, figsize, fp, **kwds)
251 """function for plotting the contingency matrix"""
252 from .analyze import wrapper_contingency_mat
--> 253 contmat = wrapper_contingency_mat(
254 y_true, y_pred, normalize_axis=norm_axis,
255 order_rows=order_rows, order_cols=order_cols, )
256 if arrange:
257 # align column- and row- names as possible
258 from .analyze import arrange_contingency_mat
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/analyze.py:1677, in wrapper_contingency_mat(y_true, y_pred, order_rows, order_cols, normalize_axis, as_df, eps, assparse)
1675 mat = mat + eps
1676 if normalize_axis is not None: # 0 for columns
-> 1677 mat = pp.normalize_norms(mat, axis=normalize_axis)
1679 if as_df:
1680 mat = pd.DataFrame(mat, index=classes, columns=clusters)
File ~/miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/preprocess.py:1619, in normalize_norms(X, scale_factor, axis, by)
1603 """ wrapper of normalize_colsum
and normalize_rowsum
1604
1605 Parameters
(...)
1616
1617 """
1618 foo = normalize_col if axis == 0 else normalize_row
-> 1619 return foo(X, scale_factor=scale_factor, by=by)
File /miniconda3/envs/env_came/lib/python3.8/site-packages/came/utils/preprocess.py:1585, in normalize_row(X, scale_factor, by) is_zero])
1583 is_zero = norms == 0
1584 scale_factor = np.median(norms[
-> 1585 norms /= scale_factor
1586 # for those rows or columns that summed to 0, just do nothing
1587 norms[norms == 0] = 1
UFuncTypeError: Cannot cast ufunc 'true_divide' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
请问怎么解决呀
Hi, I'm sorry to bother you again. TWhen I try to reload the CAME resulting data
dpair, model = came.load_dpair_and_model(came_resdir)
, I got a KeyError as follows:
`[] Setting dataset names:
0-->PR_Ms
1-->PR_NHP
[] Setting aligned features for observation nodes (self._features)
Traceback (most recent call last):
File "softs/conda/Conda_data/envs/env_came/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3802, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 165, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 5745, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 5753, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 1, in
File "softs/conda/Conda_data/envs/env_came/lib/python3.8/site-packages/came/utils/analyze.py", line 115, in load_dpair_and_model
dpair = DataPair(**element_dict)
File "softs/conda/Conda_data/envs/env_came/lib/python3.8/site-packages/came/datapair/unaligned.py", line 120, in init
self.set_features(features, varnames_feat)
File "softs/conda/Conda_data/envs/env_came/lib/python3.8/site-packages/came/datapair/unaligned.py", line 597, in set_features
self.dataset_names[0]: varnames_feat[0],
File "softs/conda/Conda_data/envs/env_came/lib/python3.8/site-packages/pandas/core/frame.py", line 3807, in getitem
indexer = self.columns.get_loc(key)
File "softs/conda/Conda_data/envs/env_came/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3804, in get_loc
raise KeyError(key) from err
KeyError: 0
`
Here is my env_came environment:
Package Version
anndata 0.8.0
came 0.1.9
certifi 2022.12.7
charset-normalizer 3.0.1
contourpy 1.0.7
cycler 0.11.0
dgl 0.9.0
fonttools 4.38.0
h5py 3.8.0
idna 3.4
igraph 0.10.4
importlib-metadata 6.0.0
importlib-resources 5.12.0
joblib 1.2.0
kiwisolver 1.4.4
leidenalg 0.9.1
llvmlite 0.39.1
matplotlib 3.7.0
natsort 8.2.0
networkx 3.0
numba 0.56.4
numpy 1.23.2
packaging 23.0
pandas 1.5.3
patsy 0.5.3
Pillow 9.4.0
pip 22.3.1
psutil 5.9.4
pynndescent 0.5.8
pyparsing 3.0.9
python-dateutil 2.8.2
python-igraph 0.10.4
pytz 2022.7.1
requests 2.28.2
scanpy 1.9.2
scikit-learn 1.1.2
scipy 1.10.1
seaborn 0.12.2
session-info 1.0.0
setuptools 65.6.3
six 1.16.0
statsmodels 0.13.5
stdlib-list 0.8.0
texttable 1.6.7
threadpoolctl 3.1.0
torch 1.12.1
tqdm 4.64.1
typing_extensions 4.5.0
umap-learn 0.5.3
urllib3 1.26.14
wheel 0.38.4
zipp 3.14.0
Hope to your reply. Thanks for your time.
Hi, thanks for developing CAME, looks great.
I am trying to go through your tutorial with your example data. When I reached to this part of your code:
######### contingency matrix for query dataset
ax, contmat = pl.plot_contingency_mat(
y_true, y_pred, norm_axis=1,
order_rows=False, order_cols=False,
)
AttributeError: module 'numpy' has no attribute 'int'.
np.int
was a deprecated alias for the builtin int
. To avoid this error in existing code, use int
by itself. Doing this will not modify any behavior and is safe. When replacing np.int
, you may wish to use e.g. np.int64
or np.int32
to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
After converting np.int to int in plot.y, now I am facing this issue:
Traceback (most recent call last):
Cell In[93], line 1
ax, contmat = pl.plot_contingency_mat(
File ~\AppData\Local\miniconda3\lib\site-packages\came\utils\plot.py:253 in plot_contingency_mat
contmat = wrapper_contingency_mat(
File ~\AppData\Local\miniconda3\lib\site-packages\came\utils\analyze.py:1677 in wrapper_contingency_mat
mat = pp.normalize_norms(mat, axis=normalize_axis)
File ~\AppData\Local\miniconda3\lib\site-packages\came\utils\preprocess.py:1624 in normalize_norms
return foo(X, scale_factor=scale_factor, by=by)
File ~\AppData\Local\miniconda3\lib\site-packages\came\utils\preprocess.py:1587 in normalize_row
is_zero = -_eps <= norms <= _eps
**ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()**
Can you help me to resolve the issue?
Many thanks!
I am working in Spyder (Python 3.10) through Anaconda Navigator
Hi, I am trying to ran pipeline with the example data. I got an error, which is below.
Here is the code I ran:
outputs = pipeline.main_for_unaligned(
**came_inputs,
df_varmap=df_varmap,
df_varmap_1v1=df_varmap_1v1,
dataset_names=dsnames,
key_class1=key_class1,
key_class2=key_class2,
do_normalize=True,
keep_non1v1_feats=keep_non1v1_feats,
n_epochs=n_epochs,
resdir=resdir,
n_pass=n_pass,
batch_size=batch_size,
plot_results=True,
)
This is the error I got:
TypeError(ypeError: np.matrix is not supported. Please convert to a numpy array with np.asarray. For more information se:https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
if I change NumPy version-1.20, but pandas and scanpy is not useful for came.
I think the reason for the error is numpy, I want to ask about the version of package that came applicable, and also how to solve the above error
Please let me know any advice on how to resolve this. Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.