Hi,

Thanks for providing the Sciplex example. Can you also please provide an example for the genetic interactions application?

Originally posted by @Naghipourfar in #1 (comment)

The dataset in example folder (https://www.dropbox.com/sh/cl8e4dm6a5peyoi/AAC2Oj200ii34k77Q0XGD2Nia?dl=0) has been deleted. Could you upload a toy dataset in anywhere else? Thx!

When attempting to load the SCIPLEX chemical dataset using the perturbnet_sciplex_example_notebook.ipynb file, my system's RAM becomes fully utilized and the process is killed. My system has 64 GB of RAM, but it appears that loading this dataset exceeds its capacity.

This issue occurs during the execution of the notebook, specifically when loading the SCIPLEX chemical dataset. Despite having sufficient RAM, the process is unable to complete due to excessive memory consumption. The error comes in the following line of code:

(2) load models

generation scvi

adata_train = adata[idx_to_train, :].copy()
adata_train = adata_train[kept_indices, :].copy()

scvi.data.setup_anndata(adata_train, layer = "counts")
scvi_model_cinn = scvi.model.SCVI.load(path_scvi_model_cinn, adata_train, use_cuda = False)
scvi_model_de = scvi_predictive_z(scvi_model_cinn)

device = 'cuda' if torch.cuda.is_available() else 'cpu'

ChemicalVAE

model_chemvae = ChemicalVAE(n_char = data_chem_onehot.shape[2], max_len = data_chem_onehot.shape[1]).to(device)
model_chemvae.load_state_dict(torch.load(path_chemvae_model, map_location = device))
model_chemvae.eval()

I would like to request assistance in understanding the system requirements for running PerturbNet and resolving this issue to successfully load the SCIPLEX chemical dataset without exhausting the available RAM.

Thanks for the nice work! we find this interesting. But when trying to follow the models, we find that:

There is one file loaded as data_gi_onehot_all. The dimension of it is cell x genes. I am not sure how is this matrix generated? Does the 1 means that specific gene is perturbed in a cell?

For the other file loaded as data_gi_onehot. The dimension of it is number of perturbed genes x number of all genes. Same, I find that when there are only 2 genes perturbed, there could be more than two 1 in that row, which indicates that this is not a matrix that tells the relationship between perturbed genes and all the genes used. So, how should I generate this matrix?
Cheers,
Yue