Giter VIP home page Giter VIP logo

uni-mol's Introduction

Official Repository for the Uni-Mol Series Methods

Welcome to the official repository for the Uni-Mol series methods!

Navigation

Fold In Short Description
Uni-Mol Molecule Representation Framework Uni-Mol is a universal 3D molecular representation learning framework designed for various downstreams tasks. You can use Uni-Mol in various molecule related tasks, including molecule property prediction, binding pose prediction etc. Released with Zhou et al. 2022 (ICLR 2023 version).
Uni-Mol+ Molecule Quantumn Chemical Modeling Un-Mol+ is design for molecule quantumn chemical property prediction, and ranks the 1st in two well-known benchmarks, OGB-LSC and OC20. You can use Uni-Mol+ in molecule geometry related tasks, such as conformation generation and optimization, quantumn property prediction. Released with Lu et al. 2023 (Nat Comm, Aug 2024).
Uni-Mol Tools Molecule Property Prediction Tools Uni-Mol tools is a easy-use wrappers for automatic property prediction, representation with Uni-Mol. You can install with pip install unimol-tools. Released with Gao et al. 2023. (Arixv, Apr 2023).
Uni-Mol Docking Protein Ligand Docking Tools Uni-Mol Docking achieves industry-leading performance in complex structure prediction, comparable to AlphaFold3. You can use Uni-Mol Docking for target docking with given pockets. Released with E Alcaide et al. 2024. (Arixv, May 2024).

Uni-Mol: A Universal 3D Molecular Representation Learning Framework

[Paper], [Uni-Mol Docking Colab]

Authors: Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, Guolin Ke

Schematic illustration of the Uni-Mol framework

Uni-Mol is a universal 3D molecular pretraining framework that offers a significant expansion of representation capacity and application scope in drug design. The framework comprises two models: a molecular pretraining model that has been trained using 209M molecular 3D conformations, and a pocket pretraining model that has been trained using 3M candidate protein pocket data. These two models can be used independently for different tasks and are combined for protein-ligand binding tasks. Uni-Mol has demonstrated superior performance compared to the state-of-the-art (SOTA) in 14 out of 15 molecular property prediction tasks. Moreover, Uni-Mol has achieved exceptional accuracy in 3D spatial tasks, such as protein-ligand binding pose prediction and molecular conformation generation.

Check this subfolder for more detalis.

Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+

arXiv PWC PWC

Schematic illustration of the Uni-Mol+ framework

Uni-Mol+ is a model for quantum chemical property prediction. Firstly, given a 2D molecular graph, Uni-Mol+ generates an initial 3D conformation from inexpensive methods such as RDKit. Then, the initial conformation is iteratively optimized to its equilibrium conformation, and the optimized conformation is further used to predict the QC properties. In the PCQM4MV2 and OC20 bencmarks, Uni-Mol+ outperforms previous SOTA methods by a large margin.

Check this subfolder for more detalis.

Uni-Mol tools for property prediction, representation and downstreams

Uni-Mol tools is a easy-use wrappers for property prediction,representation and downstreams with Uni-Mol. It includes the following tools:

  • molecular property prediction with Uni-Mol.
  • molecular representation with Uni-Mol.
  • other downstreams with Uni-Mol.

Check this subfolder for more detalis.

Documentation of Uni-Mol tools is available at https://unimol.readthedocs.io/en/latest/

Uni-Mol Docking V2: Towards realistic and accurate binding pose prediction

arXiv Static Badge

Uni-Mol Docking V2 Bohrium App

We update Uni-Mol Docking to Uni-Mol Docking V2, which demonstrates a remarkable improvement in performance, accurately predicting the binding poses of 77+% of ligands in the PoseBusters benchmark with an RMSD value of less than 2.0 Å, and 75+% passing all quality checks. This represents a significant increase from the 62% achieved by the previous Uni-Mol Docking model. Notably, our Uni-Mol Docking approach generates chemically accurate predictions, circumventing issues such as chirality inversions and steric clashes that have plagued previous ML models.

Check this subfolder for more detalis.

Service of Uni-Mol Docking V2 is avaiable at https://bohrium.dp.tech/apps/unimoldockingv2

News

May 20 2024: We release Uni-Mol Docking V2, including data, model and weight.

Jul 7 2023: We update a new version of Uni-Mol+, including the model setting for OC20 and a better performance on PCQM4MV2.

Jun 9 2023: We release Uni-Mol tools for property prediction, representation and downstreams.

Mar 16 2023: We release Uni-Mol+, a model for quantum chemical property prediction.

Jan 21 2023: Uni-Mol is accepted by ICLR 2023.

Oct 12 2022: Provide a demo to get Uni-Mol molecular representation.

Sep 20 2022: Provide Uni-Mol based IFD scoring function baseline for AIAC 2022 Competition Prediction of protein binding ability of drug molecules.

Sep 9 2022: Provide Uni-Mol binding pose prediction (docking) demo on Colab.

Sep 8 2022:

  • The code and data for protein-ligand binding pose prediction are released.
  • Finetuned model weights of molecular conformation generation and protein-ligand binding pose prediction are released.
  • Paper update.

Aug 17 2022: Pretrained models are released.

Jul 10 2022: Pretraining codes are released.

Jun 10 2022: The 3D conformation data used in Uni-Mol is released.

Contact Us

1. Github issue (recommended)
you can open an issue on GitHub.

2. WeChat
We welcome everyone to join our Uni-Mol user WeChat group. Scan the QR code below to join.

3. slack
Overseas users can scan the QR code below to join the Uni-Mol discussion channel on Slack.

4. E-mail
If you wish to have in-depth contact with the Uni-Mol development team, please send an email to [email protected].

Citation

Please kindly cite our papers if you use the data/code/model.

@inproceedings{
  zhou2023unimol,
  title={Uni-Mol: A Universal 3D Molecular Representation Learning Framework},
  author={Gengmo Zhou and Zhifeng Gao and Qiankun Ding and Hang Zheng and Hongteng Xu and Zhewei Wei and Linfeng Zhang and Guolin Ke},
  booktitle={The Eleventh International Conference on Learning Representations },
  year={2023},
  url={https://openreview.net/forum?id=6K2RM6wVqKu}
}
@misc{lu2023highly,
      title={Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+}, 
      author={Shuqi Lu and Zhifeng Gao and Di He and Linfeng Zhang and Guolin Ke},
      year={2023},
      eprint={2303.16982},
      archivePrefix={arXiv},
      primaryClass={physics.chem-ph}
}

License

This project is licensed under the terms of the MIT license. See LICENSE for additional details.

uni-mol's People

Contributors

54yyyu avatar chrisxu2016 avatar emotionor avatar guolinke avatar hongshuaiwang1 avatar hypnopump avatar i2vec avatar leoyml avatar letian88 avatar liuzan-info avatar lucifer1004 avatar naplessss avatar nbrosse avatar robotcator avatar zhangkaihua88 avatar zhougengmo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

uni-mol's Issues

Performance Validation of the down-stream tasks

Hi, thank you for providing the scripts of pre-training and fintuning task using Uni-Mol. How could we validate the results of these tasks, and compare the results to your paper? Could you provide the scripts or command line of testing using different metrics?

FloatingPointError when training on ligands for molecular pretrain

[nan, nan, nan, ..., nan, nan, nan]], device='cuda:0',
dtype=torch.float16), 'pair2coord_proj.linear1.bias': tensor([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan],
device='cuda:0', dtype=torch.float16), 'pair2coord_proj.linear2.weight': tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]],
device='cuda:0', dtype=torch.float16), 'pair2coord_proj.linear2.bias': tensor([nan], device='cuda:0', dtype=torch.float16)}
Traceback (most recent call last):
File "/opt/conda/bin/unicore-train", line 33, in
sys.exit(load_entry_point('unicore==0.0.1a0', 'console_scripts', 'unicore-train')())
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 403, in cli_main
distributed_utils.call_main(args, main)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 190, in call_main
distributed_main(args.device_id, main, args, kwargs)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 164, in distributed_main
main(args, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 126, in main
valid_losses, should_stop = train(args, trainer, task, epoch_itr, ckp_copy_thread)
File "/opt/conda/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 216, in train
log_output = trainer.train_step(samples)
File "/opt/conda/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/trainer.py", line 686, in train_step
grad_norm = self.clip_grad_norm(self.args.clip_norm)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/trainer.py", line 868, in clip_grad_norm
return self.optimizer.clip_grad_norm(
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/optim/fp16_optimizer.py", line 182, in clip_grad_norm
self.scaler.check_overflow(grad_norm)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/optim/dynamic_loss_scaler.py", line 62, in check_overflow
raise FloatingPointError(
FloatingPointError: Minimum loss scale reached (0.0001). Your loss is probably exploding. Try lowering the learning rate, using gradient clipping or increasing the batch size.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 510) of binary: /opt/conda/bin/python
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/opt/conda/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
elastic_launch(
File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

I've lower the lr from 1e-4 to 1e-5 and increase batchsize from 128 to 256. But problem still occurs. Btw, my gpu is A100*1 40gb memory.

Looking forward to your reply, thx.

what is the dataset formulation

for the pocket, i see there are something like
'coordinates':
[array([[ 56.689, -15...e=float32)]
'meta_info':
{'config': {'cx': 56.0925, 'cy': -19.674, 'cz': -36.187, 'sx': 5.4703043478260795, 'sy': 6.121739130434774, 'sz': 4.126855072463773}, 'fpocket': {'Score': '-0.046', 'Druggability Score': '0.000', 'Number of Alpha Spheres': '15', 'Total SASA': '65.409', 'Polar SASA': '37.634', 'Apolar SASA': '27.775', 'Volume': '158.569', 'Mean local hydrophobic density': '3.000', 'Mean alpha sphere radius': '3.835', ...}}
'side':
[0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, ...]
'residue':
['B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', 'B190', ...]
'pdbid':
'7cw4-fep'

can you please explain what does those things stand for?
thank you!

Prediction results acquisition

万分感谢uni-mol开发者提供的优秀项目,我想请教下对于分子性质预测问题,预测得到的性质以及相对应的标签应该如何获取,直接在unimol/losses/reg_loss.py将df["predict"]和df["target"]写到文件里是否可行?谢谢。

Thanks to uni-mol developers for the excellent project. I would like to ask how to get the predicted properties and the corresponding labels for the molecular property prediction task. Is it feasible to write df["predict"] and df["target"] to the file directly in unimol/losses/reg_loss.py? Thank you.

Unexpected MSE_draggabbility in pocket property predition task

Hi, I was using your provided script to test the validity of the pre-trained model. I added the provided testset in the --valid-subset parameter, here is my script.

image

However, I got a very low mse result (0.01) on the testset, which is not consistent with the metric provided in your paper (0.499 (0.007)).

origin_img_v2_f88970e1-82dd-42e1-a9d6-7f56d47b1f4g

Is there something wrong with my interpretation? Could you expain why the result is different from the paper?

A question about the Molecular dataset in page 15 of the Uni-Mol paper.

Hi~ Sorry to bother you again.

We have another question about the last sentence of Molecular dataset part in Appendix A Pretraining data: For each molecule, we add random conformer augmentations with ten 3D conformers generated by RDKit and one 2D graph to avoid ETKDG patterns missing match.
Could you tell us how to finish those two tasks in details? It will be much better if there exist codes.

Sincerely look forward to your answer~

How the model and task work together?

Hi. thanks for your work, but I can't figure out how the model and task work together. Maybe you use @register_task and @register_model to launch the code?

And I have another question about the molecule demo in example data. When I print the "src_tokens", "src_coord", "src_distance"... those outputs of the function "one_dataset" in file unimol/tasks/unimol.py, their shape don't contain any number about 56. But when I print the "src_tokens", "src_coord", "src_distance"... those inputs of the forward function in file unimol/models/unimol.py, their shape equal to torch.Size([128, 56]), I know the first dimension 128 is batch size, but where is the source of the second dimension 56?

Sincerely expect your answers.

runtime error when running molecular conformation finetuning task

Traceback (most recent call last):
File "/opt/conda/bin/unicore-train", line 33, in
sys.exit(load_entry_point('unicore==0.0.1a0', 'console_scripts', 'unicore-train')())
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 403, in cli_main
distributed_utils.call_main(args, main)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 190, in call_main
distributed_main(args.device_id, main, args, kwargs)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 164, in distributed_main
main(args, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 126, in main
valid_losses, should_stop = train(args, trainer, task, epoch_itr, ckp_copy_thread)
File "/opt/conda/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 216, in train
log_output = trainer.train_step(samples)
File "/opt/conda/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/trainer.py", line 628, in train_step
raise e
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/trainer.py", line 594, in train_step
loss, sample_size_i, logging_output = self.task.train_step(
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/tasks/unicore_task.py", line 283, in train_step
optimizer.backward(loss)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/optim/fp16_optimizer.py", line 74, in backward
loss.backward()
File "/opt/conda/lib/python3.8/site-packages/torch/_tensor.py", line 396, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/opt/conda/lib/python3.8/site-packages/torch/autograd/init.py", line 173, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
File "/opt/conda/lib/python3.8/site-packages/torch/autograd/function.py", line 253, in apply
return user_fn(self, *args)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/modules/softmax_dropout.py", line 23, in backward
softmax_results, dropout_mask = ctx.saved_tensors
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [CUDABFloat16Type [8192, 16, 16]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

Running cmd is:
data_path='./data/conformation_generation' # replace to your data path
save_dir='./save/save_finetune/save_confgen' # replace to your save path
n_gpu=1
MASTER_PORT=10086
dict_name='dict.txt'
weight_path='./weights/mol_pre_no_h_220816.pt' # replace to your ckpt path
task_name='qm9' # or 'drugs', conformation generation task name, as a part of complete data path
dist=8.0
recycles=4
coord_loss=1
distance_loss=1
beta=4.0
smooth=0.1
topN=20
lr=1e-4
batch_size=128
epoch=50
dropout=0.2
warmup=0.06
update_freq=1

export NCCL_ASYNC_ERROR_HANDLING=1
export OMP_NUM_THREADS=1
python -m torch.distributed.launch --nproc_per_node=$n_gpu --master_port=$MASTER_PORT $(which unicore-train) $data_path --task-name $task_name --user-dir ./unimol --train-subset train --valid-subset valid
--num-workers 8 --ddp-backend=c10d
--task mol_confG --loss mol_confG --arch mol_confG
--optimizer adam --adam-betas '(0.9, 0.99)' --adam-eps 1e-6 --clip-norm 1.0
--lr-scheduler polynomial_decay --lr $lr --warmup-ratio $warmup --max-epoch $epoch --batch-size $batch_size
--update-freq $update_freq --seed 1
--bf16 --bf16-sr --fp16-init-scale 4 --fp16-scale-window 256
--log-interval 100 --log-format simple --tensorboard-logdir $save_dir/tsb
--validate-interval 1 --keep-last-epochs 10
--keep-interval-updates 10 --best-checkpoint-metric loss --patience 50 --all-gather-list-size 102400
--finetune-from-model $weight_path --save-dir $save_dir
--coord-loss $coord_loss --distance-loss $distance_loss --dist-threshold $dist
--num-recycles $recycles --beta $beta --smooth $smooth --topN $topN
--find-unused-parameters

The docking results reproduction

Hello, it's relly a wonderful work!
I tried your scipts and checkpoint in the README as follows:

  1. I downloaded the checkpoint in binding_pose_ckpt to ./save_pose/binding_pose_220908.pt
  2. I downloaded the data in binding_data and decompressed it to ./protein_ligand_binding_pose_prediction
  3. I ran the inference code on the test set
  data_path="./protein_ligand_binding_pose_prediction"  # replace to your data path
  results_path="./infer_pose"  # replace to your results path
  weight_path="./save_pose/binding_pose_220908.pt"
  batch_size=8
  dist_threshold=8.0
  recycling=3
  
  python ./unimol/infer.py --user-dir ./unimol $data_path --valid-subset test \
         --results-path $results_path \
         --num-workers 8 --ddp-backend=c10d --batch-size $batch_size \
         --task docking_pose --loss docking_pose --arch docking_pose \
         --path $weight_path \
         --fp16 --fp16-init-scale 4 --fp16-scale-window 256 \
         --dist-threshold $dist_threshold --recycling $recycling \
         --log-interval 50 --log-format simple

The output of this script is ./infer_pose/weights_test.out.pkl
4. I ran the docking

nthreads=20  # Num of threads
predict_file="./infer_pose/weights_test.out.pkl"  # Your inference file dir
reference_file="./protein_ligand_binding_pose_prediction/test.lmdb"  # Your reference file dir
output_path="./protein_ligand_binding_pose_prediction"  # Docking results path

python ./unimol/utils/docking.py --nthreads $nthreads --predict-file $predict_file --reference-file $reference_file --output-path $output_path

and got the result below,

RMSD < 1.0 :  0.4405594405594406
RMSD < 1.5 :  0.6853146853146853
RMSD < 2.0 :  0.8041958041958042
RMSD < 3.0 :  0.8706293706293706
RMSD < 5.0 :  0.9440559440559441
avg RMSD :  1.6639526207451638

which is not consistent with the results in your paper in chemrxiv. (more like Uni-Mol random result)
So is there anything wrong with my inference and docking pipeline?
Thanks for your attention and look forward to your reply~

Running inference outside of pre-computed datasets

Dear authors,

Congratulations on Uni-Mol! This is truly exciting work 🥇

I'm writing to express interest in running inference outside of the pre-computed datasets. For example:

1.) Running docking from a pdb file + bounding box
2.) Embedding molecules from smiles
3.) Embedding proteins from pdb files

Looking at the code, it wasn't clear to me where to get started on this, but let me know if I can help.

Best wishes,

-Cyrus

Molecular Property Prediction 'qm9dft' AttributeError: 'NoneType' object has no attribute 'float'

Traceback (most recent call last):
File "/opt/conda/bin/unicore-train", line 33, in
sys.exit(load_entry_point('unicore==0.0.1', 'console_scripts', 'unicore-train')())
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 403, in cli_main
distributed_utils.call_main(args, main)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 190, in call_main
distributed_main(args.device_id, main, args, kwargs)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 164, in distributed_main
main(args, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 126, in main
valid_losses, should_stop = train(args, trainer, task, epoch_itr, ckp_copy_thread)
File "/opt/conda/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 230, in train
valid_losses, should_stop = validate_and_save(
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 305, in validate_and_save
valid_losses = validate(args, trainer, task, epoch_itr, valid_subsets)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 367, in validate
inner_logging_outputs = trainer.valid_step(sample)
File "/opt/conda/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore/trainer.py", line 809, in valid_step
_loss, sample_size, logging_output = self.task.valid_step(
File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore/tasks/unicore_task.py", line 289, in valid_step
loss, sample_size, logging_output = loss(model, sample)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/Uni-Mol/unimol/losses/reg_loss.py", line 27, in forward
net_output = model(
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/Uni-Mol/unimol/models/unimol.py", line 214, in forward
) = self.encoder(x, padding_mask=padding_mask, attn_mask=graph_attn_bias)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/Uni-Mol/unimol/models/transformer_encoder_with_pair.py", line 114, in forward
token_mask = 1.0 - input_padding_mask.float()
AttributeError: 'NoneType' object has no attribute 'float'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 34) of binary: /opt/conda/bin/python

module 'unicore.checkpoint_utils' has no attribute 'load_model_ensemble_and_task'

Traceback (most recent call last):
File "/mnt/d/projections/Uni-Mol-New/Uni-Mol/./unimol/conf_gen_infer.py", line 121, in
cli_main()
File "/mnt/d/projections/Uni-Mol-New/Uni-Mol/./unimol/conf_gen_infer.py", line 117, in cli_main
distributed_utils.call_main(args, main)
File "/mnt/d/software/install/miniconda/envs/cy/lib/python3.9/site-packages/unicore-0.0.1-py3.9-linux-x86_64.egg/unicore/distributed/utils.py", line 193, in call_main
main(args, **kwargs)
File "/mnt/d/projections/Uni-Mol-New/Uni-Mol/./unimol/conf_gen_infer.py", line 53, in main
models, saved_args, task = checkpoint_utils.load_model_ensemble_and_task(
AttributeError: module 'unicore.checkpoint_utils' has no attribute 'load_model_ensemble_and_task'

Where is the (checkpoint_utils.load_model_ensemble_and_task) in the file (conf_gen_infer.py), I didn't find it in unicore. Was it not added when the code was merged?

Exception using `notebooks/unimol_mol_repr_demo.ipynb`

Hi there,

While using your notebooks/unimol_mol_repr_demo.ipynb, my molecules result in an Exception from smi2coords function.

My molecules are following:

smi_list = [
"CC(C)CC(C(=O)NC(C(C)C)C(=O)O)NC(=O)CNC(=O)C(CCC(=O)N)NC(=O)C(CC(C)C)NC(=O)C(CC(C)C)NC(=O)C(CCCNC(=N)N)NC(=O)C(CCC(=O)N)NC(=O)C(CC(C)C)NC(=O)C(CCCNC(=N)N)NC(=O)C(C)NC(=O)C(CO)NC(=O)C(CC(=O)O)NC(=O)C(CCCNC(=N)N)NC(=O)C(CC(C)C)NC(=O)C(CCCNC(=N)N)NC(=O)C(CO)NC(=O)C(CC(C)C)NC(=O)C(CCC(=O)O)NC(=O)C(CO)NC(=O)C(C(C)O)NC(=O)C(CC1=CC=CC=C1)NC(=O)C(C(C)O)NC(=O)CNC(=O)C(CC(=O)O)NC(=O)C(CO)NC(=O)C(CC2=NC=CN2)N",
"CC(C)CC1C(=O)NC(C(=O)NC(C(=O)NC(CSSCC(C(=O)NC(C(=O)NC(C(=O)N1)CC(=O)N)CO)N)C(=O)NC(C(C)C)C(=O)NC(CC(C)C)C(=O)NCC(=O)NC(CCCCN)C(=O)NC(CC(C)C)C(=O)NC(CO)C(=O)NC(CCC(=O)N)C(=O)NC(CCC(=O)O)C(=O)NC(CC(C)C)C(=O)NC(CC2=CN=CN2)C(=O)NC(CCCCN)C(=O)NC(CC(C)C)C(=O)NC(CCC(=O)N)C(=O)NC(C(C)O)C(=O)NC(CC3=CC=C(C=C3)O)C(=O)N4CCCC4C(=O)NC(CCCNC(=N)N)C(=O)NC(C(C)O)C(=O)NC(CC(=O)N)C(=O)NC(C(C)O)C(=O)NCC(=O)NC(CO)C(=O)NCC(=O)NC(C(C)O)C(=O)N5CCCC5C(=O)N)C(C)O)CO",
"C(C1C(C(C(C(O1)OC2(C(C(C(O2)CO)O)O)COC3(C(C(C(O3)CO)O)O)COC4(C(C(C(O4)CO)O)O)COC5(C(C(C(O5)CO)O)O)COC6(C(C(C(O6)CO)O)O)COC7(C(C(C(O7)CO)O)O)COC8(C(C(C(O8)CO)O)O)COC9(C(C(C(O9)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)COC1(C(C(C(O1)CO)O)O)CO)O)O)O)O",
]

Error Stack is following:

ValueError                                Traceback (most recent call last)
Cell In [4], line 9
      7 batch_size=16
      8 results_path=data_path   # replace to your save path
----> 9 write_lmdb(smi_list, job_name=job_name, seed=seed, outpath=data_path)

Cell In [3], line 45, in write_lmdb(smiles_list, job_name, seed, outpath)
     43 txn_write = env_new.begin(write=True)
     44 for i, smiles in tqdm(enumerate(smiles_list)):
---> 45     inner_output = smi2coords(smiles, seed=seed)
     46     txn_write.put(f"{i}".encode("ascii"), inner_output)
     47 txn_write.commit()

Cell In [3], line 21, in smi2coords(smi, seed)
     19     except:
     20         pass
---> 21     coordinates = mol_tmp.GetConformer().GetPositions()
     22 assert len(atoms) == len(coordinates), "coordinates shape is not align with {}".format(smi)
     23 coordinate_list.append(coordinates.astype(np.float32))

ValueError: Bad Conformer Id

So why this happens? Is this project inable to process such long molecules?

Error running molecular conformation generation task

The following error was encountered when starting the molecular conformation generation task

2022-08-01 08:36:50 | INFO | unicore.trainer | Loaded checkpoint /root/data/save_moleculer_pretrain/checkpoint_best.pt (epoch 16 @ 0 updates)
2022-08-01 08:36:50 | INFO | unicore.trainer | loading train data for epoch 1
2022-08-01 08:36:50 | INFO | unicore.tasks.unicore_task | get EpochBatchIterator for epoch 1
2022-08-01 08:36:51 | INFO | unicore.optim.adam | using FusedAdam
2022-08-01 08:36:51 | INFO | unicore.trainer | begin training epoch 1
2022-08-01 08:36:51 | INFO | unicore_cli.train | Start iterating over samples
/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/autograd/__init__.py:173: UserWarning: Error detected in SoftmaxDropoutFastBackward. Traceback of forward call that caused the error:
  File "/root/miniconda3/envs/py38/bin/unicore-train", line 33, in <module>
    sys.exit(load_entry_point('unicore==0.0.1a0', 'console_scripts', 'unicore-train')())
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 403, in cli_main
    distributed_utils.call_main(args, main)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 190, in call_main
    distributed_main(args.device_id, main, args, kwargs)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 164, in distributed_main
    main(args, **kwargs)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 126, in main
    valid_losses, should_stop = train(args, trainer, task, epoch_itr, ckp_copy_thread)
  File "/root/miniconda3/envs/py38/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 216, in train
    log_output = trainer.train_step(samples)
  File "/root/miniconda3/envs/py38/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/trainer.py", line 613, in train_step
    loss, sample_size_i, logging_output = self.task.train_step(
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/tasks/unicore_task.py", line 279, in train_step
    loss, sample_size, logging_output = loss(model, sample)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/Uni-Mol/unimol/losses/conf_gen.py", line 29, in forward
    net_output = model(**sample["net_input"])
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/Uni-Mol/unimol/models/conf_gen.py", line 117, in forward
    x, attn_mask = single_encoder(x, padding_mask=padding_mask, attn_mask=attn_mask)
  File "/root/Uni-Mol/unimol/models/conf_gen.py", line 103, in single_encoder
    x, attn_mask, _ = self.unimol.encoder.layers[i](x, padding_mask=padding_mask, attn_bias=attn_mask, return_attn=True)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/modules/transformer_encoder_layer.py", line 71, in forward
    x = self.self_attn(
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/modules/multihead_attention.py", line 95, in forward
    attn_probs = softmax_dropout(
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/modules/softmax_dropout.py", line 106, in softmax_dropout
    return SoftmaxDropoutFast.apply(
 (Triggered internally at  /opt/conda/conda-bld/pytorch_1646755903507/work/torch/csrc/autograd/python_anomaly_mode.cpp:104.)
  Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
Traceback (most recent call last):
  File "/root/miniconda3/envs/py38/bin/unicore-train", line 33, in <module>
    sys.exit(load_entry_point('unicore==0.0.1a0', 'console_scripts', 'unicore-train')())
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 403, in cli_main
    distributed_utils.call_main(args, main)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 190, in call_main
    distributed_main(args.device_id, main, args, kwargs)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 164, in distributed_main
    main(args, **kwargs)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 126, in main
    valid_losses, should_stop = train(args, trainer, task, epoch_itr, ckp_copy_thread)
  File "/root/miniconda3/envs/py38/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore_cli/train.py", line 216, in train
    log_output = trainer.train_step(samples)
  File "/root/miniconda3/envs/py38/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/trainer.py", line 649, in train_step
    raise e
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/trainer.py", line 613, in train_step
    loss, sample_size_i, logging_output = self.task.train_step(
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/tasks/unicore_task.py", line 283, in train_step
    optimizer.backward(loss)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/optim/fp16_optimizer.py", line 74, in backward
    loss.backward()
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/_tensor.py", line 363, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/autograd/__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/autograd/function.py", line 253, in apply
    return user_fn(self, *args)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/modules/softmax_dropout.py", line 30, in backward
    softmax_results, dropout_mask = ctx.saved_tensors
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.HalfTensor [8192, 16, 16]], which is output 0 of AsStridedBackward0, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

Difference of embedding layer dimension between pretrained model and dict.txt

Hi there,

I'm trying to load pretrained weights of molecular pretrain(https://github.com/dptech-corp/Uni-Mol/releases/download/v0.1/mol_pre_no_h_220816.pt), but using the example_data/molecule/dict.txt leads to an Exception below.

RuntimeError: Error(s) in loading state_dict for MoleculeEmbeddingModel:
        size mismatch for embed_tokens.weight: copying a param with shape torch.Size([31, 512]) from checkpoint, the shape in current model is torch.Size([30, 512]).
        size mismatch for gbf.mul.weight: copying a param with shape torch.Size([961, 1]) from checkpoint, the shape in current model is torch.Size([900, 1]).
        size mismatch for gbf.bias.weight: copying a param with shape torch.Size([961, 1]) from checkpoint, the shape in current model is torch.Size([900, 1]).

I got the cause of this Exception, since adding a line code in the unimol/infer.py
self.mask_idx = dictionary.add_symbol("[MASK]", is_special=True) solves this problem.
(https://github.com/dptech-corp/Uni-Mol/blob/27ad2a0dbfafc9795b36efb279d7ed7c6d87a34a/unimol/tasks/unimol.py#L122)

My question is, why don't you just add a line of [MASK] to dict.txt to solve this problem?

My point is, whenever we use the checkpoints you offer, this code is irrelevant to coding but necessary for running.

What's your concern about this?

about dependencies

I tried to install via pip install git+git://github.com/dptech-crop/Uni-Core.git@stable#egg=Uni-Core. However, errors were reported while attempting to install on different servers, as shown below:

Collecting Uni-Core
  Cloning git://github.com/dptech-crop/Uni-Core.git (to revision stable) to /tmp/pip-install-0p4mpc94/uni-core_6f50236b1b74402c9bc79f56ec8e6144
  Running command git clone -q git://github.com/dptech-crop/Uni-Core.git /tmp/pip-install-0p4mpc94/uni-core_6f50236b1b74402c9bc79f56ec8e6144
  fatal: Unable to look up github.com (port 9418) (Name or service not known)
WARNING: Discarding git+git://github.com/dptech-crop/Uni-Core.git@stable#egg=Uni-Core. Command errored out with exit status 128: git clone -q git://github.com/dptech-crop/Uni-Core.git /tmp/pip-install-0p4mpc94/uni-core_6f50236b1b74402c9bc79f56ec8e6144 Check the logs for full command output.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x14ab3de32bb0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/uni-core/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x14ab3de32100>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/uni-core/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x14ab3de1d4f0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/uni-core/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x14ab3de2bf70>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/uni-core/
ERROR: Could not find a version that satisfies the requirement uni-core (unavailable) (from versions: none)
ERROR: No matching distribution found for uni-core (unavailable)

or

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting Uni-Core
  Cloning git://github.com/dptech-crop/Uni-Core.git (to revision stable) to /tmp/pip-install-ndj8wed7/uni-core_bcdcc56392f540958508e6f1705e1535
  Running command git clone -q git://github.com/dptech-crop/Uni-Core.git /tmp/pip-install-ndj8wed7/uni-core_bcdcc56392f540958508e6f1705e1535
  fatal: unable to connect to github.com:
  github.com[0: 20.205.243.166]: errno=连接超时

WARNING: Discarding git+git://github.com/dptech-crop/Uni-Core.git@stable#egg=Uni-Core. Command errored out with exit status 128: git clone -q git://github.com/dptech-crop/Uni-Core.git /tmp/pip-install-ndj8wed7/uni-core_bcdcc56392f540958508e6f1705e1535 Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement uni-core (unavailable) (from versions: none)
ERROR: No matching distribution found for uni-core (unavailable)

Do you have any way to install it smoothly?
Thank you for your great work!

Some abnormal data exists in the pocket pre-training dataset

When I start the pocket pretrain task, I get an error like below

File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/_utils.py", line 457, in reraise
    raise exception
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/data/base_wrapper_dataset.py", line 18, in __getitem__
    return self.dataset[index]
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/data/nested_dictionary_dataset.py", line 69, in __getitem__
    return OrderedDict((k, ds[index]) for k, ds in self.defn.items())
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/data/nested_dictionary_dataset.py", line 69, in <genexpr>
    return OrderedDict((k, ds[index]) for k, ds in self.defn.items())
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/data/base_wrapper_dataset.py", line 18, in __getitem__
    return self.dataset[index]
File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/data/append_token_dataset.py", line 21, in __getitem__
    item = self.dataset[idx]
  File "/root/miniconda3/envs/py38/lib/python3.8/site-packages/unicore-0.0.1a0-py3.8-linux-x86_64.egg/unicore/data/prepend_token_dataset.py", line 22, in __getitem__
    item = self.dataset[idx]
  File "/root/Uni-Mol/unimol/data/key_dataset.py", line 19, in __getitem__
    return self.dataset[idx][self.key]
  File "/root/Uni-Mol/unimol/data/mask_points_dataset.py", line 188, in __getitem__
    return self.__getitem_cached__(self.epoch, index)
  File "/root/Uni-Mol/unimol/data/mask_points_dataset.py", line 216, in __getitem_cached__
    ret['targets'][mask] = item[mask]
IndexError: The shape of the mask [225] at index 0 does not match the shape of the indexed tensor [224] at index 0

Then I checked the data set and found that the data size of the keywords in some of the data did not match, for example, as shown below

>>> datapoint_pickled = txn.get(keys[753139])
>>> keys[753139]
b'1677822'
>>> data = pickle.loads(datapoint_pickled)
>>> data.keys()
dict_keys(['atoms', 'coordinates', 'meta_info', 'side', 'residue', 'pdbid'])
>>> len(data['atoms'])
224
>>> len(data['coordinates'][0])
224
>>> data['coordinates'][0].shape
(224, 3)
>>> len(data['side'])
225
>>> len(data['residue'])
225

Then I checked its residue content and I can see that residue='B603' has only one atom, probably an abnormal data point

>>> from collections import Counter
>>> Counter(data['residue'])
Counter({'B424': 21, 'B321': 19, 'B252': 18, 'B296': 17, 'B365': 17, 'B322': 15, 'B302': 14, 'B363': 14, 'B301': 14, 'B300': 14, 'B402': 12, 'B364': 12, 'B293': 12, 'B366': 10, 'B701': 3, 'B703': 3, 'B706': 3, 'B705': 3, 'B704': 3, 'B603': 1})

Multi-Task Molecular Properties Prediction

感谢深势科技团队提供的unimol优秀项目,想请问下对于分子性质预测问题,目前bash“Molecular Property Prediction”中仅有task_num=n这样的选项,也就是应该只能对单个性质进行预测。非常希望unimol能够提供多个性质一同预测的模块,实现在多个性质约束下的组合性质最优。万分感谢。

Thanks for the excellent unimol project provided by DP Technology Team. Now there is only option such as “task_num=n” in bash "Molecular Property Prediction". That is to say, only a single property can be predicted at present. It is very desirable that Unimol can provide modules that predict multiple properties together. Realize optimal combination properties under multiple property constraints. Thank you!

About Loading the Pre-trained weights to finetune the down-stream tasks:

Hi, I was having doubts about whether I have correctly loaded the pre-trained weights to the model when I was trying to finetune the down-stream tasks. For example, when I was trying the task of pocket property prediction, how should I use the pre-trained weights (i.e. how to set the weight path in my training script)? Should I set the $weight_path to be the path to provided pocket_pre_220816.pt ? After I do so, the printed training log is as follows:

image

Does this mean I have loaded the pre-trained weights correctly? According to the uni-core source code and the 2022-09-05 05:03:10 | INFO | unicore.trainer | No existing checkpoint found /home/ubuntu/Uni-Mol/pretrain_models/pocket_pre_220816.pt log, I fear that nothing was loaded.

The implement

Wonder still use jax to implement like Unifold? Really fantastic work . wait for releasing

Molecular Pretraining task training loss does not converge and keeps increasing

Training Loss change curve
image

valid Loss change curve
image

Training Loss principal component change curve
image

The startup script looks like this

data_path=/root/data/chrisxu/Uni-Mol/ligands
save_dir=/root/data/chrisxu/Uni-Mol/results/moleculer_pretrain_bf16
n_gpu=8
MASTER_PORT=10086
lr=1e-4
wd=1e-4
batch_size=32
update_freq=8
masked_token_loss=1
masked_coord_loss=5
masked_dist_loss=10
x_norm_loss=0.01
delta_pair_repr_norm_loss=0.01
mask_prob=0.15
only_polar=-1
noise_type='uniform'
noise=1.0
seed=1
warmup_steps=10000
max_steps=1000000

export NCCL_ASYNC_ERROR_HANDLING=1
export OMP_NUM_THREADS=1
python -m torch.distributed.launch --nproc_per_node=$n_gpu --master_port=$MASTER_PORT $(which unicore-train) $data_path  --user-dir ./unimol --train-subset train --valid-subset valid \
       --num-workers 8 --ddp-backend=c10d \
       --task unimol --loss unimol --arch unimol_base  \
       --optimizer adam --adam-betas '(0.9, 0.99)' --adam-eps 1e-6 --clip-norm 1.0 --weight-decay $wd \
       --lr-scheduler polynomial_decay --lr $lr --warmup-updates $warmup_steps --total-num-update $max_steps \
       --update-freq $update_freq --seed $seed \
       --bf16 --bf16-sr --fp16-init-scale 4 --fp16-scale-window 256 --tensorboard-logdir $save_dir/tsb \
       --max-update $max_steps --log-interval 10 --log-format simple \
       --save-interval-updates 10000 --validate-interval-updates 10000 --keep-interval-updates 10 --no-epoch-checkpoints  \
       --masked-token-loss $masked_token_loss --masked-coord-loss $masked_coord_loss --masked-dist-loss $masked_dist_loss \
       --x-norm-loss $x_norm_loss --delta-pair-repr-norm-loss $delta_pair_repr_norm_loss \
       --mask-prob $mask_prob --noise-type $noise_type --noise $noise --batch-size $batch_size \
       --save-dir $save_dir  --only-polar $only_polar

Where is the problem, Looking forward to your reply, thx.

setup.py

If possible could you please add a setup I would love to test this

How to use the pre-trained pocket model

Hi,I want to see the effect of the pre-trained pocket model separately, but I don't know how to load the pre-trained weights to the pocket model separately. In fact, I'm not sure which one in the code is the pocket model, is this
. /unimol/models/transformer_encoder_with_pair.py?

The part I want to get is the image below
image

Loading pretrained model in Jupyter notebook

Hello,
Very nice work!
I would like to load your pretrained molecular model to use it as part of my pipeline. I would like to load it in a Jupyter Notebook but for some reason, because of the interaction with the Uni-Core library I don't get it to work.
Any help/advice is appreciated.
Thank you!

TASK_REGISTRY not updated with UniMol tasks on conformation generation

Not sure whether to post it here or in Unicore issues tracker. When trying inference step after finetuning the model, Unicore throws KeyError on TASK_REGISTRY.

Launched from the docker image dptechnology/unimol:latest-pytorch1.11.0-cuda11.3

root@vega:Uni-Mol# python ./unimol/conf_gen_infer.py --user-dir ./unimol $data_path --task-name $task_name --valid-subset test        --results-path $results_path        --num-workers 8 --ddp-backend=c10d --batch-size $batch_size        --task mol_confG --loss mol_confG --arch mol_confG        --num-recycles $recycles        --path $weight_path        --fp16 --fp16-init-scale 4 --fp16-scale-window 256        --log-interval 50 --log-format simple
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | distributed init (rank 0): tcp://localhost:16276
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | distributed init (rank 1): tcp://localhost:16276
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 1
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | distributed init (rank 2): tcp://localhost:16276
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 2
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | distributed init (rank 3): tcp://localhost:16276
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 3
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:1 to store for rank: 0
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | initialized host vega as rank 0
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Rank 3: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | initialized host vega as rank 3
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Rank 2: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | initialized host vega as rank 2
2022-08-31 07:18:38 | INFO | torch.distributed.distributed_c10d | Rank 1: Completed store-based barrier for key:store_based_barrier_key:1 with 4 nodes.
2022-08-31 07:18:38 | INFO | unicore.distributed.utils | initialized host vega as rank 1
2022-08-31 07:18:43 | INFO | torch.distributed.distributed_c10d | Added key: store_based_barrier_key:2 to store for rank: 0
2022-08-31 07:18:43 | INFO | torch.distributed.distributed_c10d | Rank 0: Completed store-based barrier for key:store_based_barrier_key:2 with 4 nodes.
2022-08-31 07:18:43 | INFO | unimol.conf_gen_infer | loading model(s) from ./save_confgen/checkpoint_best.pt
Traceback (most recent call last):
  File "./unimol/conf_gen_infer.py", line 124, in <module>
    cli_main()
  File "./unimol/conf_gen_infer.py", line 120, in cli_main
    distributed_utils.call_main(args, main)
  File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 180, in call_main
    torch.multiprocessing.spawn(
  File "/opt/conda/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 240, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/opt/conda/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 198, in start_processes
    while not context.join():
  File "/opt/conda/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 3 terminated with the following error:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
    fn(i, *args)
  File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore/distributed/utils.py", line 164, in distributed_main
    main(args, **kwargs)
  File "/home/andreev/2022/unimol/Uni-Mol/unimol/conf_gen_infer.py", line 56, in main
    task = tasks.setup_task(args)
  File "/opt/conda/lib/python3.8/site-packages/unicore-0.0.1-py3.8-linux-x86_64.egg/unicore/tasks/__init__.py", line 21, in setup_task
    return TASK_REGISTRY[args.task].setup_task(args, **kwargs)
KeyError: 'mol_confG'

Is there any way of updating UniCore TaskRegistry when launching conf_gen_infer.py?
Thanks!

lmdb file of moleculenet

Thanks for your interesting work!
Could you release the lmdb files of moleculenet, including BACE, BBBP, etc..? I want to reproduce the fine-tuning results.
Thanks again!

The number of pretraining steps

Hi Gengmo,
Thanks for your nice work ! I have a question about the number of pre-training epochs. From the Appendix in the paper, the number of max training steps for pretraining is 1M with batch size 128 (about 7 epochs).
Is such a small number of pretraining epochs due to early stopping?
Have you ever tried what happens if you train for more epochs?

Hyperparameters for protein-ligand binding pose prediction without pretraining

Hi, it’s really a wonderful work!
About the protein-ligand binding pose prediction task, you used the Uni-Mol model without pretraining, namely Uni-Molrandom . But I couldn’t found training details for this Uni-Molrandom model in your arXiv paper.
I would be very grateful if you could provide the hyperparameters for this model.

Generating atom and pair representations

Dear Authors,

Thank you and congratulations for your release of Uni-Mol! I am interested in using the atom and pair representations. From the code, I am not sure how to generate these directly (there is reference to a dictionary but I am not sure what this dictionary refers to). I would be very grateful if you could clarify how the code can be used to generate these representations using the pretrained models.

Thank you!

x_norm_loss and delta_pair_repr_norm_loss keep growing as training steps increase

As shown in the title, these two losses have been increasing. With more training steps, even using a very small loss weight (0.01) will lead to an increasing proportion of the total loss (above 80%), which will eventually lead to The whole training does not converge.
Both the demo data and the molecular pretrain dataset have the same problem.
Looking forward to your reply, thx.

image

image

Using Uni-Mol with mac

Hi, I'm trying to install and use Uni-Mol, but installing Uni-Core fails with the error:

Processing /Users/nadavschweiger/Projects/External/Uni-Core
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/Users/nadavschweiger/Projects/External/Uni-Core/setup.py", line 55, in <module>
          _, bare_metal_major, _ = get_cuda_bare_metal_version(cpp_extension.CUDA_HOME)
        File "/Users/nadavschweiger/Projects/External/Uni-Core/setup.py", line 38, in get_cuda_bare_metal_version
          raw_output = subprocess.check_output([cuda_dir + "/bin/nvcc", "-V"], universal_newlines=True)
      TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
      
      Warning: Torch did not find available GPUs on this system.
       If your intention is to cross-compile, this is not an error.
      By default, it will cross-compile for Volta (compute capability 7.0), Turing (compute capability 7.5),
      and, if the CUDA version is >= 11.0, Ampere (compute capability 8.0).
      If you wish to cross-compile for a single specific architecture,
      export TORCH_CUDA_ARCH_LIST="compute capability" before running setup.py.
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

Which is because CUDA is not installed. Since CUDA and macOS Monterey are non-compatible, what can I do to use Uni-Mol?
I intend, for now, to use the pretrained weights and not to train anything heavy, so I don't require a GPU right now. Also, even though training and heavy lifting will be done in a server, I do want to work and debug locally on my mac... What can I do?

Metric in Pocket Property Prediction

Hi, I am running an experiment of pocket property prediction using the provided weights and script. In the validation section, the log output the following mse metric.
03906ba760d0e0d5d5895920176cd9a

May I ask which MSE (MSE_Fpocket, MSE_Druggability, MSE_Total SASA or MSE_Hydrophobicity accroding to the Table 4 in your paper) is the printed metric here?

Hi @Naplessss , While using this demo, in Block `Infer from ckpt`, a `KeyError: 'unimol'` was raised.

Hi @Naplessss , While using this demo, in Block Infer from ckpt, a KeyError: 'unimol' was raised.

  File "/opt/anaconda3/envs/unimol/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
    fn(i, *args)
  File "/opt/anaconda3/envs/unimol/lib/python3.9/site-packages/unicore/distributed/utils.py", line 164, in distributed_main
    main(args, **kwargs)
  File "/root/Uni-Mol/unimol/infer.py", line 47, in main
    task = tasks.setup_task(args)
  File "/opt/anaconda3/envs/unimol/lib/python3.9/site-packages/unicore/tasks/__init__.py", line 22, in setup_task
    return TASK_REGISTRY[args.task].setup_task(args, **kwargs)
KeyError: 'unimol'

This is the whole stack.

And I also print the TASK_REGISTRY in unicore/tasks/__init__.py. well, during running, that variable is {}. Could you please tell me what happens, and how to solve this problem?

Originally posted by @zhoubay in #56 (comment)

assert total_train_steps is not None

I came accross assert total_train_steps is not None error when I was trying to use uni-mol on the pocket property prediction task. My script is directly copied from the README.md file. Could you tell me how to solve this?
image

data release plan

Hi,

Thanks for your wonderful work.

I'd like to ask when will you release the dataset.

Hi~Could you please offer the dataset of qm9dft for Molecule Property Prediction?

Hi~Sorry again.

Recently I have tried for the molecule property prediction, but I found the following question:

Traceback (most recent call last): File "/home/username/miniconda3/envs/uni/bin/unicore-train", line 8, in <module> sys.exit(cli_main()) File "/home/username/miniconda3/envs/uni/lib/python3.8/site-packages/unicore_cli/train.py", line 404, in cli_main distributed_utils.call_main(args, main) File "/home/username/miniconda3/envs/uni/lib/python3.8/site-packages/unicore/distributed/utils.py", line 191, in call_main distributed_main(args.device_id, main, args, kwargs) File "/home/username/miniconda3/envs/uni/lib/python3.8/site-packages/unicore/distributed/utils.py", line 165, in distributed_main main(args, **kwargs) File "/home/username/miniconda3/envs/uni/lib/python3.8/site-packages/unicore_cli/train.py", line 78, in main task.load_dataset(valid_sub_split, combine=False, epoch=1) File "/home/username/projects/UNIMOL/unimol/tasks/unimol_finetune.py", line 204, in load_dataset dataset = LMDBDataset(split_path) File "/home/username/miniconda3/envs/uni/lib/python3.8/site-packages/unicore/data/lmdb_dataset.py", line 19, in __init__ assert os.path.isfile(self.db_path), "{} not found".format( AssertionError: ./molecular_property_prediction/qm9dft/valid.lmdb not found
It seems that there is no qm9dft data.

So could you please offer the download url or any code for qm9dft?

Thanks a lot!

error when load the lmdb data

which rdkit version do you use?

[22:06:52] Depickling from a version number (13.0)that is higher than our version (12.1).
This probably won't work.
[22:06:52]

Pre-condition Violation
Number of atom mismatch
Violation occurred on line 617 in file /Users/glandrum/anaconda3/conda-bld/rdkit_1603167990487/work/Code/GraphMol/ROMol.cpp
Failed Expression: conf->getNumAtoms() == this->getNumAtoms()

RuntimeError Traceback (most recent call last)
in
----> 1 data = pickle.loads(datapoint_pickled)

RuntimeError: Pre-condition Violation
Number of atom mismatch
Violation occurred on line 617 in file Code/GraphMol/ROMol.cpp
Failed Expression: conf->getNumAtoms() == this->getNumAtoms()
RDKIT: 2020.09.1
BOOST: 1_73

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.