Giter VIP home page Giter VIP logo

units's Introduction

Unified Time Series Model

Project Page | Paper link

UniTS is a unified time series model that can process various tasks across multiple domains with shared parameters and does not have any task-specific modules.

Authors: Shanghua Gao Teddy Koker Owen Queen Thomas Hartvigsen Theodoros Tsiligkaridis Marinka Zitnik

Overview

Foundation models, especially LLMs, are profoundly transforming deep learning. Instead of training many task-specific models, we can adapt a single pretrained model to many tasks via few-shot prompting or fine-tuning. However, current foundation models apply to sequence data but not to time series, which present unique challenges due to the inherent diverse and multi-domain time series datasets, diverging task specifications across forecasting, classification and other types of tasks, and the apparent need for task-specialized models.

We developed UniTS, a unified time series model that supports a universal task specification, accommodating classification, forecasting, imputation, and anomaly detection tasks. This is achieved through a novel unified network backbone, which incorporates sequence and variable attention along with a dynamic linear operator and is trained as a unified model.

Across 38 multi-domain datasets, UniTS demonstrates superior performance compared to task-specific models and repurposed natural language-based LLMs. UniTS exhibits remarkable zero-shot, few-shot, and prompt learning capabilities when evaluated on new data domains and tasks.

UniTS-1

Setups

1. Requirements

Install Pytorch2.0+ and the required packages.

pip install -r requirements.txt

2. Prepare data

bash download_data_all.sh

Datasets configs for different multi-task settings are shown in .ymal files of the data_provider folder.

By default, all experiments follow the multi-task setting where one UniTS model is jointly trained on mulitple datasets.

3. Train and evaluate model

1. Multi-task learning on forecasting and classification tasks:

  • Pretraining + Prompt learning
bash ./scripts/pretrain_prompt_learning/UniTS_pretrain_x128.sh
  • Supervised learning
bash ./scripts/supervised_learning/UniTS_supervised.sh

2. Few-shot transfer learning on new forecasting and classification tasks:

Note: Please follow the instruction in following training scripts to get the pretrained ckpt first.

  • Finetuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_newdata/UniTS_finetune_few_shot_newdata_pct20.sh
  • Prompt tuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_newdata/UniTS_prompt_tuning_few_shot_newdata_pct20.sh

3. Few-shot transfer learning on anomaly detection tasks:

  • Finetuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_anomaly_detection/UniTS_finetune_few_shot_anomaly_detection.sh
  • Prompt tuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_anomaly_detection/UniTS_prompt_tuning_few_shot_anomaly_detection.sh

4. Few-shot transfer learning on imputation tasks:

  • Finetuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_imputation/UniTS_finetune_few_shot_imputation_mask050.sh
  • Prompt tuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_imputation/UniTS_prompt_tuning_few_shot_imputation_mask050.sh

5. Zero-shot learning on new forecasting length:

# please set the pretrianed model path in the script.
bash ./scripts/zero_shot/UniTS_forecast_new_length_unify.sh

6. Zero-shot learning on new forecasting datasets:

# A special verison of UniTS with shared prompt/mask tokens needs to be trained for this setting.
bash ./scripts/zero_shot/UniTS_zeroshot_newdata.sh

Use UniTS on your own data.

UniTS is a highly flexible unified time series model, supporting tasks such as forecasting, classification, imputation, and anomaly detection with a single shared model and shared weights. We provide a Tutorial to assist you in using your own data with UniTS.

Pretrained weights

We provide the pretrained weights for models mentioned above in checkpoints.

Citation

@article{gao2024building,
  title={UniTS: Building a Unified Time Series Model},
  author={Gao, Shanghua and Koker, Teddy and Queen, Owen and Hartvigsen, Thomas and Tsiligkaridis, Theodoros and Zitnik, Marinka},
  journal={arXiv},
  url={https://arxiv.org/pdf/2403.00131.pdf},
  year={2024}
}

Acknowledgement

This codebase is built based on the Time-Series-Library. Thanks!

Disclaimer

DISTRIBUTION STATEMENT: Approved for public release. Distribution is unlimited.

This material is based upon work supported by the Under Secretary of Defense for Research and Engineering under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Under Secretary of Defense for Research and Engineering.

© 2024 Massachusetts Institute of Technology.

Subject to FAR52.227-11 Patent Rights - Ownership by the contractor (May 2014)

The software/firmware is provided to you on an As-Is basis

Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work.

units's People

Contributors

gasvn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

units's Issues

A question about fine-tuning

Hi, I recently conducted a few-shot fine-tuning on my private ECG classification training dataset and was surprised by the performance on the test dataset. This high performance occurred after just two epochs, regardless of whether I used an SSL pretrained model or not. My question is: does the model utilize the pretrained weights in both scenarios? Also, are some of the weights frozen during fine-tuning, contributing to the rapid fine-tuning?

Question about Model Configuration and Checkpoint Availability in Zero-Shot Setting

Hello,

Thanks for your great work! I have a couple of questions regarding the model configurations and checkpoint availability in the context of the UniTS_zeroshot_newdata.sh script.

1. Model Dimensionality in Zero-Shot Setting:
Why is the d_model parameter set to 64 for the zero-shot setting, while in the original multi-task pretraining setting, it is set to 128? Is there a specific reason behind this choice, or is it related to the performance or efficiency considerations?

2. Checkpoint Availability:
I greatly appreciate the series of checkpoints you've provided in Pretrained weights. However, is it possible to make the checkpoint in the zero-shot setting publicly available?

Thank you for your time and assistance.

Could you please explain the meaning of prompt token?

Hello! Thanks very much for giving such splendid idea and implemetation.
But I cannot understand the concrete meaning of prompt token inspite of explaing it in paper"for different data source and task", the difference I see is just the different prompt_num.
Thanks.

The initialization of Category Embedding

Hello:
This is excellent work! Thank you for sharing it. Can I confirm if the category embedding is randomly initialized and fully trainable during training? For zero-shot classification, I believe it is initialized and fixed for continuously incoming samples.

Additionally, have you experimented with irregular time series data? I think it is interesting to explore if UnitTS also performs well in MIMIC-III or Physionet2012 datasets.

When loading the model, _IncompatibleKeys appears.

I renamed units_x128_pretrain_checkpoint.pth to pretrain_ckpt.pth.
Then ran:
bash ./scripts/zero_shot/UniTS_forecast_new_length_unify.sh

Resulted in getting the following warnings:
_IncompatibleKeys(missing_keys=['module.prompt_tokens.ECL', 'module.prompt_tokens.ETTh1', 'module.prompt_tokens.Exchange', 'module.prompt_tokens.Traffic', 'module.prompt_tokens.Weather', 'module.mask_tokens.ECL', 'module.mask_tokens.ETTh1', 'module.mask_tokens.Exchange', 'module.mask_tokens.Traffic', 'module.mask_tokens.Weather', 'module.patch_embeddings.value_embedding.weight', 'module.position_embedding.pe', 'module.prompt2forecat.weights', 'module.prompt2forecat.bias', 'module.blocks.0.seq_att_block.norm1.weight', 'module.blocks.0.seq_att_block.norm1.bias', 'module.blocks.0.seq_att_block.attn_seq.qkv.weight', 'module.blocks.0.seq_att_block.attn_seq.proj.weight', 'module.blocks.0.seq_att_block.attn_seq.proj.bias', 'module.blocks.0.seq_att_block.ls1.gate.weight', 'module.blocks.0.seq_att_block.ls1.gate.bias', 'module.blocks.0.seq_att_block.proj.weight', 'module.blocks.0.seq_att_block.proj.bias', 'module.blocks.0.var_att_block.norm1.weight', 'module.blocks.0.var_att_block.norm1.bias', 'module.blocks.0.var_att_block.attn_var.qkv.weight', 'module.blocks.0.var_att_block.attn_var.proj.weight', 'module.blocks.0.var_att_block.attn_var.proj.bias', 'module.blocks.0.var_att_block.ls1.gate.weight', 'module.blocks.0.var_att_block.ls1.gate.bias', 'module.blocks.0.var_att_block.proj.weight', 'module.blocks.0.var_att_block.proj.bias', 'module.blocks.0.dynamic_mlp.norm2.weight', 'module.blocks.0.dynamic_mlp.norm2.bias', 'module.blocks.0.dynamic_mlp.mlp.fc1.weight', 'module.blocks.0.dynamic_mlp.mlp.fc1.bias', 'module.blocks.0.dynamic_mlp.mlp.seq_fc.weights', 'module.blocks.0.dynamic_mlp.mlp.seq_fc.bias', 'module.blocks.0.dynamic_mlp.mlp.prompt_fc.weights', 'module.blocks.0.dynamic_mlp.mlp.prompt_fc.bias', 'module.blocks.0.dynamic_mlp.mlp.fc2.weight', 'module.blocks.0.dynamic_mlp.mlp.fc2.bias', 'module.blocks.0.dynamic_mlp.ls2.gate.weight', 'module.blocks.0.dynamic_mlp.ls2.gate.bias', 'module.blocks.1.seq_att_block.norm1.weight', 'module.blocks.1.seq_att_block.norm1.bias', 'module.blocks.1.seq_att_block.attn_seq.qkv.weight', 'module.blocks.1.seq_att_block.attn_seq.proj.weight', 'module.blocks.1.seq_att_block.attn_seq.proj.bias', 'module.blocks.1.seq_att_block.ls1.gate.weight', 'module.blocks.1.seq_att_block.ls1.gate.bias', 'module.blocks.1.seq_att_block.proj.weight', 'module.blocks.1.seq_att_block.proj.bias', 'module.blocks.1.var_att_block.norm1.weight', 'module.blocks.1.var_att_block.norm1.bias', 'module.blocks.1.var_att_block.attn_var.qkv.weight', 'module.blocks.1.var_att_block.attn_var.proj.weight', 'module.blocks.1.var_att_block.attn_var.proj.bias', 'module.blocks.1.var_att_block.ls1.gate.weight', 'module.blocks.1.var_att_block.ls1.gate.bias', 'module.blocks.1.var_att_block.proj.weight', 'module.blocks.1.var_att_block.proj.bias', 'module.blocks.1.dynamic_mlp.norm2.weight', 'module.blocks.1.dynamic_mlp.norm2.bias', 'module.blocks.1.dynamic_mlp.mlp.fc1.weight', 'module.blocks.1.dynamic_mlp.mlp.fc1.bias', 'module.blocks.1.dynamic_mlp.mlp.seq_fc.weights', 'module.blocks.1.dynamic_mlp.mlp.seq_fc.bias', 'module.blocks.1.dynamic_mlp.mlp.prompt_fc.weights', 'module.blocks.1.dynamic_mlp.mlp.prompt_fc.bias', 'module.blocks.1.dynamic_mlp.mlp.fc2.weight', 'module.blocks.1.dynamic_mlp.mlp.fc2.bias', 'module.blocks.1.dynamic_mlp.ls2.gate.weight', 'module.blocks.1.dynamic_mlp.ls2.gate.bias', 'module.blocks.2.seq_att_block.norm1.weight', 'module.blocks.2.seq_att_block.norm1.bias', 'module.blocks.2.seq_att_block.attn_seq.qkv.weight', 'module.blocks.2.seq_att_block.attn_seq.proj.weight', 'module.blocks.2.seq_att_block.attn_seq.proj.bias', 'module.blocks.2.seq_att_block.ls1.gate.weight', 'module.blocks.2.seq_att_block.ls1.gate.bias', 'module.blocks.2.seq_att_block.proj.weight', 'module.blocks.2.seq_att_block.proj.bias', 'module.blocks.2.var_att_block.norm1.weight', 'module.blocks.2.var_att_block.norm1.bias', 'module.blocks.2.var_att_block.attn_var.qkv.weight', 'module.blocks.2.var_att_block.attn_var.proj.weight', 'module.blocks.2.var_att_block.attn_var.proj.bias', 'module.blocks.2.var_att_block.ls1.gate.weight', 'module.blocks.2.var_att_block.ls1.gate.bias', 'module.blocks.2.var_att_block.proj.weight', 'module.blocks.2.var_att_block.proj.bias', 'module.blocks.2.dynamic_mlp.norm2.weight', 'module.blocks.2.dynamic_mlp.norm2.bias', 'module.blocks.2.dynamic_mlp.mlp.fc1.weight', 'module.blocks.2.dynamic_mlp.mlp.fc1.bias', 'module.blocks.2.dynamic_mlp.mlp.seq_fc.weights', 'module.blocks.2.dynamic_mlp.mlp.seq_fc.bias', 'module.blocks.2.dynamic_mlp.mlp.prompt_fc.weights', 'module.blocks.2.dynamic_mlp.mlp.prompt_fc.bias', 'module.blocks.2.dynamic_mlp.mlp.fc2.weight', 'module.blocks.2.dynamic_mlp.mlp.fc2.bias', 'module.blocks.2.dynamic_mlp.ls2.gate.weight', 'module.blocks.2.dynamic_mlp.ls2.gate.bias', 'module.cls_head.proj_in.weight', 'module.cls_head.proj_in.bias', 'module.cls_head.cross_att.q.weight', 'module.cls_head.cross_att.kv.weight', 'module.cls_head.cross_att.proj.weight', 'module.cls_head.cross_att.proj.bias', 'module.cls_head.mlp.norm2.weight', 'module.cls_head.mlp.norm2.bias', 'module.cls_head.mlp.mlp.fc1.weight', 'module.cls_head.mlp.mlp.fc1.bias', 'module.cls_head.mlp.mlp.fc2.weight', 'module.cls_head.mlp.mlp.fc2.bias', 'module.cls_head.mlp.ls2.gate.weight', 'module.cls_head.mlp.ls2.gate.bias', 'module.forecast_head.proj_in.weight', 'module.forecast_head.proj_in.bias', 'module.forecast_head.mlp.fc1.weight', 'module.forecast_head.mlp.fc1.bias', 'module.forecast_head.mlp.fc2.weight', 'module.forecast_head.mlp.fc2.bias', 'module.forecast_head.proj_out.weight', 'module.forecast_head.proj_out.bias', 'module.forecast_head.pos_proj.weights', 'module.forecast_head.pos_proj.bias'], unexpected_keys=['student', 'optimizer', 'epoch', 'args', 'ibot_loss'])

Does this mean the models are incompatible?

adjustment for anomaly detection

Hey!
Thanks for making the code available. I was wondering about the adjustment made for the anomaly detection. Is this done for all the competing methods? And is there some documentation for the datasets that explains that part of the evaluation?

A question about datasets.

Awesome work!
I am interesting for your work! But I have some problems for datasets.

  1. What the datasets are you used? UCR and Monash? anything else? Is table 7 the full data set listed?
  2. Do you have a dataset that uses anomaly detection? The paper mentions using 38 datasets, and Table 7 is supposed to list all of them, so I don't think you should be using the anomaly detection dataset.
  3. I have a doubt for the dataset, why not use more datasets then. I think more datasets will help the model to improve the performance.
    Finally, I desire to know how many GPU were used for training.

Thanks for your contributions!

Should requirements.txt be updated including more packages?

Some packages used by the model are not listed in requirements.txt, for example, gluonts, wandb, timm. Is it possible to know how the requirements.txt is derived? I guess perhaps there are some reasons for not including those deliberately? And is it better to include them explicitly?

Validation set

It seems that you use test data as your validation set. Is it true? If it is the case, could you please what is the reason?

Using UniTS on Unlabeled Data for Anomaly Detection Task

Hi,
Thank you for your contributions.
I am currently working on an anomaly detection task on a multivariate time series dataset, but the issue is that this dataset currently does not have labels. I want to know if UniTS can be directly applied to this dataset, and if I can extract a portion of the dataset to fine-tune it.

Thank you!

Typo in README

Hi,

Thanks for the GitHub project, I noticed a minor typo in README, could you please fix it whenever you get a chance?

In

3. Train and evaluate model
->
2. Few-shot transfer learning on new forecasting and classification tasks:

Finetuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_newdata/UniTS_finetune_few_shot_newdata_pct20.sh
Prompt tuning
# please set the pretrianed model path in the script.
bash ./scripts/few_shot_newdata/UniTS_finetune_few_shot_newdata_pct20.sh

Both Finetuning and Prompt tuning have the same bash command ..

gluonts problem

I have installed GluonTS, but when I run the code with the command you provided, it shows ModuleNotFoundError: No module named 'gluonts'. I need your help.
image

About anomaly detection

Hello, thank you very much for providing such an excellent idea and implementation. However, the performance of my anomaly detection run has not reached the level stated in your paper. Could you please offer me some suggestions? The dataset I'm currently using is the SMD dataset. My F1 score is 81.81, which is quite a bit lower than the 88.09 mentioned in your paper. If I want to achieve results similar to yours, what parts of the code should I adjust? I would greatly appreciate any advice you can give.
The experimental environment is Ubuntu 22.04.3 LTS operating system, with an Intel® Xeon® CPU E5-2609 v4 @ 1.70GHz, two NVIDIA GeForce RTX 4090 GPUs, and 173GB of RAM.

Question about DyLinear block

Thanks for the great work!
I've read your paper and got confused in DyLinear blocks and ask here to get some clarifications.

Q, Is DyLinear block applied to each variables independently? In equation (2), I believe that $z_s$ is sized $l_s$ * v * d and WInterp is sized $w_i * w_o$. How are WInterp$z_s$ calculated and what is exactly lout ? Also if it is calculated as independent, why do we need interpolation? Isn't d fixed among different time series?

Thank you

How is the internal review going? What's the status?

I see the README.md was updated with the below on 2024-03-04 in 762fafc , which also removed a large number of files (already mentioned in #5)

The code is currently undergoing an internal administrative review and is scheduled for release within this week.

Given it's been about 2 weeks, just curious what the status of the internal review is, and if there are plans to revert that commit, or update the repo with the latest information/code.

Most recent commit removed all files?

Hey, I was hoping to evaluate this after reading about it in the news, the most recent commit removed all the files, do I just step back to the commit with files/will they be the correct version?

IncompatibleKeys Error when load pre-trained model to do the fine-tuning.

Nice work and I have a question here:

I am trying to pre-train and finetune a model on my own datasets. However, some warnings were raised when loading the pre-trained model during finetuning:

loading pretrained model: checkpoints/ALL_task_UniTS_pretrain_x64_bs1024_UniTS_All_dm64_el3_Exp_0/pretrain_checkpoint.pth
_IncompatibleKeys(missing_keys=['category_tokens.CLS_dataset1', 'category_tokens.CLS_dataset2'], unexpected_keys=['pretrain_head.proj_in.weight', 'pretrain_head.proj_in.bias', 'pretrain_head.mlp.fc1.weight', 'pretrain_head.mlp.fc1.bias', 'pretrain_head.mlp.fc2.weight', 'pretrain_head.mlp.fc2.bias', 'pretrain_head.proj_out.weight', 'pretrain_head.proj_out.bias', 'pretrain_head.pos_proj.weights', 'pretrain_head.pos_proj.bias'])

Is everything correct here?

Thank you for your help!

Using UniTS for new tasks

Great repository! I would like to be able to use the code to pretrain/finetune using custom tasks that do not quite fit into any of the forecast, imputation, anomaly detection or classification tasks. Is it possible (without huge changes to the code) to train/test with other tasks?
Specifically, I would like to be able to do regression, i.e., predict one or more continuous values, that are not future values of the time series itself.
I would also like to be able to to have the output be a sequence of varying length (same length as input sequence), where each element is either a set of continuous values, or a categorical value.

I would greatly appreciate some guidance in this.
Viktor

Hello! Asking about Inference.

Thanks for innovating such a splendid work!I am enjoying watching your paper and code.
My question is:
After training and saving the model, how can I inference ?

RuntimeError: Distributed package doesn't have NCCL built in

Hello,

I tried to run UniTS_supervised with all default settings just for an initial test, but I got this error below. It seems that Torch is missing something, but I didn't see NCCL mentioned anywhere, I just installed everything in requirements.txt. I tried to install NCCL, but it seems to me that it's Linux only. Do you have an idea how to solve this on Windows 10?

C:\Users\comp\UniTS>bash ./scripts/supervised_learning/UniTS_supervised.sh
NOTE: Redirects are currently not supported in Windows or MacOs.
[W ..\torch\csrc\distributed\c10d\socket.cpp:601] [c10d] The client socket has failed to connect to [comp]:4223 (system error: 10049 - A kÚrt cÝm nem ÚrvÚnyes a hozzß tartozˇ k÷rnyezetben.).
[W ..\torch\csrc\distributed\c10d\socket.cpp:601] [c10d] The client socket has failed to connect to [comp]:4223 (system error: 10049 - A kÚrt cÝm nem ÚrvÚnyes a hozzß tartozˇ k÷rnyezetben.).
C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\gluonts\json.py:101: UserWarning: Using json-module for json-handling. Consider installing one of orjson, ujson to speed up serialization and deserialization.
warnings.warn(
[W ..\torch\csrc\distributed\c10d\socket.cpp:601] [c10d] The client socket has failed to connect to [comp]:4223 (system error: 10049 - A kÚrt cÝm nem ÚrvÚnyes a hozzß tartozˇ k÷rnyezetben.).
[W ..\torch\csrc\distributed\c10d\socket.cpp:601] [c10d] The client socket has failed to connect to [comp]:4223 (system error: 10049 - A kÚrt cÝm nem ÚrvÚnyes a hozzß tartozˇ k÷rnyezetben.).
Traceback (most recent call last):
File "C:\Users\comp\UniTS\run.py", line 114, in
init_distributed_mode(args)
File "C:\Users\comp\UniTS\utils\ddp.py", line 31, in init_distributed_mode
dist.init_process_group(
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\distributed\distributed_c10d.py", line 907, in init_process_group
default_pg = _new_process_group_helper(
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\distributed\distributed_c10d.py", line 1013, in _new_process_group_helper
raise RuntimeError("Distributed package doesn't have NCCL " "built in")
RuntimeError: Distributed package doesn't have NCCL built in
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 85344) of binary: C:\Users\comp\AppData\Local\Programs\Python\Python310\python.exe
Traceback (most recent call last):
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return run_code(code, main_globals, None,
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\Scripts\torchrun.exe_main
.py", line 7, in
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\distributed\elastic\multiprocessing\errors_init
.py", line 346, in wrapper
return f(*args, **kwargs)
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\distributed\run.py", line 794, in main
run(args)
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\distributed\run.py", line 785, in run
elastic_launch(
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\distributed\launcher\api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "C:\Users\comp\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\distributed\launcher\api.py", line 250, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

run.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2024-03-30_11:31:28
host : comp
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 85344)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

the meaning of prompt_num

Could you please explain the meaning of prompt_num?Do all pre trained models satisfy prompt_num=10?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.