Giter VIP home page Giter VIP logo

koopa's Introduction

Koopa

This is the official codebase for the paper: Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors, NeurIPS 2023.

Updates

🚩 News (2023.10) Koopa has been included in [Time-Series-Library].

Introduction

Koopa is a lightweight, MLP-based, and theory-inspired model for efficient time series forecasting.

  • Compared with the advanced but painstakingly trained deep forecasters, Koopa achieves state-of-the-art performance while saving 77.3% training time and 76.0% memory footprint.

  • Focus on portraying ubiquitous non-stationary time series, Koopa shows enhanced model capacity empowered by the modern Koopman theory that naturally addresses the nonlinear evolution of real-world time series.

  • Koopa differs from the canonical Koopman Autoencoder without the reconstruction loss function to achieve end-to-end predictive training.

Discussions

There are already several discussions about our paper, we appreciate a lot for their valuable comments and efforts: [Official], [Openreview], [Zhihu].

Preparation

  1. Install Pytorch (>=1.12.0) and other necessary dependencies.
pip install -r requirements.txt
  1. All the six benchmark datasets can be obtained from Google Drive or Tsinghua Cloud.

Training scripts

We provide the Koopa experiment scripts and hyperparameters of all benchmark datasets under the folder ./scripts.

bash ./scripts/ECL_script/Koopa.sh
bash ./scripts/Traffic_script/Koopa.sh
bash ./scripts/Weather_script/Koopa.sh
bash ./scripts/ILI_script/Koopa.sh
bash ./scripts/Exchange_script/Koopa.sh
bash ./scripts/ETT_script/Koopa.sh

Applicable for Rolling Forecast

  • By adapting the operator on the incoming time series during rolling forecast, the proposed model can achieve more accurate performance via adapting to continuous distribution shift.

  • The naïve implementation of operator adaptation is based on the DMD algorithm. We propose an iterative algorithm with reduced complexity. The details can be found in the Appendix of our paper.

  • We also provide a tutorial notebook for a better understanding of this scenario. See operator_adaptation.ipynb for the details.

Citation

If you find this repo useful, please cite our paper.

@article{liu2023koopa,
  title={Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors},
  author={Liu, Yong and Li, Chenyu and Wang, Jianmin and Long, Mingsheng},
  journal={arXiv preprint arXiv:2305.18803},
  year={2023}
}

Contact

If you have any questions or want to use the code, please contact:

koopa's People

Contributors

eltociear avatar lichenyu20 avatar wenweithu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

koopa's Issues

关于算子是啥的问题

作者你好,请问算子到底是啥,是如何得到的呢?通过查看源码发现时不变部分的算子是可学习参数的SVD的左右奇异矩阵相乘,时变部分的算子是可学习参数的最小二乘法的解,请问为什么是将这两个当成算子?

关于时间序列非平稳的疑问

您好,我对于时间序列非平稳带来的问题一直存在一些疑惑,时间序列非平稳是指序列的统计信息随时间变化,可是这到底是如何影响神经网络模型的预测呢?神经网络预测输入的是历史序列,输出的是未来序列,只要这个映射关系是存在一定规律的,那么神经网络应该就能够在一定程度上学到这种关系,序列的非平稳是如何影响这种映射关系的呢?为什么平稳的序列就一定更好呢?(白噪声也是一种平稳序列,可是却完全无法预测。

Prediction on entire dataset

Hello,

Thank you for this project and contribution.

Currently, the model predicts only one batch given a CSV file. Is there a convinient way to modify it so that it can predict on the entire dataset, similar to how predictions are made on the test set?

FFT - BUG

Dear author,
In your code, I found a mistake where the code does not match the paper, like picture
21f8e6ba87bfcbd64e0d162d0521679

Therefore, after I fix this bug, my mse and mae decline a little
image

Looking forward to your reply

能适用于短时高维的时间序列吗

利用koopman理论有限维逼近无限维,可以处理高维度的时间序列,看到作者提出的算子适应适合长期预测,不知道是否适用于短期预测呢?

在vscode运行bash语句报错

RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.请问这是怎么回事,我在linux里面就可以正常执行代码

run problem

RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

when I run the run.py, Program encountered operational issues,so how to resolve it?

loss反向传播报错

torch版本:1.10.0+cu113
报错信息:NotImplementedError: the derivative for 'linalg_lstsq' is not implemented.
出错位置:

self.K = torch.linalg.lstsq(x, y).solution # B E E

运行命令:python3 run.py --dynamic_dim=256 --hidden_dim=512 --seq_len=32 --seg_len=16 --hidden_layers=3 --num_blocks=3 --des='Exp' --model=Koopa --cache_dir=rawData_5M_sl32 --debug --learning_rate=1e-3 --test_bs=2048
image.png

Why does the time-variant Koopman operator predict an extra step?

I'm confused about the code in the model part, that is, line 74 - line 98 in models/Koopa.py.

Why isn't the variable z_pred get from one_step_forward() function put into the list z_preds?

There is a time interval between variable x and variable y, and the least square method is used to calculate $K_{\mathrm{var}}$ . Next, the value z_pred is calculated with the last segment of the look-back window and $K_{\mathrm{var}}$ as input, corresponding to #L85. This z_pred variable seems not belong to the look-back window, but the predicting window. So why isn't this z_pred variable in the z_preds ?

C70CDA73AB01AEA64D6BEB8F98336274

I try to change the code snippets of line 94 - line 95 to this:

        z_preds = []
        z_preds.append(z_pred)
        for i in range(1, pred_len):

And the result gets worse, well, that makes me confused.

dynamics research

作者您好,这篇文章提到了动力学系统,时间序列的不平稳和非线性研究方向贵课题组后续还会有涉及吗?

Degree of Variation Experiment

Hello Author,I am curious about two component features after Fourier Filter, can you tell me the details or show me the code how you did the visualization study on features and how can I calculate Degree of Variation

关于标准化操作的疑问

作者您好,我阅读代码的时候看到 dataloader里已经使用StandardScaler()做了处理,为什么在Model的forward里面还要做Stationarization呢?

A RuntimeError of "loss.backward()"

hello,when I run your code,its has a error, when runnging at "loss.backward()", its has a mistake :"RuntimeError: the derivative for 'linalg_lstsq' is not implemented." How to resolve it,can you help me? what's the pytorch version?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.