Giter VIP home page Giter VIP logo

alphanetv3's Introduction

AlphaNet

unittest Congyuwang publish

A Recurrent Neural Network For Predicting Stock Prices

AlphaNetV2

Below is the structure of AlphaNetV2

input: (batch_size, history time steps, features)

              stride = 5
input -> expand features -> BN -> LSTM -> BN -> Dense(linear)

AlphaNetV3

Below is the structure of AlphaNetV3

input: (batch_size, history time steps, features)

                 stride = 5
        +-> expand features -> BN -> GRU -> BN -+
input --|       stride = 10                     |- concat -> Dense(linear)
        +-> expand features -> BN -> GRU -> BN -+

Installation

Either clone this repository or just use pypi: pip install alphanet.

The pypi project is here: alphanet.

Example

Step 0: import alphanet

from alphanet import AlphaNetV3, load_model
from alphanet.data import TrainValData, TimeSeriesData
from alphanet.metrics import UpDownAccuracy

Step 1: build data

# read data
df = pd.read_csv("some_data.csv")

# compute label (future return)
df_future_return = here_you_compute_it_by_your_self
df = df_future_return.merge(df,
                            how="inner",
                            left_on=["date", "security_code"],
                            right_on=["date", "security_code"])

# create an empty list
stock_data_list = []

# put each stock into the list using TimeSeriesData() class
security_codes = df["security_code"].unique()
for code in security_codes:
    table_part = df.loc[df["security_code"] == code, :]
    stock_data_list.append(TimeSeriesData(dates=table_part["date"].values,                   # date column
                                          data=table_part.iloc[:, 3:].values,                # data columns
                                          labels=table_part["future_10_cum_return"].values)) # label column

# put stock list into TrainValData() class, specify dataset lengths
train_val_data = TrainValData(time_series_list=stock_data_list,
                              train_length=1200,   # 1200 trading days for training
                              validate_length=150, # 150 trading days for validation
                              history_length=30,   # each input contains 30 days of history
                              sample_step=2,       # jump to days forward for each sampling
                              train_val_gap=10     # leave a 10-day gap between training and validation

Step 2: get datasets from desired period

# get one training period that start from 20110131
train, val, dates_info = train_val_data.get(20110131, order="by_date")
print(dates_info)

Step 3: compile the model and start training

# get an AlphaNetV3 instance
model = AlphaNetV3(l2=0.001, dropout=0.0)

# you may use UpDownAccuracy() here to evaluate performance
model.compile(metrics=[tf.keras.metrics.RootMeanSquaredError(),
                       UpDownAccuracy()]

# train
model.fit(train.batch(500).cache(),
          validation_data=val.batch(500).cache(),
          epochs=100)

Step 4: save and load

saving

# save model by save method
model.save("path_to_your_model")

# or just save weights
model.save_weights("path_to_your_weights")

loading

# load entire model using load_model() from alphanet module
model = load_model("path_to_your_model")

# only load weights by first creating a model instance
model = AlphaNetV3(l2=0.001, dropout=0.0)
model.load_weights("path_to_your_weights")

Note: only alphanet.load_model(filename) recognizes custom UpDownAccuracy. If you do not use UpDownAccuracy, you can also use tf.keras.models.load_model(filename).

Documentation

For detailed documentation, go to alphanet documentation.

For implementation details, go to alphanet source folder.

One Little Caveat

The model expands features quadratically. So, if you have 5 features, it will be expanded to more than 50 features (for AlphaNetV3), and if you have 10 features, it will be expanded to more than 200 features. Therefore, do not put too many features inside.

One More Note

alphanet.datamodule is completely independent from alphanet module, and can be a useful tool for training any timeseries neural network.

alphanetv3's People

Contributors

congyuwang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

alphanetv3's Issues

请问模型训练好之后怎么应用?

十分感谢分享!我用自己的数据训练了一个模型,请问怎么应用训练好的模型?(即测试集应用)
即:如何给模型输入测试集序列,输出return的label?

我尝试model.predict(test_data),其中test_data是一个TimeSeriesData,但似乎并不能成功。

` # load entire model using load_model() from alphanet module
model = load_model("model.bt")

# only load weights by first creating a model instance
model = AlphaNetV3(l2=0.001, dropout=0.0)
model.load_weights("weights.bt")
test_data=TimeSeriesData(
    dates=test_df["int_date"].values,                   # date column            
    data=test_df[metrics_std].values,                # data columns
    labels=test_df["future_return"].values # label column
)
output=model.predict(test_data.data)
print(output)`

run "python tests.py" error

======================================================================
ERROR: test_save_model (main.TestAlphaNetV2)

Traceback (most recent call last):
File "c:\Users\Administrator\Desktop\AlphaNetV3\tests\tests.py", line 172, in test_save_model
alpha_net.save("./.test_alpha_net_save/model")
File "D:\base\path\py3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "D:\base\path\py3\lib\site-packages\tensorflow\python\saved_model\function_serialization.py", line 27, in _serialize_function_spec
raise NotImplementedError(
NotImplementedError: Cannot serialize a method function without a named 'self' argument.

======================================================================
ERROR: test_save_model (main.TestAlphaNetV3)

Traceback (most recent call last):
File "c:\Users\Administrator\Desktop\AlphaNetV3\tests\tests.py", line 240, in test_save_model
alpha_net_v3.save("./.test_alpha_net_save/model")
File "D:\base\path\py3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "D:\base\path\py3\lib\site-packages\tensorflow\python\saved_model\function_serialization.py", line 27, in _serialize_function_spec
raise NotImplementedError(
NotImplementedError: Cannot serialize a method function without a named 'self' argument.

======================================================================
ERROR: test_save_model (main.TestAlphaNetV4)

Traceback (most recent call last):
File "c:\Users\Administrator\Desktop\AlphaNetV3\tests\tests.py", line 308, in test_save_model
alpha_net_v4.save("./.test_alpha_net_save/model")
File "D:\base\path\py3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "D:\base\path\py3\lib\site-packages\tensorflow\python\saved_model\function_serialization.py", line 27, in _serialize_function_spec
raise NotImplementedError(
NotImplementedError: Cannot serialize a method function without a named 'self' argument.

======================================================================
FAIL: test_last_batch_of_training_dataset (main.TestDataNormalizationModule)

Traceback (most recent call last):
File "c:\Users\Administrator\Desktop\AlphaNetV3\tests\tests.py", line 504, in test_last_batch_of_training_dataset
self.assertTrue(is_all_close(
AssertionError: False is not true : last batch of training data (start 20116595): failure


Ran 31 tests in 64.568s

FAILED (failures=1, errors=3)

===========================
python 3.9
torch 1.12.1
tensorflow 2.9.2
Pillow 9.2.0
pandas 1.4.4
numpy 1.23.2
numba 0.56.2
keras 2.9.0

Delete this useless function

AlphaNetV3/alphanet/data.py

Lines 351 to 364 in cdaeaa5

def __get_file_names__(dates_info, order):
"""
:return: train_x_file, train_y_file, val_x_file, val_y_file
"""
train_start_date = dates_info["training"]["start_date"]
train_end_date = dates_info["training"]["end_date"]
val_start_date = dates_info["validation"]["start_date"]
val_end_date = dates_info["validation"]["end_date"]
train_x_file = f"train_x_{train_start_date}_{train_end_date}_{order}"
train_y_file = f"train_y_{train_start_date}_{train_end_date}_{order}"
val_x_file = f"val_x_{val_start_date}_{val_end_date}_{order}"
val_y_file = f"val_y_{val_start_date}_{val_end_date}_{order}"
json_file = f"{train_start_date}_{val_end_date}.json"
return train_x_file, train_y_file, val_x_file, val_y_file, json_file

Hello blogger, I would like to ask if you can provide the requirements.txt file?

Hello blogger, I would like to ask if you can provide the requirements.txt file? Or tell me the version of keras or tensorflow?
Because the APIs of different versions of tensorflow are particularly confusing, I want to use your code to reproduce the content in the series of Huatai Financial Engineering articles. Thank you again for your contribution!

data_producer

请问这个 data_producer 是在哪里实现的?

UpDownAccuracy

你好,最近正在用pytorch实现AlphaNet,想请问下UpDownAccuracy的设计思路。感谢🙏

我还是想询问一下predict所需要的数据应该如何传入?

作者你好,很抱歉再次打扰。我看了之前的issues,也有问到如何使用model.predict。我尝试了一下,发现还是有一些问题。故来询问。
我的demo例子是假设我现在有000723.SZ的三十条数据,我需要如何导入测试?
image
我使用你说的两种方法,但是发现都有报错。
能否麻烦你帮忙给一个使用小例子?

关于数据预处理的问题

作者你好,我想请教一下,该份数据中的所有包含nan值的sample是直接扔掉不加入训练中吗

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.