iminders / tbase Goto Github PK
View Code? Open in Web Editor NEWThis repo was moved to: https://github.com/tradingAI/tbase
License: MIT License
tbase's People
Contributors
Stargazers
Watchers
tbase's Issues
增加一个BuyAndHold Agent 作为策略参考基准
增加一个BuyAndHold Agent 作为策略参考基准:
- 个股
- 指数
Ubuntu GPU 环境下多进程运行时内存增长问题
系统环境
Ubuntu 18.04
GPU: GTX 1080
Memory: 8G
运行命令： python3 -m tbase.run --num_env 2
错误信息：
Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/ubuntu/gopath/src/github.com/iminders/tbase/tbase/run.py", line 56, in <module> main() File "/home/ubuntu/gopath/src/github.com/iminders/tbase/tbase/run.py", line 50, in main agent.learn() File "/home/ubuntu/gopath/src/github.com/iminders/tbase/tbase/agents/ddpg/agent.py", line 152, in learn sample_size=self.args.sample_size) File "/home/ubuntu/gopath/src/github.com/iminders/tbase/tbase/agents/ddpg/agent.py", line 63, in explore worker.start() File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in __init__ super().__init__(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 59, in _launch cmd, self._fds) File "/usr/lib/python3.6/multiprocessing/util.py", line 417, in spawnv_passfds False, False, None) OSError: [Errno 12] Cannot allocate memory Segmentation fault (core dumped)
windows anaconda Pytorch EOFError: Ran out of input when num_workers>0
https://discuss.pytorch.org/t/pytorch-windows-eoferror-ran-out-of-input-when-num-workers-0/25918
solved the issue by changing protocol=None in pythons multiprocessing/reduction.py to 4.
class ForkingPickler(pickle.Pickler):
'''Pickler subclass used by multiprocessing.'''
_extra_reducers = {}
_copyreg_dispatch_table = copyreg.dispatch_table
def __init__(self, *args):
super().__init__(*args)
self.dispatch_table = self._copyreg_dispatch_table.copy()
self.dispatch_table.update(self._extra_reducers)
@classmethod
def register(cls, type, reduce):
'''Register a reduce function for a type.'''
cls._extra_reducers[type] = reduce
@classmethod
def dumps(cls, obj, protocol=4):
buf = io.BytesIO()
cls(buf, protocol).dump(obj)
return buf.getbuffer()
loads = pickle.loads
register = ForkingPickler.register
def dump(obj, file, protocol=4):
'''Replacement for pickle.dump() using ForkingPickler.'''
ForkingPickler(file, protocol).dump(obj)
portfolio抖动现象
可能的原因：当模型在最优解附近时，再使用梯度下降或其他的基于梯度的最优化方法求解时，很有可能直接跳过最优解，从而导致交易失败。
例如：000001.SZ(平安银行), 2020.02.27日最高价: 15.15, 最低价:14.89
如果模型这时的输出决策：15.15卖出，14.89买入，再往梯度方向调整，很有可能就会出现卖出价格过高，买进价格过低，无法达成交易，最终使收益率阶梯式下降
这种现象是否是合理？
build time out
docker-compose up timeout
liuwen@pro tbase % docker-compose up
Starting tbase_bazel_1 ... done
Attaching to tbase_bazel_1
bazel_1 | Loading:
bazel_1 | Loading: 0 packages loaded
bazel_1 | Loading: 0 packages loaded
bazel_1 | Analyzing: 7 targets (8 packages loaded, 0 targets configured)
bazel_1 | Analyzing: 7 targets (26 packages loaded, 100 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | Analyzing: 7 targets (28 packages loaded, 157 targets configured)
bazel_1 | INFO: Call stack for the definition of repository 'remotejdk11_linux' which is a http_archive (rule definition at /root/cache/bazel/c50f8f05fa35fa56205e52
507df67c8f/external/bazel_tools/tools/build_defs/repo/http.bzl:292:16):
bazel_1 | - /root/cache/bazel/c50f8f05fa35fa56205e52507df67c8f/external/bazel_tools/tools/build_defs/repo/utils.bzl:205:9
bazel_1 | - /root/cache/bazel/c50f8f05fa35fa56205e52507df67c8f/external/bazel_tools/tools/build_defs/repo/utils.bzl:205:9
bazel_1 | - /DEFAULT.WORKSPACE.SUFFIX:216:1
bazel_1 | WARNING: Download from https://mirror.bazel.build/openjdk/azul-zulu11.2.3-jdk11.0.1/zulu11.2.3-jdk11.0.1-linux_x64.tar.gz failed: class javax.net.ssl.SSLP
rotocolException Read timed out
bazel_1 | ERROR: An error occurred during the fetch of repository 'remotejdk11_linux':
bazel_1 | java.io.IOException: Error downloading [https://mirror.bazel.build/openjdk/azul-zulu11.2.3-jdk11.0.1/zulu11.2.3-jdk11.0.1-linux_x64.tar.gz] to /root/ca
che/bazel/c50f8f05fa35fa56205e52507df67c8f/external/remotejdk11_linux/zulu11.2.3-jdk11.0.1-linux_x64.tar.gz: Read timed out
bazel_1 | ERROR: /root/cache/bazel/c50f8f05fa35fa56205e52507df67c8f/external/bazel_tools/tools/jdk/BUILD:494:1: @bazel_tools//tools/jdk:remote_jdk11 depends on @rem
otejdk11_linux//:jdk in repository @remotejdk11_linux which failed to fetch. no such package '@remotejdk11_linux//': java.io.IOException: Error downloading [https://
mirror.bazel.build/openjdk/azul-zulu11.2.3-jdk11.0.1/zulu11.2.3-jdk11.0.1-linux_x64.tar.gz] to /root/cache/bazel/c50f8f05fa35fa56205e52507df67c8f/external/remotejdk1
1_linux/zulu11.2.3-jdk11.0.1-linux_x64.tar.gz: Read timed out
bazel_1 | ERROR: Analysis of target '//tbase/common:cmd_util_test' failed; build aborted: Analysis failed
gpu支持
多进程gpu支持, 加带训练
相同的参数，由于随机过程导致的差异常大
- 设置随机数种子，使模型训练可复现？
- 由于撮合成交机制，微小的价格差异，导致行为差别非常大。微小的随机差异，可能使portfolio值相差非常大，容易形成非常剧烈的抖动现象， 如何避免这种情况？
这两种方法可行性探索，如果可行则更新到repo中 @iminders
添加Model Graph到tensorboard中
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.