happydasch / btconfig Goto Github PK

View Code? Open in Web Editor NEW

37.0 4.0 21.0 316 KB

btconfig provides a simple way to initialize a strategy using config files with additional features.

Python 100.00%

backtrader plotting live trading stocks

btconfig's People

Contributors

Stargazers

Watchers

Forkers

ultra1971 hectordevone ajmalas reywalter cz9874308 webclinic017 ilikerolls juanparker1 huuquynh terrypang89 forex-trading-bot-engineering marcellinamichie291 raptoravis alihaskar

btconfig's Issues

Got error when using multiprocessing for multi assets/instruments

Hi,

btconfig runs well with single strategy and single asset/instrument, both backtest and live.

But when I tried to use with multiple instruments by define some json files then put in a folder, and using multiprocessing like this:

from __future__ import division, absolute_import, print_function

import os
import traceback
import btconfig
import multiprocessing


os.environ['BOKEH_ALLOW_WS_ORIGIN'] = '*'


if __name__ == '__main__':

    try:
        path_to_json = './codes'
        stocks = [os.path.join(path_to_json, pos_json) for pos_json in os.listdir(path_to_json) if pos_json.endswith('.json')]

        with multiprocessing.Pool() as pool:
            pool.map(btconfig.run_live, stocks)

    except Exception as err:
        pool.close()
        pool.join()
        traceback.print_tb(err.__traceback__)

It got a strange error, while run_live will not return until broken or finished:

  File "runlive.py", line 19, in <module>
    pool.map(btconfig.run_live, stocks)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value

I tried both plotting config in json to enable/disable live-view, but got the same error:

    "_live": {
        "common": {
            "create_plot": true,
            "create_report": false
        },

        "cerebro": {
            "quicknotify": true
        }
    },

and

    "_live": {
        "common": {
            "create_plot": false,
            "create_report": false
        },

        "cerebro": {
            "quicknotify": true
        }
    },

Please help me! I don't have enough experience in multiprocessing programming.

Results backtest missing when using OandaV2 datas and stores

After your swift reply on the previous issue I got directly into having the OandaV2 to work. After some puzzling the feed seems to work (as the data from oanda is being stored at the data folder), but the trades aren't displayed.

From what I understand is that both the datas and stores objects should be defined where stores contain the credentials.

After I run run_backtest.py all seems to be fine except No PnL, empty report message and the time of execution is very low.

Do you have an explanation or direction how to solve this? I added the run_backtest.py and the config.json with it.

run_backtest.py

import btconfig
from envelope_strategy import EnvelopeStrategy

btconfig.PATH_COMMINFO.extend(['btoandav20.commissions'])
btconfig.PATH_STORE.extend(['btoandav20.stores'])

if __name__ == '__main__':
    res = btconfig.run(btconfig.MODE_BACKTEST, "config.json")

config.json

{
    "common": {
        "strategy": "EnvelopeStrategy",
        "timezone": "Europe/Amsterdam",
        "create_plot": true,
        "create_tearsheet": true,
        "create_log": true,
        "add_observer": true,
        "add_analyzer": true,
        "optimizer_iterations": 100,
        "optimizer_exceptions": true,
        "optimizer": "HillClimbingOptimizer"
    },
    "logging": {
        "log_to_console": true,
        "log_to_file": true
    },
    "feeds": {
        "primary": ["Minutes", 5, "add", {}]
    },
    "datas": {
        "ident_oanda": {
            "store": "oanda",
            "dataname": "SPX500_USD",
            "granularity": ["Minutes", 5],
            "backfill_days": null,
            "params": {
                "historical": true
            },
            "for": ["primary"]
        },
        "ident_oanda_downloader": {
            "classname": "OandaV20Dataloader",
            "store": "oanda",
            "dataname": "SPX500_USD",
            "granularity": ["Minutes", 5],
            "backfill_days": null,
            "fromdate": "2022-05-01 00:00:00",
            "todate": null,
            "params": {
                "adjstarttime": true,
                "useRTH": false
            },
            "for": ["primary"]
        }
    },
    "stores": {
        "oanda": {
            "classname": "OandaV20Store",
            "params": {
                "token": "0xxxxx4",
                "account": "1xx-xxx-xxxxxxxx-xx1",
                "practice": true,
                "stream_timeout": 5,
                "poll_timeout": 5,
                "notif_transactions": true
            }
        }
    },
    "plot": {
        "volume": true,
        "bar_dist": 0.0003,
        "style": "bar"
    },
    "optimize": {
        "revert_pos": ["list", [true, false]],
        "period1": ["range", 2, 30, 2],
        "period2": ["range", 2, 30, 2]
    },
    "strategy": {
        "EnvelopeStrategy": {
            "period1": 20,
            "period2": 22,
            "revert_pos": true
        }
    },
    "_backtest": {
        "common": {
            "cash": 10000,
            "create_plot": true,
            "create_report": true
        },
        "cerebro": {
            "preload": true,
            "runonce": true
        }
    },
    "_optimize": {
        "common": {
            "create_log": true
        }
    }
}

plotting default port

the plotting default port is 80, which requires extra privilege, please set to 8080

Exception: Strategy SimpleStrategy not found on optimize modes simple strategy

While running the simple demo, run_backtest.py works fine, but both run_optimize.py and run_optimizegenetic.py runs into a similar issue, Exception: Strategy SimpleStrategy not found.

I fail to understand why this would occurs, the lines of code is around strategy.py where in all cases the exact same information is being dispatched:

        stratname = commoncfg.get('strategy', None)
        all_classes = get_classes(self._instance.PATH_STRATEGY)
        if stratname not in all_classes:
            raise Exception(f'Strategy {stratname} not found')

stratname = SimpleStrategy
all_classes = {'SimpleStrategy': <class 'simple_strategy.SimpleStrategy'>}
, but in both optimize modes there it will raise the error below:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/bjornmooijekind/Projects/btconfig/demo/simple/run_optimize.py", line 4, in <module>
    res = btconfig.run(btconfig.MODE_OPTIMIZE, "config.json")
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/btconfig-0.2.5-py3.8.egg/btconfig/__init__.py", line 776, in run
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/btconfig-0.2.5-py3.8.egg/btconfig/__init__.py", line 512, in run
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/btconfig-0.2.5-py3.8.egg/btconfig/__init__.py", line 461, in _setup
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/btconfig-0.2.5-py3.8.egg/btconfig/parts/strategy.py", line 64, in setup
Exception: Strategy SimpleStrategy not found

Threads and btconfig

When using optimization using multiple CPU's backtrader breaks because the instance in the threaded execution of btconfig is not initialized. Workaround for now is setting maxcpus = 1 for now.

AttributeError while running demo

OS: Ubuntu 21.04
Python: 3.9.6
backtrader: origin/master
btconfig: origin/master

python run_backtest.py

Traceback (most recent call last):
File "/home/mep/trade/pyht/btconfig/demo/simple/run_backtest.py", line 16, in
res = btconfig.run(btconfig.MODE_BACKTEST, "config.json")
File "/home/mep/trade/pyht/env/lib/python3.9/site-packages/btconfig-0.2.0-py3.9.egg/btconfig/init.py", line 721, in run
File "/home/mep/trade/pyht/env/lib/python3.9/site-packages/btconfig-0.2.0-py3.9.egg/btconfig/init.py", line 479, in run
File "/home/mep/trade/pyht/env/lib/python3.9/site-packages/btconfig-0.2.0-py3.9.egg/btconfig/init.py", line 356, in _loadParts
File "/home/mep/trade/pyht/env/lib/python3.9/site-packages/btconfig-0.2.0-py3.9.egg/btconfig/helper.py", line 153, in get_classes
File "/home/mep/trade/pyht/env/lib/python3.9/site-packages/btconfig-0.2.0-py3.9.egg/btconfig/helper.py", line 123, in _iter_classes_submodules
File "/home/mep/trade/pyht/env/lib/python3.9/site-packages/btconfig-0.2.0-py3.9.egg/btconfig/helper.py", line 138, in _import_module
AttributeError: 'zipimporter' object has no attribute 'exec_module'

pip list
Package Version

backcall 0.2.0
backtrader 1.9.76.123
beautifulsoup4 4.9.3
bs4 0.0.1
btconfig 0.2.0
certifi 2021.5.30
chardet 4.0.0
charset-normalizer 2.0.3
click 8.0.1
cycler 0.10.0
debugpy 1.3.0
decorator 5.0.9
empyrical 0.5.5
IbPy2 0.8.0
idna 2.10
ipykernel 6.0.3
ipython 7.25.0
ipython-genutils 0.2.0
jedi 0.18.0
joblib 1.0.1
jupyter-client 6.2.0
jupyter-core 4.7.1
kiwisolver 1.3.1
lxml 4.6.3
matplotlib 3.4.2
matplotlib-inline 0.1.2
multitasking 0.0.9
nest-asyncio 1.5.1
nltk 3.6.2
numpy 1.21.1
pandas 1.3.0
pandas-datareader 0.10.0
parso 0.8.2
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.3.1
pip 21.1.3
pkg-resources 0.0.0
prompt-toolkit 3.0.19
ptyprocess 0.7.0
pyfolio 0.9.2+75.g4b901f6.dirty
Pygments 2.9.0
pyparsing 2.4.7
python-dateutil 2.8.2
pytz 2021.1
pyzmq 22.1.0
QuantStats 0.0.34
regex 2021.7.6
requests 2.26.0
scikit-learn 0.24.2
scipy 1.7.0
seaborn 0.11.1
setuptools 57.4.0
simplejson 3.17.3
six 1.16.0
soupsieve 2.2.1
TA-Lib 0.4.21
tabulate 0.8.9
threadpoolctl 2.2.0
torch 1.9.0
tornado 6.1
tqdm 4.61.2
traitlets 5.0.5
tushare 1.2.64
typing-extensions 3.10.0.0
urllib3 1.26.6
wcwidth 0.2.5
websocket-client 1.1.0
yfinance 0.1.63

please help here.

cant import btconfig in python shell

First of all i don't know too much regarding this module . When i try to run the demo/simple file for understand better but it's showing " no module found named btconfig ' . That's why i try to import btconfig into my python shell it's showing this error.

Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import btconfig
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/arif/Main/Programs/R_&_D/Python/btconfig/btconfig/__init__.py", line 258, in <module>
   import backtrader as bt
 File "/home/arif/Main/Programs/R_&_D/Python/btconfig/env/lib/python3.8/site-packages/backtrader/__init__.py", line 62, in <module>
   from .cerebro import *
 File "/home/arif/Main/Programs/R_&_D/Python/btconfig/env/lib/python3.8/site-packages/backtrader/cerebro.py", line 35, in <module>
   from .brokers import BackBroker
 File "/home/arif/Main/Programs/R_&_D/Python/btconfig/env/lib/python3.8/site-packages/backtrader/brokers/__init__.py", line 30, in <module>
   from .ibbroker import IBBroker
 File "/home/arif/Main/Programs/R_&_D/Python/btconfig/env/lib/python3.8/site-packages/backtrader/brokers/ibbroker.py", line 30, in <module>
   import ib.ext.Order
 File "/home/arif/Main/Programs/R_&_D/Python/btconfig/env/lib/python3.8/site-packages/ib/ext/Order.py", line 9, in <module>
   from ib.lib import Double, Integer
 File "/home/arif/Main/Programs/R_&_D/Python/btconfig/env/lib/python3.8/site-packages/ib/lib/__init__.py", line 239
   except (socket.error, ), ex:
                          ^
SyntaxError: invalid syntax

add genetic optimization support

add support using gradient-free-optimizers

see https://community.backtrader.com/topic/186/genetic-optimization/27 for a example on how to use.

How to get the cerebro instance or another global vars in btconfig in strategy?

For example: to get the global vars from btconfig: config, mode or cerebro to use inside strategy

Thank so much!

happydasch / btconfig Goto Github PK

btconfig's People

Contributors

Stargazers

Watchers

Forkers

btconfig's Issues

Got error when using multiprocessing for multi assets/instruments

Results backtest missing when using OandaV2 datas and stores

plotting default port

Exception: Strategy SimpleStrategy not found on optimize modes simple strategy

Threads and btconfig

AttributeError while running demo

cant import btconfig in python shell

add genetic optimization support

How to get the cerebro instance or another global vars in btconfig in strategy?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent