:mag_right: :chart_with_upwards_trend: :snake: :moneybag: Backtest trading strategies in Python.
Home Page: https://kernc.github.io/backtesting.py/
License: GNU Affero General Public License v3.0
Hello,
I'm using the lib with a dataset like this:
and I have the following error:
C:\Users\capel\Anaconda3\lib\site-packages\backtesting\_util.py:32: FutureWarning: Converting timezone-aware DatetimeArray to timezone-naive ndarray with 'datetime64[ns]' dtype. In the future, this will return an ndarray with 'object' dtype where each element is a 'pandas.Timestamp' with the correct 'tz'.
To accept the future behavior, pass 'dtype=object'.
To keep the old behavior, pass 'dtype="datetime64[ns]"'.
obj = np.asarray(array).view(cls)
C:\Users\capel\Anaconda3\lib\site-packages\backtesting\_util.py:23: FutureWarning: The default of the 'keep_tz' keyword will change to True in a future release. You can set 'keep_tz=True' to obtain the future behaviour and silence this warning.
return df.index[:100].to_series().diff().median()
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-7-b0362348a33a> in <module>
1 backtest = Backtest(dados, BarStrategy, commission=.0075, cash=100)
----> 2 backtest.run()
~\Anaconda3\lib\site-packages\backtesting\backtesting.py in run(self, **kwargs)
703 strategy.next()
704
--> 705 self._results = self._compute_stats(broker, strategy)
706 return self._results
707
~\Anaconda3\lib\site-packages\backtesting\backtesting.py in _compute_stats(self, broker, strategy)
870 s = pd.Series()
871 s['Start'] = df.index[0]
--> 872 s['End'] = df.index[-1]
873 # Assigning Timedeltas needs the key to exist beforehand,
874 # otherwise the value is interpreted as nanosec *int*. See:
~\Anaconda3\lib\site-packages\pandas\core\series.py in __setitem__(self, key, value)
1037 # do the setitem
1038 cacher_needs_updating = self._check_is_chained_assignment_possible()
-> 1039 setitem(key, value)
1040 if cacher_needs_updating:
1041 self._maybe_update_cacher()
~\Anaconda3\lib\site-packages\pandas\core\series.py in setitem(key, value)
1033 pass
1034
-> 1035 self._set_with(key, value)
1036
1037 # do the setitem
~\Anaconda3\lib\site-packages\pandas\core\series.py in _set_with(self, key, value)
1083 self._set_values(key.astype(np.bool_), value)
1084 else:
-> 1085 self._set_labels(key, value)
1086
1087 def _set_labels(self, key, value):
~\Anaconda3\lib\site-packages\pandas\core\series.py in _set_labels(self, key, value)
1093 mask = indexer == -1
1094 if mask.any():
-> 1095 raise ValueError('%s not contained in the index' % str(key[mask]))
1096 self._set_values(indexer, value)
1097
ValueError: ['End'] not contained in the index
I suggest change this code to use the .loc method expressly, like this:
871 s.loc['Start'] = df.index[0]
872 s.loc['End'] = df.index[-1]
bt.run() should produce statistics where 'Buy & Hold Return [%]' returns a negative value.
'Buy & Hold Return [%]' always returns a positive value
bt = Backtest(df, EmaCross, cash=10, commission=.002)
bt.run()
The reason for this bug is in line 906 of backtesting/backtesting.py
s.loc['Buy & Hold Return [%]'] = abs(c[-1] - c[0]) / c[0] * 100 # long OR short
returns the absolute value. It loses very important information. Anyone who needs the absolute value can always take the absolute value of the returned value. But now the information is lost if 'Buy & Hold Return [%]' is positive or negative. This makes it useless to compare in both long and short strategies.
Returning the stats on the best run for backtest.run()
raised a BrokenProcessPool error : A process in the process pool was terminated abruptly while the future was running or pending.
bt = Backtest(df, Strat,, cash=starting_money, commission=.002)
bt.run()
stats = bt.optimize(tresholdVol=range(5,100, 5), maximize='Equity Final [$]')
position.open.price should be 0.00 after closing position.
When running an Optimize routine, it appears as though the child processes that are created aren't releasing memory after they finish. If I write a simple routine that just evaluates multiple SMA values for a closing price crossover (doesn't really matter what) the routine will gradually consume all memory available on the machine and then produces a "raised a BrokenProcessPool error : A process in the process pool was terminated abruptly while the future was running or pending." error. I am currently running in AWS on an Ubuntu box.
Thanks in advance.
Beginning of the routine:
After 5 minutes:
I found the buy and sell signal was wrong.
please check out the attachment.
wrongly P/L -1.538
because the trade is a win trade
1.
2.
3.
I've two question:
I'm using Talib SMA , sma data can output normally, but can't count in the Equity and other results.
Can backtest plot with "Strategy" value. That means only display the indicator like sma
from backtesting import Strategy
from backtesting.lib import crossover
from backtesting import Backtest
import backtesting
import pandas as pd
import talib
df = pd.read_csv("C:/Users/Peter/Desktop/EURUSD_Candlestick_1_Hour_01.09.2018-19.10.2019_b.csv")
df.index = pd.to_datetime(df['Date'])
df.sort_index(ascending = True, inplace=True)
class SmaCross(Strategy):
def init(self):
FSMA = talib.SMA(df['Close'], timeperiod=14)
SSMA = talib.SMA(df['Close'], timeperiod=30)
def next(self):
if crossover(FSMA, SSMA):
self.buy()
elif crossover(SSMA, FSMA):
self.sell()
bt = Backtest(df, SmaCross, cash=1000, commission=.002)
bt.run()Start 2018-01-10 00:00:00
End 2019-12-09 23:00:00
Duration 698 days 23:00:00
Exposure [%] 0
Equity Final [$] 1000
Equity Peak [$] 1000
Return [%] 0
Buy & Hold Return [%] 4.74283
Max. Drawdown [%] -0
Avg. Drawdown [%] NaN
Max. Drawdown Duration NaN
Avg. Drawdown Duration NaN
Trades 0
Win Rate [%] NaN
Best Trade [%] NaN
Worst Trade [%] NaN
Avg. Trade [%] NaN
Max. Trade Duration NaT
Avg. Trade Duration NaT
Expectancy [%] NaN
SQN NaN
Sharpe Ratio NaN
Sortino Ratio NaN
Calmar Ratio NaN
_strategy SmaCross
dtype: object
install via pip
i just tried to install using pip and i got unknown command. also we have an error here as of build status : https://travis-ci.org/kernc/backtesting.py
I was simply trying to run the demo, but I am getting the following error:
Traceback (most recent call last):
File "backtest.py", line 49, in <module>
output = bt.run()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/backtesting/backtesting.py", line 675, in run
strategy.init()
File "backtest.py", line 29, in init
self.sma1 = self.I(SMA, self.data.Close, self.n1)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/backtesting/backtesting.py", line 124, in I
value = func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/backtesting/test/__init__.py", line 23, in SMA
return pd.Series(arr).rolling(n).mean()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pandas/core/window.py", line 1875, in mean
return super().mean(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pandas/core/window.py", line 1178, in mean
return self._apply("roll_mean", "mean", **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pandas/core/window.py", line 910, in _apply
values = self._prep_values(b.values)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pandas/core/window.py", line 239, in _prep_values
values[np.isinf(values)] = np.NaN
ValueError: assignment destination is read-only
I have downloaded the latest version of numpy (1.17.0) in addition to pandas (0.25.0), so I'm suspecting there may be something different in one of the newer version of these libraries. Any insight to why this may be throwing this error would be appreciated.
any idea why i keep getting this error?
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-54-9b8afa777b14> in <module>
20 cash=10000, commission=.002)
21
---> 22 bt.run()
23 bt.plot()
~/miniconda3/lib/python3.7/site-packages/backtesting/backtesting.py in run(self, **kwargs)
681 strategy._set_params(**kwargs)
682
--> 683 strategy.init()
684 indicator_attrs = {attr: indicator
685 for attr, indicator in strategy.__dict__.items()
<ipython-input-54-9b8afa777b14> in init(self)
7 def init(self):
8 data_df = self.data
----> 9 self.supertrend = self.I(ST_numpy_wrapper, data_df, ATR_factor, period)
10
11 def next(self):
~/miniconda3/lib/python3.7/site-packages/backtesting/backtesting.py in I(self, func, name, plot, overlay, color, scatter, *args, **kwargs)
119 """
120 if name is None:
--> 121 params = ','.join(filter(None, map(_as_str, chain(args, kwargs.values()))))
122 func_name = func.__name__.replace('<lambda>', 'λ')
123 name = ('{}({})' if params else '{}').format(func_name, params)
~/miniconda3/lib/python3.7/site-packages/backtesting/_util.py in _as_str(value)
9 if isinstance(value, (Number, str)):
10 return str(value)
---> 11 name = str(getattr(value, 'name', '') or '')
12 if callable(value):
13 name = value.__name__.replace('<lambda>', '')
~/miniconda3/lib/python3.7/site-packages/backtesting/_util.py in __getattr__(self, item)
96 return self.__get_array(item)
97 except KeyError:
---> 98 raise KeyError("Column '{}' not in data".format(item)) from None
99
100 def _set_length(self, i):
KeyError: "Column 'name' not in data"
Hi there,
I am unable to run the example code. I've also attempted to use my own data and I receive the same error as when running the example. Any idea how to make this run?
I get this code repeated a few times when I attempt to run it -
`D:\Programs\Programming\Python\lib\site-packages\bokeh\util\deprecation.py:68: BokehDeprecationWarning:
'legend' keyword is deprecated, use explicit 'legend_label', 'legend_field', or 'legend_group' keywords instead`
Thank you in advance.
Hi,
Is there a way to pass a variable to the class being called by Backtest? I would like to loop through my own strategy using range defined variables.
Thanks in advance.
Im trying to backtest a pairs trading strategy but i don't know how to add multiple data frames to the backtester in order to add positions to each instruments according to the hedge ratio.
Is there a way to do this?
Is there a way to enter a trade and just close it and stay in cash without going the opposite direction? Looking at the docs (please correct me if I am wrong), the buy and sell methods close out all existing positions and use all available capital to enter a new position in the opposite direction. Is there a way to just close a position and stay that way until a strategy condition is met to re-enter?
Thanks in advance.
[New Feature]
After optimizing a strategy, I'd like to be able to access the optimized values or add them as new fields since I'm backtesting across multiple instruments.
An example:
Cci(parameter=130, parameter2=140) is the output in the _strategy field.
I'd like to access parameter=130 and create a new column on the dataframe.
When using JupyterLab hosted on Google Cloud Platform, backtesting.py does not save plots to a file.
This is the code I used:
bt.plot(filename="/home/jupyter/test.html", open_browser=True)The plot is being displayed in the Jupyter notebook but somehow is not saved to file.
bt.plot(filename=)Unable to run bt.plot() using sample code. I'm a beginner with python and would really appreciate some help with this issue.
Ran this on both main anaconda environment (python 3.7.6) and a virtual anaconda environment (python 3.8.1)
This was the error produce when I ran the sample code on VSC.
Traceback (most recent call last):
File "c:/Users/tohka/Desktop/Python Stuff/backtest.py", line 23, in <module>
bt.plot()
File "C:\Users\tohka\Anaconda3\envs\py38\lib\site-packages\backtesting\backtesting.py", line 1000, in plot
plot(
File "C:\Users\tohka\Anaconda3\envs\py38\lib\site-packages\backtesting\_plotting.py", line 545, in plot
fig_ohlc.x_range.callback = CustomJS(args=custom_js_args,
File "C:\Users\tohka\Anaconda3\envs\py38\lib\site-packages\bokeh\core\has_props.py", line 280, in __setattr__
raise AttributeError("unexpected attribute '%s' to %s, %s attributes are %s" %
AttributeError: unexpected attribute 'callback' to Range1d, similar attributes are js_event_callbacks
PnL in pips
PnL in money or percentage, which is also useful
Having an example in the documentation on how use this library for Forex trading with lot size, position size, leverage, margin call, multiple positions opening, etc
I can't figure out how/if this is possible with this library even if you mention forex and have a sample set of EURUSD.
The methods buy and sell of Strategy mention that they "close any current position and use all available funds" which seems a bit extreme and inflexible when money management recommends to use only 1% of your account on each trade.
Maybe I didn't understand what you mean here but I don't see the point to use all or none of the account and not give a strategy a way to set the account's amount/percent we want per trade and to open several trades on the same pair, moreover for Forex with lots.
If not already coded, would love to help improve that.
Disclaimer: I'm new to trading and forex trading, maybe there's something I didn't understand on this part.
Awesome library btw
if we say are defining the next in the strategy class like buy and sell actions how can we track this in the _trade_data object so we can track easily when the order was placed and when the broker executed the buy sell and hold to present in the plots ?
Is there a way to place orders that are limited in size?
I'm working on a Forex strategy where I want to specify the size of the order based on the SL/TP levels provided by a prediction model. Your library mentions that the Strategy.buy/sell methods use "all available funds" which cannot be used in this case.
Thank you!
I have a very simple test:
df=load_dataframe(r’./historical.xlsx')
df = df.set_index('date')
# the backtest is a strategy that does nothing for now
bt = Backtest(df, empty_strategy, cash=1, commission=.002)
bt.run()
# and the display simply
bt.plot()
with all the data, bt.plot() will fail with Javascript Error: Maximum call stack size exceeded.
if I add:
df = df[0:50000]
then the display works fine.
I have about 1.3M data points (and need them); is there a way to resample the data but ONLY for the display?
Is there a way to extract the returns of the strategy and not only have a plot?
I would like to use other data such as sentiment for my backtests. Is there any way to custom data to self.data?
May I ask, what's the best way to pass the Date to the trading system to make it recognize it correctly? This one I'm working with is dealing with US format, so first it goes month, then the day. Also I can combine the time string to that by removing the comma. I may have relied on the auto-detect date format feature, but it may be obscure for the dates where month and day integer are the same, so I'd prefer to pass this format explicitly. Can you advise pls?
US format: mm/dd/yyyy 24h
09/12/2016,08:00:00
Technically I'll have something like this
dt = np.genfromtxt(filename, delimiter=',', encoding="utf-8", dtype=str, skip_header=True, usecols = (fDate_col, fTime_col))
I throw this kind of Forex data to backtesting runner :
Open | High | Low | Close | Volume
1.11293 | 1.11308 | 1.11293 | 1.11304 | 9
1.11306 | 1.11306 | 1.11300 | 1.11300 | 5
1.11298 | 1.11298 | 1.11288 | 1.11296 | 32
1.11298 | 1.11307 | 1.11298 | 1.11307 | 9
1.11310 | 1.11312 | 1.11306 | 1.11306 | 4
1.11304 | 1.11306 | 1.11298 | 1.11300 | 7
1.11298 | 1.11300 | 1.11296 | 1.11298 | 5
1.11296 | 1.11296 | 1.11296 | 1.11296 | 1
1.11298 | 1.11298 | 1.11296 | 1.11296 | 2
1.11298 | 1.11300 | 1.11296 | 1.11298 | 5
1.11296 | 1.11300 | 1.11296 | 1.11300 | 3
Whenever I run my strategy, it gives me a 0% Win rate:
Start 2019-07-29 00:00:00
End 2019-07-30 16:24:00
Duration 1 days 16:24:00
Exposure [%] 50.9076
Equity Final [$] 8581.46
Equity Peak [$] 10000
Return [%] -14.1854
Buy & Hold Return [%] 0.199454
Max. Drawdown [%] -14.2093
Avg. Drawdown [%] NaN
Max. Drawdown Duration NaN
Avg. Drawdown Duration NaN
# Trades 78
Win Rate [%] 0
Best Trade [%] -0.170316
Worst Trade [%] -0.2566
Avg. Trade [%] -0.196327
Max. Trade Duration 0 days 02:14:00
Avg. Trade Duration 0 days 00:16:00
Expectancy [%] NaN
SQN -71.8455
Sharpe Ratio -8.79786
Sortino Ratio -8.79786
Calmar Ratio -0.0138168
But when plotting, I can see mutliple winning trades :
Why does it give me negative P/L ?
On the example of the SMA strategy, it should buy when the fast sma crosses above slow sma,
but i don't understand what it is plotted.
Documentation website
load Bokeh and open a window in chrome with plot
unexpected attribute 'callback' to Range1d, similar attributes are js_event_callbacks
tested with tutorial SMA cross example
Hi,
Somebody can help me to know if it's possible to modify values like stoploss or takeprofit from a order?
I want to check a strategy that when the order take TP, this move the StopLoss to EMA value.
This can be possible?
Thanks guys !
Do you have a quick functional example showing Backtesting working with tulipy? I can't seem to get the data structure correct when passing to self.I().
Optimize strategy
When I try optimize strategy with 3 variables (two emas and 1 dmi) it begin and after a few seconds return this error
print(bt.optimize(dmi=range(3,17,1),n1=range(3,17,1),n2=range(18,40,1),maximize='Equity
Final [$]'))
File "C:\Users\maxi\Anaconda3\lib\site-packages\backtesting\backtesting.py", line 831, in optimize
for _, stats in self._mp_task([params]):
File "C:\Users\maxi\Anaconda3\lib\site-packages\backtesting\backtesting.py", line 850, in _mp_task
for params in param_combos)
File "C:\Users\maxi\Anaconda3\lib\site-packages\backtesting\backtesting.py", line 849, in <listcomp>
return [(params, stats) for params, stats in ((params, self.run(**params))
File "C:\Users\maxi\Anaconda3\lib\site-packages\backtesting\backtesting.py", line 850, in <genexpr>
for params in param_combos)
File "C:\Users\maxi\Anaconda3\lib\site-packages\backtesting\backtesting.py", line 713, in run
self._results = self._compute_stats(broker, strategy)
File "C:\Users\maxi\Anaconda3\lib\site-packages\backtesting\backtesting.py", line 885, in _compute_stats
dd_dur, dd_peaks = _drawdown_duration_peaks(dd, data.index)
File "C:\Users\maxi\Anaconda3\lib\site-packages\backtesting\backtesting.py", line 870, in _drawdown_duration_peaks
peaks[j] = dd[i:j].max()
File "C:\Users\maxi\Anaconda3\lib\site-packages\numpy\core\_methods.py", line 30, in _amax
return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity1.print(bt.optimize(dmi=range(3,17,1),n1=range(3,17,1),n2=range(18,40,1),maximize='Equity Final [$]'))
2.
3.
I can do bt.optimize(n1=range(3,17,1),n2=range(18,40,1),maximize='Equity Final [$]'))
or bt.optimize(dmi=range(3,17,1),maximize='Equity Final [$]'))
I'm optimizing some parameters, and some configurations result in a null price in the middle of the optimization. I'm not sure what is causing it, maybe trying to sell twice in the same period somehow
I use this Function:
stats, heatmap = bc.optimize(
iaf1 = [0.015],
maxaf1 = [0.16],
mgain = [0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.10],
mloss = [0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.10],
constraint=lambda p: p.iaf1 < p.maxaf1,
maximize='Equity Final [$]',
return_heatmap=True)WIth a parabolic SAR indicator:
def next(self):
if not self.position:
if self.boole == 1:
self.buy()
elif self.boole == -1 and not self.position:
self.sell()
else:
if self.position.pl_pct >= self.mgain or self.position.pl_pct <= -self.mloss:
self.position.close()Getting this assertion error
~\Anaconda3\lib\site-packages\backtesting\backtesting.py in sell(self, price, sl, tp)
465
466 def sell(self, price=None, sl=None, tp=None):
--> 467 assert (tp or -np.inf) <= (price or self.last_close) <= (sl or np.inf), "For short orders should be: TP ({}) < BUY PRICE ({}) < SL ({})".format(tp, price or self.last_close, sl) # noqa: E501
468 self.orders._update(price, sl, tp, is_long=False)
469
AssertionError: For short orders should be: TP (None) < BUY PRICE (nan) < SL (None)
Where the price is null, but the model works fine on other parameters.
I would like to test strategies that are long / cash and not only long / short. Is this possible?
I have a M15 timeframe serie that looks like this :
index is supposed to be correctly formated as a datetime:
df['timestamp'] = pd.to_datetime(df['timestamp'])
df = df.set_index(['timestamp'])
But whenever I try to call resample_apply(), for example :
self.kijun_m15 = self.I(Kijun, self.data.High, self.data.Low)
self.kijun_h1 = resample_apply('H', Kijun, self.data.High, self.data.Low)
self.kijun_h4 = resample_apply('4H', Kijun, self.data.High, self.data.Low)
I have the following error:
ValueError: mixed datetimes and integers in passed array
Any idea on what it could mean? I know it's a pandas error but I'm kind of lost here
Here is the complete stacktrace:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\arrays\datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
1860 try:
-> 1861 values, tz_parsed = conversion.datetime_to_datetime64(data)
1862 # If tzaware, these values represent unix timestamps, so we
pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.datetime_to_datetime64()
TypeError: Unrecognized value type: <class 'int'>
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-114-2634449b147c> in <module>
2
3 bt = Backtest(df_backtest, Ichimoku, cash=10000, commission=.00015)
----> 4 bt.run()
~\Anaconda3\envs\Machine Learning\lib\site-packages\backtesting\backtesting.py in run(self, **kwargs)
673 strategy._set_params(**kwargs)
674
--> 675 strategy.init()
676 indicator_attrs = {attr: indicator
677 for attr, indicator in strategy.__dict__.items()
<ipython-input-113-8506b452361b> in init(self)
13
14 #H1
---> 15 self.kijun_h1 = resample_apply('H', Kijun, self.data.High, self.data.Low)
16 self.tenkan_h1 = resample_apply('H', Tenkan, self.data.High, self.data.Low)
17 self.senkou_a_h1 = resample_apply('H', SenkouA, self.data.High, self.data.Low)
~\Anaconda3\envs\Machine Learning\lib\site-packages\backtesting\lib.py in resample_apply(rule, func, series, *args, **kwargs)
228 wrap_func.__name__ = func.__name__
229
--> 230 array = strategy_I(wrap_func, resampled, *args, **kwargs)
231 return array
232
~\Anaconda3\envs\Machine Learning\lib\site-packages\backtesting\backtesting.py in I(self, func, name, plot, overlay, color, *args, **kwargs)
122 **dict(zip(kwargs.keys(), map(_as_str, kwargs.values()))))
123
--> 124 value = func(*args, **kwargs)
125
126 try:
~\Anaconda3\envs\Machine Learning\lib\site-packages\backtesting\lib.py in wrap_func(resampled, *args, **kwargs)
222 def wrap_func(resampled, *args, **kwargs):
223 ind = func(resampled, *args, **kwargs)
--> 224 ind = ind.reindex(index=series.index | ind.index,
225 method='ffill').reindex(series.index)
226 return ind
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\indexes\base.py in __or__(self, other)
2221
2222 def __or__(self, other):
-> 2223 return self.union(other)
2224
2225 def __xor__(self, other):
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\indexes\datetimes.py in union(self, other)
482 if not isinstance(other, DatetimeIndex):
483 try:
--> 484 other = DatetimeIndex(other)
485 except TypeError:
486 pass
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\indexes\datetimes.py in __new__(cls, data, freq, start, end, periods, tz, normalize, closed, ambiguous, dayfirst, yearfirst, dtype, copy, name, verify_integrity)
301 data, dtype=dtype, copy=copy, tz=tz, freq=freq,
302 dayfirst=dayfirst, yearfirst=yearfirst, ambiguous=ambiguous,
--> 303 int_as_wall_time=True)
304
305 subarr = cls._simple_new(dtarr, name=name,
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\arrays\datetimes.py in _from_sequence(cls, data, dtype, copy, tz, freq, dayfirst, yearfirst, ambiguous, int_as_wall_time)
374 data, dtype=dtype, copy=copy, tz=tz,
375 dayfirst=dayfirst, yearfirst=yearfirst,
--> 376 ambiguous=ambiguous, int_as_wall_time=int_as_wall_time)
377
378 freq, freq_infer = dtl.validate_inferred_freq(freq, inferred_freq,
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\arrays\datetimes.py in sequence_to_dt64ns(data, dtype, copy, tz, dayfirst, yearfirst, ambiguous, int_as_wall_time)
1755 # or M8[ns] to denote wall times
1756 data, inferred_tz = objects_to_datetime64ns(
-> 1757 data, dayfirst=dayfirst, yearfirst=yearfirst)
1758 tz = maybe_infer_tz(tz, inferred_tz)
1759 # When a sequence of timestamp objects is passed, we always
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\arrays\datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
1864 return values.view('i8'), tz_parsed
1865 except (ValueError, TypeError):
-> 1866 raise e
1867
1868 if tz_parsed is not None:
~\Anaconda3\envs\Machine Learning\lib\site-packages\pandas\core\arrays\datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
1855 dayfirst=dayfirst,
1856 yearfirst=yearfirst,
-> 1857 require_iso8601=require_iso8601
1858 )
1859 except ValueError as e:
pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()
pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()
ValueError: mixed datetimes and integers in passed array
To dig into each backtest order, specifically to evaluate target prices which I don't believe are working correctly or I'm not calling properly. Didn't see an example in the docs.
I tried adding self.orders to _broker.log but without success - the idea was use copy.copy(self.orders) in backtesting.py for a PIT (point in time) snapshot.
So 2 points I'd like addressed:
In class _Broker.next(), I called this function when opening or closing a position.
for example in class _Broker.next():
if entry or orders._close:
self._update_order_log(self.orders, i)
where:
def _update_order_log(self, order, i):
self.log.order_info[i] = copy.copy(order)
then in class _Broker:
class _Log:
def __init__(self, length):
...
self.order_info = {k: '' for k in range(length)}
class SmaCrossTarget(Strategy):
def init(self):
self.sma1 = self.I(SMA, self.data.Close, 2)
self.sma2 = self.I(SMA, self.data.Close, 6)
self.Close = self.data.Close
def next(self):
if (self.sma1[-2] < self.sma2[-2] and self.sma1[-1] > self.sma2[-1]):
self.buy(tp=self.Close[-1]*1.05)
elif (self.sma1[-2] > self.sma2[-2] and self.sma1[-1] < self.sma2[-1]):
self.sell()
GOOG = GOOG.head(40)
bt = Backtest(GOOG, SmaCrossTarget, cash=1000, commission=.001)
result = bt.run()
result._trade_data.dropna(subset=['Entry Price']).head(3)
Equity Exit Entry Exit Position Entry Price Entry Direction Exit Price Exit Comment P/L Returns
2004-08-30 1,030.0300 nan nan 105.2800 -1.0000 nan nan nan
2004-09-09 1,021.8743 7.0000 -9.4890 102.5300 1.0000 102.5300 NaN 25.0957 0.0244
2004-09-28 1,154.8585 14.0000 9.9880 121.3000 -1.0000 121.3000 NaN 186.4511 0.1825
The 3rd line should have been closing the position when target_price was hit.
Hello!
Copy/pasting the example code for optimisation of the SMA strategy gives the following error:
AttributeError: Strategy 'SmaCross' is missing parameter 'n1'. Strategy class should define parameters as class variables before they can be optimized or run with.
Not sure of what's wrong.
I copy and run the SMAcross example exactly from user guide, still I got nothing returned, no statistics, no chart. I run the code on Spyder. What might be the problem? Would you pls advise?
Is there a way to plot Stop loss and Take profit points? or indeed add any such additional plot points for specified data in scatter or line format. Useful for ensuring strategy is operating as per
Is there an elegant and simple way to visualize a few trading models plotted on 1 chart? In fact a lot of projects like mine are centered around comparing various models (and even groups of models), so that would be handy to have a sugary way around this process, so:
works with just 1 model
After importing this .csv: https://github.com/CrossfireX/Crypto-TA/blob/master/BTCUSDT_1h.csv
to the df dataframe in backtesting sample under USAGE with and run
from backtesting import Backtest, Strategy
from backtesting.lib import crossover
import csv
import pandas as pd
#IMPORT DATA
filename='BTCUSDT_1h.csv'
df = pd.read_csv(filename, sep=',')
#print(df)
def SMA(values, n):
"""
Return simple moving average of `values`, at
each step taking into account `n` previous values.
"""
return pd.Series(values).rolling(n).mean()
class SmaCross(Strategy):
# Define the two MA lags as *class variables*
# for later optimization
n1 = 10
n2 = 20
def init(self):
# Precompute two moving averages
self.sma1 = self.I(SMA, self.data.Close, self.n1)
self.sma2 = self.I(SMA, self.data.Close, self.n2)
def next(self):
# If sma1 crosses above sma2, buy the asset
if crossover(self.sma1, self.sma2):
self.buy()
# Else, if sma1 crosses below sma2, sell it
elif crossover(self.sma2, self.sma1):
self.sell()
bt = Backtest(df, SmaCross, cash=10000, commission=.002)
bt.run()
I get "UserWarning: Data index is not datetime. Assuming simple periods.
bt = Backtest(df, SmaCross, cash=10000, commission=.002)"
The dataframes look identical to the samples in EURUSD.csv.
Can you please help with what is to be done here?
Hvala ti in lp!
I downloaded the sample:
Quick Start User Guide.jpynb
When I run it, and try to plot a graph, I get this:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-6-5ae8d19b00ab> in <module>
----> 1 bt.plot()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/backtesting/backtesting.py in plot(self, results, filename, plot_width, plot_equity, plot_pl, plot_volume, plot_drawdown, smooth_equity, relative_equity, omit_missing, superimpose, show_legend, open_browser)
1022 superimpose=superimpose,
1023 show_legend=show_legend,
-> 1024 open_browser=open_browser)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/backtesting/_plotting.py in plot(results, df, indicators, filename, plot_width, plot_equity, plot_pl, plot_volume, plot_drawdown, smooth_equity, relative_equity, omit_missing, superimpose, show_legend, open_browser)
561 toolbar_options=dict(logo=None),
562 merge_tools=True,
--> 563 **kwargs
564 )
565 show(fig, browser=None if open_browser else 'none')
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/bokeh/layouts.py in gridplot(children, sizing_mode, toolbar_location, ncols, plot_width, plot_height, toolbar_options, merge_tools)
293
294 # Integrity checks & set-up
--> 295 _verify_sizing_mode(sizing_mode)
296
297 if toolbar_location:
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/bokeh/layouts.py in _verify_sizing_mode(sizing_mode)
449 def _verify_sizing_mode(sizing_mode):
450 if sizing_mode not in SizingMode:
--> 451 raise ValueError("Invalid value of sizing_mode: %s" % sizing_mode)
452
453
ValueError: Invalid value of sizing_mode: stretch_width
what could be the reason? the same sample works well within https://notebooks.gesis.org/binder/jupyter/user/kernc-backtesting.py-5o73q35k/lab
Hi - I 'm trying to understand how to use indicators from the TA library (e.g. https://pypi.org/project/ta/) within a strategy ?
Has anyone created an example they can share ?
thank you
First of all, thank you for this project it is very well done.
This is a feature request to add support for bar graphs in addition to the current line and scatter plots.
Oscillators might also benefit from an alternative style with an area colouring like so:
And having two lines together in one graph:
My data source is returning the following columns: ['open', 'high', 'low', 'close', 'tick_volume', 'spread', 'real_volume']
In my case, I could call Backtest(stock_data, SmaCross, commission=.002, column_open='open', column_high='high'....)
This is not a bug but an improvement
Hi!
I tried running the example as shown in your website for the simple case. No errors were seen in installing backtesting-0.1.0.
The whole code:
from backtesting import Backtest, Strategy
from backtesting.lib import crossover
from backtesting.test import SMA, GOOG
class SmaCross(Strategy):
n1 = 10
n2 = 30
def init(self):
self.sma1 = self.I(SMA, self.data.Close, self.n1)
self.sma2 = self.I(SMA, self.data.Close, self.n2)
def next(self):
if crossover(self.sma1, self.sma2):
self.buy()
elif crossover(self.sma2, self.sma1):
self.sell()
bt = Backtest(GOOG, SmaCross, cash=10000, commission=.002)
output = bt.run()
bt.plot()The error seems to come out only when running this line
output = bt.run()Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/series.py", line 760, in _set_with_engine
self.index._engine.set_value(values, key, value)
File "pandas/index.pyx", line 121, in pandas.index.IndexEngine.set_value (pandas/index.c:3600)
File "pandas/index.pyx", line 129, in pandas.index.IndexEngine.set_value (pandas/index.c:3426)
File "pandas/index.pyx", line 159, in pandas.index.IndexEngine.get_loc (pandas/index.c:4018)
File "pandas/hashtable.pyx", line 675, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12368)
File "pandas/hashtable.pyx", line 683, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12322)
KeyError: 'Duration'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/indexes/base.py", line 1945, in get_loc
return self._engine.get_loc(key)
File "pandas/index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas/index.c:4154)
File "pandas/index.pyx", line 159, in pandas.index.IndexEngine.get_loc (pandas/index.c:4018)
File "pandas/hashtable.pyx", line 675, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12368)
File "pandas/hashtable.pyx", line 683, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12322)
KeyError: 'Duration'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/series.py", line 701, in setitem
self._set_with_engine(key, value)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/series.py", line 763, in _set_with_engine
values[self.index.get_loc(key)] = value
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/indexes/base.py", line 1947, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas/index.c:4154)
File "pandas/index.pyx", line 159, in pandas.index.IndexEngine.get_loc (pandas/index.c:4018)
File "pandas/hashtable.pyx", line 675, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12368)
File "pandas/hashtable.pyx", line 683, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12322)
KeyError: 'Duration'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/backtesting/backtesting.py", line 705, in run
self._results = self._compute_stats(broker, strategy)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/backtesting/backtesting.py", line 876, in _compute_stats
s['Duration'] = 0
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/series.py", line 753, in __setitem__
setitem(key, value)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/series.py", line 729, in setitem
self.loc[key] = value
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 132, in __setitem__
self._setitem_with_indexer(indexer, value)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 340, in _setitem_with_indexer
new_values])
TypeError: invalid type promotion
Any idea why this error is raised?
raceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "c:\Users\User\.vscode\extensions\ms-python.python-2020.2.64397\pythonFiles\lib\python\new_ptvsd\wheels\ptvsd\__main__.py", line 45, in <module>
cli.main()
File "c:\Users\Bhawaniya\.vscode\extensions\ms-python.python-2020.2.64397\pythonFiles\lib\python\new_ptvsd\wheels\ptvsd/..\ptvsd\server\cli.py", line 361, in main
run()
File "c:\Users\User\.vscode\extensions\ms-python.python-2020.2.64397\pythonFiles\lib\python\new_ptvsd\wheels\ptvsd/..\ptvsd\server\cli.py", line 203, in run_file
runpy.run_path(options.target, run_name="__main__")
File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "c:\Users\User\source\repos\fx-algo\bt.py", line 107, in <module>
output = bt.run()
File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\backtesting\backtesting.py", line 684, in run
strategy.init()
File "c:\Users\User\source\repos\fx-algo\bt.py", line 76, in init
self.ma1 = self.I(ta.DEMA, Open, self.n1)
File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\backtesting\backtesting.py", line 138, in I
x = value / self._data.Close
TypeError: unsupported operand type(s) for /: 'float' and 'str'
class SmaCross(Strategy):
n1 = 6
n2 = 6
def init(self):
Open = pd.Series(self.data.Open).astype(float).diff().fillna(0)
Close = pd.Series(self.data.Close).astype(float).diff().fillna(0)
self.ma1 = self.I(ta.DEMA, Close, self.n1)
self.ma2 = self.I(ta.DEMA, Open, self.n1)
def next(self):
if crossover(self.ma1, self.ma2):
self.buy()
elif crossover(self.ma2, self.ma1):
self.sell()
dataframe = dataframe.to_pickle("./EUR_USD.pkl")
bt = Backtest(dataframe, SmaCross, cash=10000)
output = bt.run()Note:
dataframe pickle here is available to troubleshoot.
When dataframe is saved as csv and when loaded for back testing then it works fine but when generated dataframe is passed AS IS, it raises the above error.
It's a rare situation - you need both stop loss and new order to happen in the same time step - but when happens it is such a pain to nail down what's really wrong, since the framework won't break. Instead, all the data following will be slightly wrong. Started debugging this when trying to understand why ohlc_trades plot would draw two trades going on at the same time, if clearly existing positions need to be closed before opening new ones. That "double trade" plotting is result of that misaligned data due to the erroneous open position and instant stop loss.
Close the original position, ignore the stop loss and create a new order
Close the original position, create a new order, and instantly close it due to stop loss
A couple of fix suggestions:
Update open, high, low once self._open_position(entry, is_long) is called (backtesting.py line 539)
if entry or orders._close:
self._close_position()
orders._close = False
################ THIS
# First make the entry order, if hit
if entry:
if entry is _MARKET_PRICE or high > orders._entry > low:
self._open_position(entry, is_long)
################ THIS
# Check if stop-loss threshold was hit
if sl and self._position:
price = (sl if low <= sl <= high else # hit
open if (is_long and open < sl or # gapped hit
not is_long and open > sl) else
None) # not hit
if price is not None:
self._close_position(price)
self.orders.cancel()
# Check if take-profit threshold was hit
if tp and self._position:
price = (tp if low < tp < high else
open if (is_long and open > tp or
not is_long and open > sl) else
None)
if price is not None:
self._close_position(price)
self.orders.cancel()
###########################################
#### SHOULD PROBABLY BE MOVED HERE #######
###########################################When working with O,H,L,C and having TP/SL which is within the range of H/L, does the lib automatically close orders inside the bar, not on actual borders of the range? Sorry, not an issue, just to be sure...
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.