`backfill_start` no longer working as before about alpaca-backtrader-api HOT 14 CLOSED

Hi guys, sorry it took me more than expected.
@kr011 just a small note prior to my new input - you should use datetime.utcnow() not datetime.now() since backtrader works in utc.

there was a problem with fetching data for back_filling minute data.
so I fixed that.
still I feel that I'm missing something so I would appreciate your response.
the latest code is in the master branch.
install it like so:
pip install -U git+https://github.com/alpacahq/alpaca-backtrader-api
also make sure that you have the latest python-sdk
pip install -U git+https://github.com/alpacahq/alpaca-trade-api-python

from alpaca-backtrader-api.

seems to be solved... thanks!!

from alpaca-backtrader-api.

Did some more debugging. It appears that the backfill data no longer triggers a next() call. This may be desired functionality, but it does seem like a "regression" from previous behavior. If I add a Filter to the data stream I can log out the incoming data, but my next() function never sees it. It appears that it still serves the purpose of updating all the Signals in my strategy which was why I needed to backfill in the first place. Still not sure if this change is correct or not.

from alpaca-backtrader-api.

if I understand this correctly, the backfilled data should not trigger a next() call. the backfilled data is not live data, right? so it should trigger next.
does your next() function get triggered for live data?

from alpaca-backtrader-api.

@shlomikushchi I think backfill data should call next if backfill is large enough to satisfy defined technical indicator needs, just like backtest data would.

from alpaca-backtrader-api.

but would it call next() if the algorithm doesn't need the indicator for that particular time?

from alpaca-backtrader-api.

It shouldn't matter. I give you an example. Imagine there is a strategy gives direction if you should be in short, long, or closed position based on a crossover indicator (so the flag is set exactly when the crossover happens and if you miss that timeframe the flag isn't updated). You want to make it so that it's resilient so that if your server blows up and you start it back up sometime after the signal went into long, you still want to see it and take action, even if it's a little late. In such case the backfill data will be chosen to be large, much larger than what is needed for the indicators. The direction flag will get changed in next(). There would be a notify_data() to update data status to live so orders wouldn't be placed when live if data isn't live.

from alpaca-backtrader-api.

could you provide an example algo that I could execute locally and see what happens?

from alpaca-backtrader-api.

I think in the latest release backfill is broken (as this bug is pointing out I guess). When I try something simple like this. Next is never called. If I comment out "fromdate", next is called but backfill data goes way back and we start getting the "sleep 3 seconds" thing and it will take forever for it to catch up to live.

import alpaca_backtrader_api
import backtrader as bt
from datetime import datetime, timedelta
from secrets import alpaca_paper_id, alpaca_paper_key

class TestAlgo(bt.Strategy):
    def __init__(self):
        self.live_data = False
        sma1 = bt.indicators.SMA(period=5)
        sma2 = bt.indicators.SMA(period=10)
        self.direction = bt.indicators.CrossOver(sma1, sma2)

    def notify_data(self, data, status, *args, **kwargs):
        if status == data.LIVE:
            self.live_data = True

    def next(self):
        if self.live_data and self.direction > 0:
            self.buy()
        elif self.live_data and self.direction < 0:
            self.sell()

if __name__ == '__main__':
    cerebro = bt.Cerebro()
    cerebro.addstrategy(TestAlgo)

    store = alpaca_backtrader_api.AlpacaStore(
        key_id=alpaca_paper_id,
        secret_key=alpaca_paper_key,
        paper=True
    )

    broker = store.getbroker()
    cerebro.setbroker(broker)

    DataFactory = store.getdata
    datakwargs = dict(
        historical=False,
        timeframe=bt.TimeFrame.Minutes,
        backfill_start=True,
        fromdate=datetime.now()-timedelta(hours=3),
    )
    data0 = DataFactory(dataname='SPY', **datakwargs)
    cerebro.adddata(data0)
    cerebro.run()

from alpaca-backtrader-api.

thanks I will try it out tomorrow

from alpaca-backtrader-api.

Unfortunately the master branches of both are broken in some other way which I am trying to figure out before testing this. My backtests are all failing to pull more than just a few months of data no matter what start date I set.

from alpaca-backtrader-api.

what do you mean of both? what packages?
is it an api issue or a package issue?
and could you give me an example I could try

from alpaca-backtrader-api.

Hi, Im trying to run backtrader with Alpaca, on PAPER account with live data (only 1 symbol) in minutes timeframe.
The first run was fine, retriving some historical data and then retriving LIVE data... then subsequent runs only get historical data, once Notify says LIVE, no additional data was retrieved... any idea?

from alpaca-backtrader-api.

I think there was a server issue. do you still see a problem?

from alpaca-backtrader-api.