Giter VIP home page Giter VIP logo

Comments (18)

arahmed24 avatar arahmed24 commented on July 28, 2024 1

@shlomikushchi Thanks for the update.

We will try to update the children ids in the list and see how that works out for us. We will keep you posted as well.

Thanks

from alpaca-backtrader-api.

nelfata avatar nelfata commented on July 28, 2024

Here is the strategy I have used:

class Template(bt.Strategy):
    """A simple strategy template"""
    params = {'slow'        : 20,
              'usebracket'  : True,
              'rawbracket'  : False,
              'pentry'      : 0,   # % below previous close
              'plimit'      : 5,  # % limit price (make profit)
              'pstop'       : 2,   # % stop price (stop loss)
              'valid'       : 1,   #
              }

    def __init__(self):
        self.slowma = dict()
        self._addobserver(True, bt.observers.BuySell)

        self.buycnt  = 0
        self.sellcnt = 0
        self.total   = 0

        self.tLastOrder = time.time()

        # create array of dictionary
        self.o = (dict())  # orders per data (main, stop, limit, manual-close)
        # fill the array index with symbols each with its own dictionary
        for sym in self.getdatanames():
            self.o[sym] = dict()
        for sym in self.getdatanames():
            # The moving averages
            self.slowma[sym] = bt.indicators.SimpleMovingAverage( self.getdatabyname(sym),      # The symbol for the moving average
                                                                period=self.params.slow,    # Slow moving average
                                                                plotname="SlowMA: " + sym)

    def log(self, txt, dt=None):
        dt = dt or self.datas[0].datetime.datetime(0)
        print(dt.strftime('%m-%d %H:%M:%S'), txt)

    def notify_cashvalue(self, cash, value):
        return
        self.log('Cash %s Value %s' % (cash, value))

    def notify_trade(self, trade):
        sym = trade.getdataname()
        #self.log(f"{sym}: SUBMITTING trade: {trade.size:.2f} {trade.price:.2f}")
        if trade.isclosed:
            #self.log(f"{sym}: CLOSED gross {trade.pnl:.2f} net {trade.pnlcomm:.2f}")
            self.log(f"{sym}: PROFIT {trade.pnlcomm:.2f}")

    def notify_order(self, order):
        sym    = order.data._name
        ref    = order.ref
        date   = self.datetime.date()
        status = order.getstatusname()

        # get the order from the
        whichord = ['main', 'stop', 'limit', 'close']
        dorders  = self.o[sym][order.data]
        idx      = dorders.index(order)

        self.log(f'   {sym}: ref {ref} {status}: {whichord[idx]}')

        # show some possible errors
        if order.status in [order.Expired, order.Margin, order.Rejected]:
            self.log(f'   {sym}: ref {ref} {status}: {whichord[idx]}')

        # NOTE:
        # occasionally in live trading we receive Partial instead of Completed even though the order has been filled

        if order.status in [order.Completed, order.Partial, order.Canceled, order.Expired, order.Margin, order.Rejected]:
            # order canceled, remove it
            #self.log(f'   {sym}: ref {ref} {status}: {whichord[idx]}')
            try:
                dorders[idx] = None
                self.log(f'   {sym}: ref {ref} removed: {whichord[idx]}')
                if all(x is None for x in dorders):
                    dorders[:] = []  # empty list - New orders allowed
                    self.log(f'   {sym}: order list empty')
            except:
                print('   ERR *** order.data empty')

    def next(self):
        """Define what will be done in a single step, including creating and closing trades"""
        for sym in self.getdatanames():    # Looping through all symbols
            d     = self.getdatabyname(sym)
            pos   = self.getpositionbyname(sym) # pos.size, pos.price
            dt    = self.datetime.datetime(0)
            qty   = self.getsizing(d, isbuy=True)
            price = d.close[0]

            self.log(f'{sym}: CHECK  {price:.2f} {pos.size:.2f} {pos.price:.2f} {time.time()}')

            # no position, no orders
            if not pos.size and not self.o[sym].get(d, None):
                # Consider the possibility of entrance
                # Notice the indexing; [0] always means the present bar, and [-1] the bar immediately preceding
                # Thus, the condition below translates to: "If today the regime is bullish (greater than
                # 0) and yesterday the regime was not bullish"

                if True: #self.slowma[sym][0] > self.slowma[sym][-1]:    # A buy signal

                    # upon startup, not all data is valid, so we wait a few seconds...
                    # also we need to allow time between placing orders...this is still under test
                    if time.time() - self.tLastOrder < 10.0:
                       return
                    self.tLastOrder = time.time()

                    self.log(f'{sym}: CHECK  {price:.2f} {pos.size:.2f} {pos.price:.2f} {time.time()}')

                    if self.params.usebracket:
                        price = price * (1.0 - self.params.pentry / 100.0)
                        pstp  = price * (1.0 - self.params.pstop  / 100.0)
                        plmt  = price * (1.0 + self.params.plimit / 100.0)
                        valid = datetime.timedelta(self.params.valid)

                        #pstp  = 118.55
                        #plmt  = 123.64

                        self.log(f'{sym}: BUY BRKT @{pstp:.2f} {price:.2f} {plmt:.2f} {valid} qty: {qty:.2f}')

                        if self.p.rawbracket:
                            o1 = self.buy (data=d, exectype=bt.Order.Limit, price=price, valid=valid, transmit=False)
                            o2 = self.sell(data=d, exectype=bt.Order.Stop,  price=pstp, size=o1.size,transmit=False, parent=o1)
                            o3 = self.sell(data=d, exectype=bt.Order.Limit, price=plmt, size=o1.size,transmit=True,  parent=o1)
                            self.o[sym][d] = [o1, o2, o3]
                        else:
                            self.o[sym][d] = self.buy_bracket(  data       = d,
                                                                price      = price,
                                                                stopprice  = pstp,
                                                                limitprice = plmt,
                                                                exectype   = bt.Order.Market,
                                                                #oargs      = dict(valid=valid)
                                                                    )
                        #self.log('{}: BUY REF [Main {} Stp {} Lmt {}]'.format(sym, *(x.ref for x in self.o[sym][d])))
                    else:
                        self.o[sym][d] = [self.buy(data=d)]
                        self.log(f'{sym}: BUY @{price:.2f} qty: {qty:.2f}')
                        #self.log('{}: Buy {}'.format(sym, self.o[sym][d][0].ref))

            else:    # We have an open position
                return
                if self.slowma[sym][0] < self.slowma[sym][-1]:    # A sell signal
                    self.log(f'{sym}: SELL *qty {self.getsizing(d, isbuy=False):.2f}')
                    o = self.close(data=d)
                    try:
                        self.o[sym][d].append(o)  # manual order to list of orders
                    except:
                        print('Not part of order')
                        #return
                    try:
                        #for xo in self.o[sym][d]:
                        #    if xo is not None:
                        #        print('REF ',xo.ref)
                        #self.log('{}: CLOSE [Main {} Stp {} Lmt {}]'.format(sym, *(x.ref for x in self.o[sym][d])))
                        self.log('{}: Manual Close {}'.format(sym, o.ref))
                    except:
                        self.log('{sym}: No ref')
                    if self.p.usebracket:
                        try:
                            o = self.cancel(self.o[sym][d][1])  # cancel stop side
                            self.log('{}: Cancel {}'.format(sym, self.o[sym][d][1].ref))
                        except:
                            #print(self.o[sym][d])
                            #print('bad order sequence')
                            pass

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

first try installing the most updated code for both packages like so:
pip install -U git+https://github.com/alpacahq/alpaca-trade-api-python
pip install -U git+https://github.com/alpacahq/alpaca-backtrader-api
and let's see if this issue still occurs.

from alpaca-backtrader-api.

nelfata avatar nelfata commented on July 28, 2024

I tried the latest updates that you mentioned on the live market today, but with the same behavior.
Cancelling pending orders (stop on loss) does not get any notifications and the position is not updated. This is based on the code that is based above.
Please keep in mind that the code is written to handle multiple symbols (only one is in use). I may be using the wrong class members, but the code is based on:
https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example/

from alpaca-backtrader-api.

nelfata avatar nelfata commented on July 28, 2024

Ctrl-C out of the program shows the following (if that might help):

await handler(self, channel, ent)
06-15 00:00:00 TVIX: CHECK 168.92 0.00 0.00 1592241781.8473768
06-15 00:00:00 TVIX: BUY BRKT @165.54 168.92 177.37 1 day, 0:00:00 qty: 526.67
06-15 13:23:00 TVIX: ref 1 Submitted: main
06-15 13:23:00 TVIX: ref 2 Submitted: stop
06-15 13:23:00 TVIX: ref 3 Submitted: limit
06-15 13:23:01 TVIX: ref 1 Accepted: main
06-15 13:23:01 TVIX: ref 2 Accepted: stop
06-15 13:23:01 TVIX: ref 3 Accepted: limit
06-15 13:23:01 TVIX: ref 1 Accepted: main
06-15 13:23:02 TVIX: ref 2 Accepted: stop
06-15 13:23:02 TVIX: ref 3 Accepted: limit
06-15 13:23:02 TVIX: ref 1 Partial: main
06-15 13:23:02 TVIX: ref 1 removed: main
Traceback (most recent call last):
File "C:\Users\NEF\Desktop\NextCloud\Clones\Projects\Trading\Backtrader\btTest1.py", line 100, in
results = cerebro.run()
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\site-packages\backtrader\cerebro.py", line 1127, in run
runstrat = self.runstrategies(iterstrat)
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\site-packages\backtrader\cerebro.py", line 1298, in runstrategies
self._runnext(runstrats)
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\site-packages\backtrader\cerebro.py", line 1542, in _runnext
drets.append(d.next(ticks=False))
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\site-packages\backtrader\feed.py", line 407, in next
ret = self.load()
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\site-packages\backtrader\feed.py", line 479, in load
_loadret = self._load()
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\site-packages\alpaca_backtrader_api\alpacadata.py", line 259, in _load
self.qlive.get(timeout=self._qcheck))
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\queue.py", line 179, in get
self.not_empty.wait(remaining)
File "C:\Users\NEF\AppData\Local\Programs\Python\Python37\lib\threading.py", line 300, in wait
gotit = waiter.acquire(True, timeout)
KeyboardInterrupt

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

That can be easily tested by performing a buy order through the alpaca-trade-api, then closing the position on the Alpaca website.

what you have to understand is that this repo is an integration between 2 different entities.
each entity manages its own resources (account, porfolio, positions, ..)
so when you change it on the web, you basically creating a difficult situation for the alpaca-backtrader instance, because the data is really stored on the alpaca servers. not on your local running instance.
one solution (which I actually tried before) is to always make sure the positions are synced
but, that creates an overload api requests to the servers.

now, there's another way for you get the exact and synced data, and it's by doing the api call from next()
so if you do this: self.broker.update_positions() you will get the exact position values even if you change it in the website

from alpaca-backtrader-api.

nelfata avatar nelfata commented on July 28, 2024

You have a point here, but I think the data should never be stale when performing live trading.
The fact that backtesting works and live trading does not, shows that some fundamental design needs to be reviewed, I am assuming here that this is not a library for backtesting only, but also for live trading. It is ok if we need to perform extra steps for live trading, but this is not documented.
The position is not the only problem, the notification updates for all the transactions are also not reported properly during live trading.
This is just my opinion as I am also interested in live trading and this library is great.

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

image

I just filled an order on the web and got the transaction inside the app
the entire processing of the transaction relies on having an order object attached to it.
and we don't. because we didn't generate the order inside the app
so we cannot process it.
so we cannot notify the user about the transaction

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

I have created a patch that updates the positions even if we didn't generate the order inside the app.
try it, and let me know if that helps you
it's in this PR:
#72
and you could install it like this:
pip install -U git+https://github.com/alpacahq/alpaca-backtrader-api@update_positions_outside_of_scope

from alpaca-backtrader-api.

nelfata avatar nelfata commented on July 28, 2024

Thanks for the follow up.
I tried your changes, but nothing changed. I made a bracket order as shown in the code above. The main order was filled the other orders were still open. Cancelling the open orders did not update and closing the position did not update.
On the third try, the main order was filled but for some reason it did not update in the app.

from alpaca-backtrader-api.

 avatar commented on July 28, 2024

@shlomikushchi Hey, I've been facing the same issue any help on how to solve it?

from alpaca-backtrader-api.

arahmed24 avatar arahmed24 commented on July 28, 2024

@shlomikushchi I was trying to review/understand "def _t_order_create(self):" and it seems like the store only stores the parent mapping and does not store the bracket child orders and thats why the status does not update the order properly.

Here's the line which maps the parent only.

self._ordersrev[oid] = oref # maps ids to backtrader order

Any help would be greatly appreciated or lets us know how to map children ids and we update the code and test/valid.

Thanks in advance.

cc: @ichippa96

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

Hi guys, thanks for the input.
I'm working on this issue still.
I will update you once I have anything new

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

so the issue is that when we get the notification, we don't have a backtrader order object. this is because when it is created in the API it is a new order, not part of the backtrader bracket order.
so it is difficult to notify through the regular pipes (still trying)
but for now, I can notify you through notify_store. will this help you guys with the issue?

from alpaca-backtrader-api.

 avatar commented on July 28, 2024

@shlomikushchi we managed to fix it by iterating through the legs of the parent order and assigning the child/legs order the parents ref as well.

def _t_order_create(self):
        while True:
            try:
                # if self.q_ordercreate.empty():
                #     continue
                msg = self.q_ordercreate.get()
                if msg is None:
                    break
                print(f'{datetime.now()} msg in _t_order_create: {msg}')
                oref, okwargs = msg
                try:
                    o = self.oapi.submit_order(**okwargs)
                except Exception as e:
                    self.put_notification(e)
                    self.broker._reject(oref)
                    return


                try:
                    oid = o.id
                except Exception:
                    if 'code' in o._raw:
                        self.put_notification(o.message)
                    else:
                        self.put_notification(
                            "General error from the Alpaca server")
                    self.broker._reject(oref)
                    return
                self._orders[oref] = oid
                self.broker._submit(oref)

                if okwargs['type'] == 'market':
                    self.broker._accept(oref)  # taken immediately

                oids=list()
                oids.append(oid)
                if o.legs is not None:
                    for leg in o.legs:
                        oids.append(leg.id)

                self._orders[oref] = oids[0]
                self.broker._submit(oref)


                if okwargs['type'] == 'market':
                    self.broker._accept(oref)  # taken immediately

                for oid in oids:
                    self._ordersrev[oid] = oref  # maps ids to backtrader order

                    # An transaction may have happened and was stored
                    tpending = self._transpend[oid]
                    tpending.append(None)  # eom marker
                    while True:
                        trans = tpending.popleft()
                        if trans is None:
                            break
                        self._process_transaction(oid, trans)
            except Exception as e:
                print(str(e))

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

thanks, I will try it tomorrow when the market opens

from alpaca-backtrader-api.

arahmed24 avatar arahmed24 commented on July 28, 2024

@shlomikushchi Thanks!

We made progress and it works when the take/limit side is executed versus the stop side. When the stop is executed we get an issue processing replaced status of take/limit side. We tried to execute _cancel for the take/limit side but it's giving us an error. We are thinking of trying to either just do nothing (just return) or write a new cancel function to handle replaced order.

Thanks

from alpaca-backtrader-api.

shlomiku avatar shlomiku commented on July 28, 2024

Hi Guys, I created this PR that addresses this issue:
#77
thank you @arahmed24 @ichippa96 and Ibrahim for helping out with this issue.
it will be merged after I am sure it is working correctly.

you could install it like this: pip install -U git+https://github.com/alpacahq/alpaca-backtrader-api@update_bracket_orders

from alpaca-backtrader-api.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.