Comments (27)
shlomiku
commented on July 28, 2024
1
let's try something.. I'm creating a branch for you with some added resilience to the alpacastore. let's see if that helps.
install it like this: pip install -U git+https://github.com/alpacahq/alpaca-backtrader-api@more_resilience_for_alpacastore
now remember, it's ok to get the concurrent.futures._base.CancelledError as long as the execution continues.
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
this is a websocket diconnection error.
I'm currently trying to fix it with this branch:
https://github.com/alpacahq/alpaca-trade-api-python/tree/websocket_reconnect
you could install it locally and see if that helps, even before we merge and release it.
do this:
pip install -U git+https://github.com/alpacahq/alpaca-trade-api-python@websocket_reconnect
I believe it will help you with that issue
from alpaca-backtrader-api.
jtbales
commented on July 28, 2024
Thanks, I'm using the branch now and will find out during today's market
from alpaca-backtrader-api.
jtbales
commented on July 28, 2024
I ran paper trading live a few times and got the same error. A couple times I even got this error:
Traceback (most recent call last):
File "./main.py", line 90, in <module>
main()
File "./main.py", line 49, in main
cerebro.run()
File "/app/.heroku/python/lib/python3.7/site-packages/backtrader/cerebro.py", line 1127, in run
runstrat = self.runstrategies(iterstrat)
File "/app/.heroku/python/lib/python3.7/site-packages/backtrader/cerebro.py", line 1298, in runstrategies
self._runnext(runstrats)
File "/app/.heroku/python/lib/python3.7/site-packages/backtrader/cerebro.py", line 1630, in _runnext
strat._next()
File "/app/.heroku/python/lib/python3.7/site-packages/backtrader/strategy.py", line 351, in _next
self._next_observers(minperstatus)
File "/app/.heroku/python/lib/python3.7/site-packages/backtrader/strategy.py", line 379, in _next_observers
observer._next()
File "/app/.heroku/python/lib/python3.7/site-packages/backtrader/lineiterator.py", line 280, in _next
self.next()
File "/app/.heroku/python/lib/python3.7/site-packages/backtrader/observers/broker.py", line 110, in next
self.lines.value[0] = value = self._owner.broker.getvalue()
File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_backtrader_api/alpacabroker.py", line 152, in getvalue
self.value = float(self.o.oapi.get_account().portfolio_value)
File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_trade_api/entity.py", line 29, in __getattr__
return super().__getattribute__(key)
AttributeError: 'Account' object has no attribute 'portfolio_value'
Process exited with status 1
State changed from up to crashed
Not sure if this is related to the branch changes but it's a new error for me.
Sorry I don't know exactly what's triggering it. I'm going to add more logging to find out what's happening.
For now all I've recognized is
- It consistently throws the
concurrent.futures._base.CancelledErrorwithin 2 hours of starting the app. - With the new branch my logs and activities appear to freeze after it reaches data status LIVE.
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
the disconnection error is supposed to appear - but the app should recover and continue execution.
is this what you get?
your other exception is not related. open a new issue for it
from alpaca-backtrader-api.
jtbales
commented on July 28, 2024
They don't appear to reconnect.
One thread fails and it keeps going for 20 minutes
Jun 01 11:36:06 WARNING:root:code = 1006 (connection closed abnormally [internal]), no reason
Jun 01 11:36:06 Exception in thread Thread-2:
Jun 01 11:36:06 Traceback (most recent call last):
Jun 01 11:36:06 File "/app/.heroku/python/lib/python3.7/threading.py", line 926, in _bootstrap_inner
Jun 01 11:36:06 self.run()
Jun 01 11:36:06 File "/app/.heroku/python/lib/python3.7/threading.py", line 870, in run
Jun 01 11:36:06 self._target(*self._args, **self._kwargs)
Jun 01 11:36:06 File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 341, in _t_streaming_events
Jun 01 11:36:06 streamer.run()
Jun 01 11:36:06 File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 132, in run
Jun 01 11:36:06 self.conn.run(channels)
Jun 01 11:36:06 File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_trade_api/stream2.py", line 276, in run
Jun 01 11:36:06 loop.run_until_complete(self.consume())
Jun 01 11:36:06 File "/app/.heroku/python/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
Jun 01 11:36:06 return future.result()
Jun 01 11:36:06 concurrent.futures._base.CancelledError
Then a second fails and the app stops doing anything
Jun 01 11:55:34 Exception in thread Thread-6:
Jun 01 11:55:34 Traceback (most recent call last):
Jun 01 11:55:34 File "/app/.heroku/python/lib/python3.7/threading.py", line 926, in _bootstrap_inner
Jun 01 11:55:34 self.run()
Jun 01 11:55:34 File "/app/.heroku/python/lib/python3.7/threading.py", line 870, in run
Jun 01 11:55:34 self._target(*self._args, **self._kwargs)
Jun 01 11:55:34 File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 483, in _t_streaming_prices
Jun 01 11:55:34 streamer.run()
Jun 01 11:55:34 File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 132, in run
Jun 01 11:55:34 self.conn.run(channels)
Jun 01 11:55:34 File "/app/.heroku/python/lib/python3.7/site-packages/alpaca_trade_api/stream2.py", line 276, in run
Jun 01 11:55:34 loop.run_until_complete(self.consume())
Jun 01 11:55:34 File "/app/.heroku/python/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
Jun 01 11:55:34 return future.result()
Jun 01 11:55:34 concurrent.futures._base.CancelledError
Jun 01 11:55:34 app/worker.1
Jun 01 17:05:00 heroku/worker.1 State changed from up to down
It always seems to be the case the Thread-2 and Thread-6 eventually go down, don't come back up, and when they both fail the app stops.
As you can see here from my Heroku build logs, I'm using the branches. Please verify though, I hope I'm just installing incorrectly.
-----> Installing requirements with pip
Collecting git+https://github.com/alpacahq/alpaca-backtrader-api (from -r /tmp/build_45b90d6c97a3e4ae7f0cac6ac09953e9/requirements.txt (line 7))
Cloning https://github.com/alpacahq/alpaca-backtrader-api to /tmp/pip-req-build-qxgdki86
Running command git clone -q https://github.com/alpacahq/alpaca-backtrader-api /tmp/pip-req-build-qxgdki86
Collecting git+https://github.com/alpacahq/alpaca-trade-api-python@websocket_reconnect (from -r /tmp/build_45b90d6c97a3e4ae7f0cac6ac09953e9/requirements.txt (line 8))
Cloning https://github.com/alpacahq/alpaca-trade-api-python (to revision websocket_reconnect) to /tmp/pip-req-build-rbhpty9t
Running command git clone -q https://github.com/alpacahq/alpaca-trade-api-python /tmp/pip-req-build-rbhpty9t
Running command git checkout -b websocket_reconnect --track origin/websocket_reconnect
Switched to a new branch 'websocket_reconnect'
Branch 'websocket_reconnect' set up to track remote branch 'websocket_reconnect' from 'origin'.
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
- are you using more than 1 datas in your algo? (more than 1 symbol e.g AAPL and GOOG)
- run this command:
pip freeze|grep alpacaI want to see the output - you should create a fresh install with a new virtual env installing the requirements fresh that might help
from alpaca-backtrader-api.
jtbales
commented on July 28, 2024
- Only 1 data for 1 symbol.
alpaca-backtrader-api==0.8.0
alpaca-trade-api==0.48
- I've tried removing all dependencies locally then re-installing, upgrading dependencies locally, and as for Heroku I see from the build logs that it clears the cache before installing those.
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
about this issue - I don't experience anything like this so you need to try to provide more information. try to figure out what's going on on that server because it works smoothly here with the updated alpaca-trade-api-python
from alpaca-backtrader-api.
jtbales
commented on July 28, 2024
I'll try
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
did you try to execute it locally? do you experience the same issues?
from alpaca-backtrader-api.
jtbales
commented on July 28, 2024
Yes I get the same errors eventually running locally as I do on Heroku.
from alpaca-backtrader-api.
jtbales
commented on July 28, 2024
I tried the branch and it's been stuck doing nothing for a over an hour now
...
WARNING:root:code = 1006 (connection closed abnormally [internal]), no reason
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 341, in _t_streaming_events
streamer.run()
File "/usr/local/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 132, in run
self.conn.run(channels)
File "/usr/local/lib/python3.7/site-packages/alpaca_trade_api/stream2.py", line 276, in run
loop.run_until_complete(self.consume())
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
return future.result()
concurrent.futures._base.CancelledError
('notify_fund --- Cash: 99994.36, Value: 99995.70, Fundvalue: 99995.70, Shares: 1',)
('notify_fund --- Cash: 99993.02, Value: 99995.66, Fundvalue: 99995.66, Shares: 1',)
('notify_fund --- Cash: 99993.02, Value: 99995.58, Fundvalue: 99995.58, Shares: 1',)
('notify_fund --- Cash: 99993.02, Value: 99995.50, Fundvalue: 99995.50, Shares: 1',)
WARNING:root:code = 1006 (connection closed abnormally [internal]), no reason
Exception in thread Thread-6:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 483, in _t_streaming_prices
streamer.run()
File "/usr/local/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 132, in run
self.conn.run(channels)
File "/usr/local/lib/python3.7/site-packages/alpaca_trade_api/stream2.py", line 276, in run
loop.run_until_complete(self.consume())
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
return future.result()
concurrent.futures._base.CancelledError
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
just making sure you have the most updated python-sdk?
install it like this:
pip install -U git+https://github.com/alpacahq/alpaca-trade-api-python
also, let's try one more thing. let's run the example code: sma_crossover_strategy.py
are you still getting these errors with the sample code from the repo?
also, a reminder - it is ok to get the concurrent.futures._base.CancelledError as long as the script continues its execution
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
have you tried my seggestions?
from alpaca-backtrader-api.
algoeng
commented on July 28, 2024
If the error isn't important, can we catch this exception like this:
except asyncio.CancelledError:
pass
I saw this in https://docs.aiohttp.org/en/stable/web_advanced.html#background-tasks
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
it's not that it isn't important. we overcome it.
if we make this disappear, we will not know that we have a connection issue
from alpaca-backtrader-api.
algoeng
commented on July 28, 2024
I tried you fixed by the way cuz I am running into the same issue. I had a simple code resampled data to minute intervals and after 2 minutes (two calls to my next()) I started getting this for like two minutes before turning it off:
WARNING:root:code = 1006 (connection closed abnormally [internal]), no reason
ERROR:root:error while consuming ws messages:
ERROR:root:error while consuming ws messages: server rejected WebSocket connection: HTTP 502
ERROR:root:error while consuming ws messages: server rejected WebSocket connection: HTTP 502
ERROR:root:error while consuming ws messages: server rejected WebSocket connection: HTTP 502
ERROR:root:error while consuming ws messages: [Errno 10060] Connect call failed ('35.186.171.205', 443)
ERROR:root:error while consuming ws messages: [Errno 10060] Connect call failed ('35.186.171.205', 443)
ERROR:root:error while consuming ws messages: [Errno 10060] Connect call failed ('35.186.171.205', 443)
ERROR:root:error while consuming ws messages: [Errno 10060] Connect call failed ('35.186.171.205', 443)
ERROR:root:error while consuming ws messages: code = 1006 (connection closed abnormally [internal]), no reason
ERROR:root:error while consuming ws messages: server rejected WebSocket connection: HTTP 502
ERROR:root:error while consuming ws messages: server rejected WebSocket connection: HTTP 502
ERROR:root:error while consuming ws messages: [Errno 10060] Connect call failed ('35.186.171.205', 443)
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
ERROR:root:error while consuming ws messages: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
from alpaca-backtrader-api.
algoeng
commented on July 28, 2024
This is the repro code. I hit it in about two minutes:
import alpaca_backtrader_api
import backtrader as bt
from datetime import datetime
from secrets import alpaca_paper_id, alpaca_paper_key
class TestAlgo(bt.Strategy):
def next(self):
d = self.data0
dn = d._name
dt = self.datetime.datetime(ago=0)
print(dt, dn, d.open[0], d.high[0], d.low[0], d.close[0], d.volume[0])
if __name__ == '__main__':
cerebro = bt.Cerebro()
cerebro.addstrategy(TestAlgo)
store = alpaca_backtrader_api.AlpacaStore(
key_id=alpaca_paper_id,
secret_key=alpaca_paper_key,
paper=True
)
broker = store.getbroker()
cerebro.setbroker(broker)
DataFactory = store.getdata
datakwargs = dict(
historical=False,
backfill_start=False,
)
data0 = DataFactory(dataname='SPY', **datakwargs)
cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes)
cerebro.run()
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
I see it too. I'm looking into it now.
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
are you getting this with the latest release too?
from alpaca-backtrader-api.
algoeng
commented on July 28, 2024
latest release's error looks like this:
WARNING:root:server rejected WebSocket connection: HTTP 502
Exception in thread Thread-11:
Traceback (most recent call last):
File "C:\Users\kr\Miniconda3\envs\kr\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Users\kr\Miniconda3\envs\kr\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_backtrader_api\alpacastore.py", line 483, in _t_streaming_prices
streamer.run()
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_backtrader_api\alpacastore.py", line 132, in run
self.conn.run(channels)
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_trade_api\stream2.py", line 275, in run
loop.run_until_complete(self.subscribe(initial_channels))
File "C:\Users\kr\Miniconda3\envs\kr\lib\asyncio\base_events.py", line 484, in run_until_complete
return future.result()
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_trade_api\stream2.py", line 250, in subscribe
await self.data_ws.subscribe(data_channels)
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_trade_api\stream2.py", line 104, in subscribe
await self._ensure_ws()
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_trade_api\stream2.py", line 88, in _ensure_ws
await self._connect()
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_trade_api\stream2.py", line 29, in _connect
ws = await websockets.connect(self._endpoint)
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\websockets\client.py", line 537, in __await_impl__
transport, protocol = await self._create_connection()
File "C:\Users\kr\Miniconda3\envs\kr\lib\asyncio\base_events.py", line 794, in create_connection
raise exceptions[0]
File "C:\Users\kr\Miniconda3\envs\kr\lib\asyncio\base_events.py", line 781, in create_connection
yield from self.sock_connect(sock, address)
File "C:\Users\kr\Miniconda3\envs\kr\lib\asyncio\selector_events.py", line 439, in sock_connect
return (yield from fut)
File "C:\Users\kr\Miniconda3\envs\kr\lib\asyncio\selector_events.py", line 469, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
TimeoutError: [Errno 10060] Connect call failed ('35.186.171.205', 443)
from alpaca-backtrader-api.
algoeng
commented on July 28, 2024
I just tried my live account (not paper). and got this using the latest release:
WARNING:root:code = 1006 (connection closed abnormally [internal]), no reason
Exception in thread Thread-11:
Traceback (most recent call last):
File "C:\Users\kr\Miniconda3\envs\kr\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Users\kr\Miniconda3\envs\kr\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_backtrader_api\alpacastore.py", line 483, in _t_streaming_prices
streamer.run()
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_backtrader_api\alpacastore.py", line 132, in run
self.conn.run(channels)
File "C:\Users\kr\Miniconda3\envs\kr\lib\site-packages\alpaca_trade_api\stream2.py", line 276, in run
loop.run_until_complete(self.consume())
File "C:\Users\kr\Miniconda3\envs\kr\lib\asyncio\base_events.py", line 484, in run_until_complete
return future.result()
concurrent.futures._base.CancelledError
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
so I think this is a server issue, not a package issue.
I just executed with the latest code and it's working as before again.
could you check it on your end?
from alpaca-backtrader-api.
algoeng
commented on July 28, 2024
Yeah looks to be working. But wow I have been hitting this issue every damn morning for the past week. Do you know if is there a known issue with Alpaca API endpoints? Or their stuff is just unreliable?
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
I haven't faced these issues. but if you do you should open an issue in https://github.com/alpacahq/alpaca-trade-api-python describing this problem
from alpaca-backtrader-api.
shlomiku
commented on July 28, 2024
closing due to lack of activity
from alpaca-backtrader-api.
Related Issues (20)
- Not a reliable API
- backtrader-api uses deprecated alpaca-trade-api - will it be migrated to alpaca-py?
- Documentation for creating new Alpaca is broken. HOT 1
- Trading Calender - Package Issue HOT 3
- Cannot import alpaca-backtrader-api (Python 3.9.9 MacOS) HOT 5
- backtest not going back far enough HOT 2
- Multiple datas has connection limited error?
- alpaca-trade-api uses UTC while `_clear_out_of_market_hours` assumes NY Time. HOT 1
- Alpaca AttributeError: 'Asset' has not attribute 'status'
- Data websocket error with paper trading. Error code = 1002 (protocol error)
- Multiple WebSocket Open or Proxy - for multiple strategies
- Cannot purchase Long and create a trailing stop
- The Default Value for 'stocklike'
- only getting minute data until 12pm EST everyday in historical mode HOT 1
- Multiple datas sample no longer works HOT 1
- alpaca-trade-api-python is being deprecated by alpaca-py HOT 1
- Replay with historical data only retrieve daily bars instead replaying every minute
- module 'finplot' has no attribute 'add_order' HOT 1
- error installing on Windows 11, Python 3.9 HOT 3
- Wrong timezone for DataFactory
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from alpaca-backtrader-api.