Comments (4)
I just did a test and the out-of-the-box default for this setting for both TWS and Gateway appearas to be 30 secs. So I don't know why you have 300 secs in your TWS: maybe your settings are stored on the server, so even though you may be starting a brand new instance of TWS, that setting is being overriden by the stored settings?
I don't really want to implement this. I've never found any use for it. I'm not even convinced that IB have actually implemented it properly: the various levels of buffering involved in the path from TWs/Gateway to API socket reading means that there's a very large amount of data that needs to be sent before TWS/Gateway will receive any kind of error. In fact, I really don't understand what it's supposed to mean. I've often had debugging sessions where my API client hasn't read the socket for ages, and TWS (or maybe the Java runtime) is writing messges to the logfile saying things like 'The buffer has grown to 800 KBytes", but the API is not disconnected.
So feel free to try to convince me that there is benefit in being able to configure this...
from ibc.
the value in the documentation is 30 as well, https://www.ibkrguides.com/tws/usersguidebook/configuretws/apisettings.htm?Highlight=timeout . I might have changed it manually.
In any case my rational for this enhancement is to be able to increase the timeout through a parameter in config.ini. This is mainly for a headless IB Gateway. The case I have come up with is to request large amounts of high frequency historical data. it works well if I request it one after the other (for example SPX, historical volatility and VIX 5min bars for the last 2 years) however this is slow, so i run my requests asynchronously which takes less time in total but triggers the timeout limit
If this could be available as an option, then there is no need to manually change the value manually.
Another option could be to save IB gateway settings in a container volume. But i haven't find a way separate configuration, which i want to save, from the application, which i want to refresh with every new container start-up.
from ibc.
Sorry for the delay in responding.
My view of this timeout is that if it triggers, there is potentially something badly wrong with the performance of the API client. It really ought to be possible for the client to read data from the socket fast enough to prevent this happening. The fact that it happens in the scenario you describe indicates to me that there is something you need to investigate and improve.
I ran my downloader last night to download 2 years-worth of 5-minute data for the first 9 stocks in my S&P 500 list. The downloader does everything asynchronously, making up to 20 concurrent contract details requests and up to 15 concurrent historical data requests. I set it to write the data to text files.
Here is the logfile from the run, which shows the results:
The first download starts at 00:14:12.061, and everything finishes at 00:42:15.054. That's 28mins to download 352,134 bars, so
about 209 bars per second on average. Each bar is 60-70 bytes of network data, so that's about 24 MBytes of data in total, insignificant over that period. If I repeat the run with a larger or smaller number of stocks, the throughput is about the same.
So it seems to me that something about your client program is simply not keeping up. If you're doing a lot of processing per bar, perhaps it would be better to simply read the bars from the API into some sort of list, and then process them on another thread or with async tasks. You really don't ever want the API tied up for 30 seconds or more, you could be holding up all sorts of important messages, such as order status and executions (though admittedly a program that places orders is perhaps unlikely to be doing large historical data requests simultaneously).
So I don't think you've convinced me that this is a worthwhile enhancement: it's more likely to just paper over cracks that really need to be filled.
from ibc.
thanks for your detailed and factual answer.
sorry for my late reply, but i could not spare some time to do my research.
I found the issue, and indeed it was on the client side. I'm using ib_insync, which works great. and i found that it was timing out my long running requests. so I'm able now to replicate your results by increasing this timeout.
I did find as well to address the other potential solution that I mentioned on my previous comment. this is to find a way to save settings in a separate directory, so I could save then on a docker volume. and i could not find a way to do it on ib gateway, but i realized that actually IBC can do it (so there must be a way through IB gateway, but i could not find in). This will allow me to save settings across containers which gives a lot of flexibility.
thanks for your help
from ibc.
Related Issues (20)
- Fix suggestion for RECONNECTDATA and RECONNECTACCOUNT which never worked on macOS (probably) HOT 2
- Strange condition TWS 1025 (paper) IBC 3.15.2 HOT 4
- Why IBC ended? HOT 6
- Issue with restart TWS HOT 22
- Failed to Re-Login HOT 4
- Using environment in config.ini HOT 4
- Invalid ClosedownAt setting HOT 3
- Launching gatewaystart.sh from user crontab is impossible HOT 3
- stable version HOT 12
- Unknown Username.Password/Duplicate Session? HOT 30
- how to prevent auto-login? HOT 1
- IBC unable to restart after 2.5 days of running. Happened multiple times. HOT 8
- Unable to find Java installation on Docker container HOT 2
- IBC Not selecting Second factor device 'IB Key' HOT 12
- Wrong Username/Password Bug TWS Paper Trading Automated Login Failure HOT 3
- Lock and Exit > Never lock Trader Workstation HOT 13
- Strange behaviour when using ConfigureAutoLogoffOrRestartTimeTask HOT 5
- IBRK don't open anymore with IBC HOT 2
- How to force IB Gateway to start with usfuture market data farm on? HOT 4
- java.lang.ArrayIndexOutOfBoundsException when running ibc HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ibc.