deephaven-examples / deephaven-ib Goto Github PK

View Code? Open in Web Editor NEW

59.0 59.0 24.0 5.58 MB

An Interactive Brokers integration for Deephaven

License: Apache License 2.0

Shell 0.37% Dockerfile 0.65% Python 98.54% Makefile 0.35% CSS 0.10%

deephaven-ib's People

Contributors

Stargazers

Watchers

deephaven-ib's Issues

Invalid account code error

ERROR:ibapi.wrapper:ERROR 5 321 Error validating request.-'cq' : cause - Invalid account code

requestFA(2) for PROFILE does not work as documented by IB

This error message appears. It does not seem to negatively impact the program's execution.

ERROR:ibapi.wrapper:ERROR 2147483647 321 Error validating request.-'ce' : cause - Non-existent FA data operation request.

This error message seems to result from the following line in tws_client.py:

self.requestFA(2)  # request PROFILE.  See FaDataTypeEnum.

I do note that the accounts_allocation_profiles table does not seem to be populated. See #5.

`IbTwsClient` is sleeping to avoid race conditions upon connection

The ibapi library seems to have race conditions when connecting to TWS. Sleeps have been added to IbTwsClient.connect() to avoid the races. Once the problem is resolved on the IB side, the sleeps should be removed.

Incorrect generic ticks list

Unmapped value: 2157

ERROR:root:Unmapped value.  Please file an issue at https://github.com/deephaven-examples/deephaven-ib/issues: '2157'
  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/client.py", line 263, in run
    self.decoder.interpret(fields)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/decoder.py", line 1294, in interpret
    self.interpretWithSignature(fields, handleInfo)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/decoder.py", line 1275, in interpretWithSignature
    method(*args)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/tws_client.py", line 408, in error
    [reqId, errorCode, map_values(errorCode, _error_code_message_map), errorString,
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/tablewriter.py", line 92, in map_values
    msg = f"Unmapped value.  Please file an issue at https://github.com/deephaven-examples/deephaven-ib/issues: '{value}'\n{trace_str()}\n-----"
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/trace.py", line 9, in trace_str
    return "".join(traceback.format_stack())

Invalid account code error.

ERROR:ibapi.wrapper:ERROR 5 321 Error validating request.-'cq' : cause - Invalid account code

Support full Deephaven system

Add support to easily run a full Deephaven system. This functionality was previously removed.

The contents of `accounts_allocation_profiles` need to be validated

The contents of accounts_allocation_profiles need to be validated. The account I am using for validation is not an advisor account, so someone else needs to check the table contents.

Error importing deephaven_ib

Importing the deephaven_ib module fails while attempting to import the deephaven._jcompat module (with DH 0.12.0):

r-Scheduler-Serial-1 | i.d.s.s.SessionState      | Internal Error 'e743dd26-a00c-413d-8a46-83696bc77504' java.lang.RuntimeException: Error in Python interpreter:
Type: <class 'ModuleNotFoundError'>
Value: No module named 'deephaven._jcompat'
Line: 18
Namespace: <module>
File: /opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_query_inputs.py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/__init__.py", line 9, in <module>
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_query_inputs.py", line 18, in <module>

        at org.jpy.PyLib.executeCode(PyLib.java:-2)
        at org.jpy.PyObject.executeCode(PyObject.java:138)

Instead of using ibkr real time data I want to use the real time data from some place else: tda-api

Instead of
ticks_bid_ask = client.tables["ticks_bid_ask"]
client.request_tick_data_realtime(rc, dhib.TickDataType.BID_ASK)

I want to use tda-api level one streaming data: https://tda-api.readthedocs.io/en/latest/streaming.html

how can tda-api be integrated into Deephaven docker and
tick_bid_ask be replace by tda-api streaming data

The contents of `news_bulletins` need to be validated

During testing, there were no news bulletins, so the contents of news_bulletins could not be validated. Once there is a bulletin, this table can be asserted as correct.

Update requirements.txt use reflect published Deephaven and jpy packages

Once Deephaven publishes standard pypi packages, requirements.txt needs to be updated to reflect this. Files also need to be updated when Deephaven switches to using the standard jpy module.

`request_tick_data_realtime` column type mismatch

from typing import Dict

from deephaven.time import to_datetime
from ibapi.contract import Contract
from ibapi.order import Order

import deephaven_ib as dhib

###########################################################################
# WARNING: THIS SCRIPT EXECUTES TRADES!! ONLY USE ON PAPER TRADING ACCOUNTS
###########################################################################

print("==============================================================================================================")
print("==== Create a client and connect.")
print("==== ** Accept the connection in TWS **")
print("==============================================================================================================")

client = dhib.IbSessionTws(host="host.docker.internal", port=7497, client_id=0, download_short_rates=False, read_only=False)
print(f"IsConnected: {client.is_connected()}")

client.connect()
print(f"IsConnected: {client.is_connected()}")


print("==============================================================================================================")
print("==== Request tick data.")
print("==============================================================================================================")

contract = Contract()
contract.symbol = "GOOG"
contract.secType = "STK"
contract.currency = "USD"
contract.exchange = "SMART"

rc = client.get_registered_contract(contract)
print(contract)

now = to_datetime("2021-01-01T00:00:00 NY")

# client.request_tick_data_historical(rc, dhib.TickDataType.MIDPOINT, 100, start=now)
# client.request_tick_data_historical(rc, dhib.TickDataType.MIDPOINT, 100, end=now)
# client.request_tick_data_realtime(rc, dhib.TickDataType.MIDPOINT)

client.request_tick_data_realtime(rc, dhib.TickDataType.BID_ASK)  #TODO: broken ***

# client.request_tick_data_historical(rc, dhib.TickDataType.LAST, 100, start=now)
# client.request_tick_data_historical(rc, dhib.TickDataType.LAST, 100, end=now)
# client.request_tick_data_realtime(rc, dhib.TickDataType.LAST)

yields:

ERROR:root:TableWriter column type and value type are mismatched: column_name=AskSize column_type=long value_type=<class 'float'> value=900.0
  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/client.py", line 263, in run
    self.decoder.interpret(fields)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/decoder.py", line 1379, in interpret
    handleInfo.processMeth(self, iter(fields))
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/decoder.py", line 1086, in processTickByTickMsg
    askSize, tickAttribBidAsk)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/tws_client.py", line 886, in tickByTickBidAsk
    self._table_writers["ticks_bid_ask"].write_row([reqId, *logger_hist_tick_bid_ask.vals(t)])
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/tablewriter.py", line 76, in write_row
    self._check_logged_value_types(values)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/tablewriter.py", line 60, in _check_logged_value_types
    f"TableWriter column type and value type are mismatched: column_name={n} column_type={t} value_type={type(v)} value={v}\n{trace_str()}\n-----")
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/trace.py", line 9, in trace_str
    return "".join(traceback.format_stack())

CI is not publishing whls to pypi

CI is failing to publish whls to pypi.

Inconsistent use of `Account` and `AcctNumber` column names

Some tables use Account and others use AcctNumber for the same column values. Naming should be consistent.

Publish artifacts to Pypi

There is currently a github action to build whl files. This action should be made to publish to pypi.

Clean up `_query_inputs.py`

_query_inputs.py contains very ugly code to get around this open Deephaven bug. Once the bug is resolved, the code should be cleaned up.

See: deephaven/deephaven-core#1072

Port deephaven-ib to the new deephaven2 API

deephaven-ib needs to be ported to the new deephaven2 API

Unmapped value: 10190

ERROR:root:Unmapped value.  Please file an issue at https://github.com/deephaven-examples/deephaven-ib/issues: '10190'
  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/client.py", line 263, in run
    self.decoder.interpret(fields)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/decoder.py", line 1294, in interpret
    self.interpretWithSignature(fields, handleInfo)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/decoder.py", line 1275, in interpretWithSignature
    method(*args)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/tws_client.py", line 408, in error
    [reqId, errorCode, map_values(errorCode, _error_code_message_map), errorString,
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/tablewriter.py", line 92, in map_values
    msg = f"Unmapped value.  Please file an issue at https://github.com/deephaven-examples/deephaven-ib/issues: '{value}'\n{trace_str()}\n-----"
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/trace.py", line 9, in trace_str
    return "".join(traceback.format_stack())

Split `ticks_string` into multiple output tables

ticks_string is a dumping ground of data that should be split into multiple tables. These should include at least:

A stringset of decoded exchange codes. See #3 .
Decoded real-time volume, including RT_TRD_VOLUME and RT_VOLUME (https://interactivebrokers.github.io/tws-api/tick_types.html#rt_volume)
Timestamps

Ubuntu22.04 - tws client connect error Python3.10 local install no docker

Server started on port 10000
import deephaven-ib as dhib
r-Scheduler-Serial-1 | i.d.s.s.SessionState | Internal Error '29f24170-7764-4a2b-82a8-ee995663e89f' java.lang.RuntimeException: Error in Python interpreter:
import deephaven_ib as dhib
IB_PORT=7496
HOST="localhost"
client = dhib.IbSessionTws(host=HOST, port=PORT, read_only=True)
client.connect()
r-Scheduler-Serial-1 | i.d.s.s.SessionState | Internal Error '63d7f573-33ee-4ffd-a1a1-c837e4253b17' java.lang.RuntimeException: Error in Python interpreter:
client = dhib.IbSessionTws(host=HOST, port=IB_PORT, read_only=True)
client.connect()
ERROR:ibapi.wrapper:ERROR -1 2104 Market data farm connection is OK:usopt
ERROR:ibapi.wrapper:ERROR -1 2104 Market data farm connection is OK:usfarm
ERROR:ibapi.wrapper:ERROR -1 2106 HMDS data farm connection is OK:ushmds
ERROR:ibapi.wrapper:ERROR -1 2158 Sec-def data farm connection is OK:secdefil
ERROR:ibapi.wrapper:ERROR 31 321 Error validating request.-'bQ' : cause - 'BAG' isn't supported for contract data request. Please enter a valid security type

Exception in thread IbTwsClient:
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/ibapi-10.19.1-py3.10.egg/ibapi/client.py", line 263, in run
File "/usr/local/lib/python3.10/dist-packages/ibapi-10.19.1-py3.10.egg/ibapi/decoder.py", line 1387, in interpret
File "/usr/local/lib/python3.10/dist-packages/ibapi-10.19.1-py3.10.egg/ibapi/decoder.py", line 1277, in processErrorMsg
File "/home/ss317/.local/lib/python3.10/site-packages/deephaven_ib/_tws/tws_client.py", line 428, in error
self.contract_registry.add_error_data(req_id=reqId, error_string=errorString)
File "/home/ss317/.local/lib/python3.10/site-packages/deephaven_ib/_tws/contract_registry.py", line 110, in add_error_data
event.set()
AttributeError: 'NoneType' object has no attribute 'set'

====================================

local install on Ubuntu22.04
python3 version = python3.10.6

dev and release docker examples are broken

The ./docker/{dev,release} examples are broken.

Exception when canceling orders

IB broke their API and did not document the change.

from typing import Dict

from deephaven.time import to_datetime
from ibapi.contract import Contract
from ibapi.order import Order

import deephaven_ib as dhib
from deephaven.updateby import ema_time_decay
from deephaven import time_table
from deephaven.plot import Figure


###########################################################################
# WARNING: THIS SCRIPT EXECUTES TRADES!! ONLY USE ON PAPER TRADING ACCOUNTS
###########################################################################

print("==============================================================================================================")
print("==== Create a client and connect.")
print("==== ** Accept the connection in TWS **")
print("==============================================================================================================")

client = dhib.IbSessionTws(host="host.docker.internal", port=7497, client_id=0, download_short_rates=False, read_only=False)
print(f"IsConnected: {client.is_connected()}")

client.connect()
print(f"IsConnected: {client.is_connected()}")

## Setup

account = "DU4943848"
ticks_bid_ask = client.tables["ticks_bid_ask"]
orders_submitted = client.tables["orders_submitted"]
orders_status = client.tables["orders_status"]
positions = client.tables["accounts_positions"].where("Account = account")

print("==============================================================================================================")
print("==== Request data.")
print("==============================================================================================================")


contract = Contract()
contract.symbol = "GOOG"
contract.secType = "STK"
contract.currency = "USD"
contract.exchange = "SMART"

rc = client.get_registered_contract(contract)
id = rc.contract_details[0].contract.conId
print(f"Registered contract: id={id} rc={rc}")

order_sell = Order()
order_sell.account = account
order_sell.action = "SELL"
order_sell.orderType = "LIMIT"
order_sell.totalQuantity = 100
order_sell.lmtPrice = 101

order = client.order_place(rc, order_sell)
order.cancel()

r-Scheduler-Serial-1 | i.d.s.s.SessionState      | Internal Error 'ffda8bbd-3f96-4c13-bc7d-825605890ce5' java.lang.RuntimeException: Error in Python interpreter:
Type: <class 'TypeError'>
Value: cancelOrder() missing 1 required positional argument: 'manualCancelOrderTime'
Line: 1169
Namespace: order_cancel
File: /opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/__init__.py
Traceback (most recent call last):
  File "<string>", line 59, in <module>
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/__init__.py", line 307, in cancel
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/__init__.py", line 1169, in order_cancel

        at org.jpy.PyLib.executeCode(PyLib.java:-2)
        at org.jpy.PyObject.executeCode(PyObject.java:138)
        at io.deephaven.engine.util.PythonEvaluatorJpy.evalScript(PythonEvaluatorJpy.java:73)
        at io.deephaven.integrations.python.PythonDeephavenSession.lambda$evaluate$1(PythonDeephavenSession.java:185)
        at io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:49)
        at io.deephaven.integrations.python.PythonDeephavenSession.evaluate(PythonDeephavenSession.java:184)
        at io.deephaven.engine.util.AbstractScriptSession.lambda$evaluateScript$1(AbstractScriptSession.java:146)
        at io.deephaven.engine.context.ExecutionContext.lambda$apply$0(ExecutionContext.java:117)
        at io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:128)
        at io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:116)
        at io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:146)
        at io.deephaven.engine.util.DelegatingScriptSession.evaluateScript(DelegatingScriptSession.java:87)
        at io.deephaven.engine.util.ScriptSession.evaluateScript(ScriptSession.java:113)
        at io.deephaven.server.console.ConsoleServiceGrpcImpl.lambda$executeCommand$8(ConsoleServiceGrpcImpl.java:170)
        at io.deephaven.server.session.SessionState$ExportBuilder.lambda$submit$2(SessionState.java:1308)
        at io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:856)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at io.deephaven.server.runner.DeephavenApiServerModule$ThreadFactory.lambda$newThread$0(DeephavenApiServerModule.java:156)
        at java.lang.Thread.run(Thread.java:829)

Support the IB Client Portal REST Web API

Some users are requesting support for the IB Client Portal REST Web API in addition to the TWS API.

https://www.interactivebrokers.com/en/trading/ib-api.php
https://github.com/Voyz/ibeam

Possible `OrderIdRequest.get()` deadlock

EFAULT.refreshThread | .AsyncClientErrorNotifier | Error in table update: io.deephaven.engine.table.impl.select.FormulaEvaluationException: In formula: NumNewOrders = longCast(update_orders.call(ContractId, PredLow, PredHigh, BuyOrder, SellOrder))
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.applyFormulaPerItem(Formula.java:187)
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.lambda$fillChunkHelper$4(Formula.java:176)
        at io.deephaven.engine.rowset.RowSequence.lambda$forAllRowKeys$0(RowSequence.java:179)
        at io.deephaven.engine.rowset.impl.singlerange.SingleRangeMixin.forEachRowKey(SingleRangeMixin.java:17)
        at io.deephaven.engine.rowset.RowSequence.forAllRowKeys(RowSequence.java:178)
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.fillChunkHelper(Formula.java:174)
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.fillChunk(Formula.java:147)
        at io.deephaven.engine.table.impl.select.Formula.getChunk(Formula.java:161)
        at io.deephaven.engine.table.impl.sources.ViewColumnSource.getChunk(ViewColumnSource.java:239)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.doApplyUpdate(SelectColumnLayer.java:424)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.lambda$doSerialApplyUpdate$1(SelectColumnLayer.java:237)
        at io.deephaven.engine.util.systemicmarking.SystemicObjectTracker.executeSystemically(SystemicObjectTracker.java:56)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.doSerialApplyUpdate(SelectColumnLayer.java:236)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer$1.lambda$onAllRequiredColumnsCompleted$1(SelectColumnLayer.java:193)
        at io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer$ImmediateJobScheduler.submit(SelectAndViewAnalyzer.java:632)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer$1.onAllRequiredColumnsCompleted(SelectColumnLayer.java:191)
        at io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer$SelectLayerCompletionHandler.onLayerCompleted(SelectAndViewAnalyzer.java:473)
        at io.deephaven.engine.table.impl.select.analyzers.BaseLayer.applyUpdate(BaseLayer.java:75)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.applyUpdate(SelectColumnLayer.java:137)
        at io.deephaven.engine.table.impl.SelectOrUpdateListener.onUpdate(SelectOrUpdateListener.java:90)
        at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.lambda$run$0(InstrumentedTableUpdateListener.java:37)
        at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRunInternal(InstrumentedTableListenerBase.java:293)
        at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRun(InstrumentedTableListenerBase.java:271)
        at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.run(InstrumentedTableUpdateListener.java:37)
        at io.deephaven.engine.updategraph.NotificationQueue$Notification.runInContext(NotificationQueue.java:60)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.runNotification(UpdateGraphProcessor.java:1292)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor$QueueNotificationProcessor.doWork(UpdateGraphProcessor.java:1453)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNormalNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1171)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1116)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.lambda$doRefresh$10(UpdateGraphProcessor.java:1730)
        at io.deephaven.util.locks.FunctionalLock.doLocked(FunctionalLock.java:31)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.doRefresh(UpdateGraphProcessor.java:1720)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshAllTables(UpdateGraphProcessor.java:1707)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshTablesAndFlushNotifications(UpdateGraphProcessor.java:1561)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor$1.run(UpdateGraphProcessor.java:278)
caused by:
java.lang.RuntimeException: Error in Python interpreter:
Type: <class 'Exception'>
Value: OrderIdRequest.get() timed out after 60.0 sec.  A possible deadlock or TWS bug was detected!  You may be able to avoid this problem by using a different OrderIdStrategy.  Please create an issue at https://github.com/deephaven-examples/deephaven-ib/issues containing this error message
Stack Traces:
<_MainThread(MainThread, started 139983437600512)>Thread stack not found: thread=139983437600512
<Thread(DeadlockMonitor, started daemon 139980511135488)>  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/threading.py", line 34, in _run
    time.sleep(self.sleep_sec)

<Thread(OrderIdEventQueueRetry, started daemon 139978650887936)>  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/order_id_queue.py", line 95, in _retry
    sleep(0.01)

<_DummyThread(Dummy-2, started daemon 139980520056576)>Thread stack not found: thread=139980520056576
<EReader(Thread-1, started daemon 139978642233088)>  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/reader.py", line 35, in run
    data = self.conn.recvMsg()
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/connection.py", line 94, in recvMsg
    buf = self._recvAllMsg()
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/connection.py", line 119, in _recvAllMsg
    buf = self.socket.recv(4096)

<Thread(IbTwsClient, started daemon 139978633840384)>  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/client.py", line 263, in run
    self.decoder.interpret(fields)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/decoder.py", line 1379, in interpret
    handleInfo.processMeth(self, iter(fields))
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi-10.16.1-py3.7.egg/ibapi/decoder.py", line 504, in processExecutionDataMsg
    self.wrapper.execDetails(reqId, contract, execution)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/tws_client.py", line 1009, in execDetails
    self.contract_registry.request_contract_details_nonblocking(contract)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/contract_registry.py", line 143, in request_contract_details_nonblocking
    self._request_contract_details(contract=contract)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/contract_registry.py", line 202, in _request_contract_details
    req_id = self._client.request_id_manager.next_id()
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/requests.py", line 19, in next_id
    with self._lock:
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/threading.py", line 131, in __enter__
    self.acquire()
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/threading.py", line 113, in acquire
    ret = self.lock.acquire(blocking)



Line: 57
Namespace: get
File: /opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/order_id_queue.py
Traceback (most recent call last):
  File "<string>", line 103, in update_orders
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/__init__.py", line 1148, in order_place
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/tws_client.py", line 942, in next_order_id
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/requests.py", line 27, in next_order_id
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/order_id_queue.py", line 57, in get

        at org.jpy.PyLib.callAndReturnObject(PyLib.java:-2)
        at org.jpy.PyObject.callMethod(PyObject.java:432)
        at io.deephaven.engine.util.PythonScopeJpyImpl$CallableWrapper.call(PythonScopeJpyImpl.java:99)
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.applyFormulaPerItem(Formula.java:185)
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.lambda$fillChunkHelper$4(Formula.java:176)
        at io.deephaven.engine.rowset.RowSequence.lambda$forAllRowKeys$0(RowSequence.java:179)
        at io.deephaven.engine.rowset.impl.singlerange.SingleRangeMixin.forEachRowKey(SingleRangeMixin.java:17)
        at io.deephaven.engine.rowset.RowSequence.forAllRowKeys(RowSequence.java:178)
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.fillChunkHelper(Formula.java:174)
        at io.deephaven.temp.c1321943764987228486v55_0.Formula.fillChunk(Formula.java:147)
        at io.deephaven.engine.table.impl.select.Formula.getChunk(Formula.java:161)
        at io.deephaven.engine.table.impl.sources.ViewColumnSource.getChunk(ViewColumnSource.java:239)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.doApplyUpdate(SelectColumnLayer.java:424)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.lambda$doSerialApplyUpdate$1(SelectColumnLayer.java:237)
        at io.deephaven.engine.util.systemicmarking.SystemicObjectTracker.executeSystemically(SystemicObjectTracker.java:56)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.doSerialApplyUpdate(SelectColumnLayer.java:236)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer$1.lambda$onAllRequiredColumnsCompleted$1(SelectColumnLayer.java:193)
        at io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer$ImmediateJobScheduler.submit(SelectAndViewAnalyzer.java:632)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer$1.onAllRequiredColumnsCompleted(SelectColumnLayer.java:191)
        at io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer$SelectLayerCompletionHandler.onLayerCompleted(SelectAndViewAnalyzer.java:473)
        at io.deephaven.engine.table.impl.select.analyzers.BaseLayer.applyUpdate(BaseLayer.java:75)
        at io.deephaven.engine.table.impl.select.analyzers.SelectColumnLayer.applyUpdate(SelectColumnLayer.java:137)
        at io.deephaven.engine.table.impl.SelectOrUpdateListener.onUpdate(SelectOrUpdateListener.java:90)
        at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.lambda$run$0(InstrumentedTableUpdateListener.java:37)
        at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRunInternal(InstrumentedTableListenerBase.java:293)
        at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRun(InstrumentedTableListenerBase.java:271)
        at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.run(InstrumentedTableUpdateListener.java:37)
        at io.deephaven.engine.updategraph.NotificationQueue$Notification.runInContext(NotificationQueue.java:60)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.runNotification(UpdateGraphProcessor.java:1292)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor$QueueNotificationProcessor.doWork(UpdateGraphProcessor.java:1453)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNormalNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1171)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1116)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.lambda$doRefresh$10(UpdateGraphProcessor.java:1730)
        at io.deephaven.util.locks.FunctionalLock.doLocked(FunctionalLock.java:31)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.doRefresh(UpdateGraphProcessor.java:1720)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshAllTables(UpdateGraphProcessor.java:1707)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshTablesAndFlushNotifications(UpdateGraphProcessor.java:1561)
        at io.deephaven.engine.updategraph.UpdateGraphProcessor$1.run(UpdateGraphProcessor.java:278)

Timestamps for historical data are parsed incorrectly

The timestamps for historical data bars are incorrect — it looks like the seconds of the timestamp are the date, encoded as an integer:

from ibapi.contract import Contract

import deephaven_ib as dhib

print("==============================================================================================================")
print("==== ** Accept the connection in TWS **")
print("==============================================================================================================")

client = dhib.IbSessionTws(host="host.docker.internal", port=7497, download_short_rates=False)
client.connect()

# Make bars_historical table a global variable so it is displayed in the user interface
bars_historical = client.tables['bars_historical']

c = Contract()
c.symbol = 'SPY'
c.secType = 'STK'
c.exchange = 'SMART'
c.currency = 'USD'

rc = client.get_registered_contract(c)
print(rc)

client.request_bars_historical(
    rc,
    duration=dhib.Duration.days(252),
    bar_size=dhib.BarSize.DAY_1,
    bar_type=dhib.BarDataType.TRADES,
)


bars_historical_test = bars_historical.view([
    'Symbol', 'Timestamp',
    'Seconds = (long) (millis(Timestamp)/1000)',
    'Date = java.time.LocalDate.parse(Long.toString(Seconds), java.time.format.DateTimeFormatter.BASIC_ISO_DATE).toString()',
    'Open', 'High', 'Low', 'Close', 'Volume'
])

Update to Deephaven 0.16

Potential deadlock in `order_place`

I've seen frequent cases where the code seems to deadlock in the req_id = self._client.next_order_id() line below. Deadlock monitoring has been added to try to find the problem.

    def order_place(self, contract: RegisteredContract, order: Order) -> Request:
        """Places an order.

        Args:
            contract (RegisteredContract): contract to place an order on
            order (Order): order to place
        """
        self._assert_connected()

        if contract.is_multi():
            raise Exception(
                f"RegisteredContracts with multiple contract details are not supported for orders: {contract}")

        req_id = self._client.next_order_id()
        cd = contract.contract_details[0]
        self._client.log_request(req_id, "PlaceOrder", cd.contract, {"order": f"Order({order})"})
        self._client.placeOrder(req_id, cd.contract, order)
        return Request(request_id=req_id, cancel_func=self.order_cancel)

Make IB API version easier to update

Make the IB API version easier to update. The version should be associated with IB_VERSION, so that it is easy to do a find and replace.

Sphinx is broken because of a circular dependency in DH

Sphinx is failing to generate documentation because of a circular dependency in DH.

The following code is from dh_sphinx.py. Here you will notice that setup_sphinx_environment is a function to setup and configure the sphinx environment as well as the jpy jvm.

import os
import pkgutil
import shutil
import sys
from pathlib import Path

import jpy
from deephaven.start_jvm import start_jvm

def setup_sphinx_environment():
    # add the deephaven-ib path
    new_python_path = Path(os.path.realpath(__file__)).parents[2].joinpath("src")
    sys.path.append(str(new_python_path))

    # start the jvm so that deephaven can be loaded
    if not jpy.has_jvm():
        os.environ['JAVA_VERSION'] = '11'
        start_jvm(devroot="/tmp", workspace="/tmp", propfile='dh-defaults.prop',
                  java_home=os.environ.get('JDK_HOME', None),
                  jvm_classpath="/opt/deephaven/server/lib/*", skip_default_classpath=True)

As of DH v0.12.0, the following exception results. Note that here the from deephaven.start_jvm import start_jvm import is triggering jpy calls, but these calls are before the jpy JVM has been configured.

Traceback (most recent call last):
  File "/opt/deephaven-venv/lib/python3.7/site-packages/sphinx/config.py", line 332, in eval_config_file
    exec(code, namespace)
  File "/build/deephaven-ib/sphinx/source/conf.py", line 79, in <module>
    import dh_sphinx
  File "/build/deephaven-ib/sphinx/lib/dh_sphinx.py", line 8, in <module>
    from deephaven.start_jvm import start_jvm
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven/__init__.py", line 12, in <module>
    from .table import SortDirection, AsOfMatchRule
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven/table.py", line 12, in <module>
    from deephaven import DHError, dtypes
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven/dtypes.py", line 17, in <module>
    _JQstType = jpy.get_type("io.deephaven.qst.type.Type")
RuntimeError: jpy: No JVM available.

Update README.md note when DHaaL supports more architectures

Update the README note about local installs when DHaaL supports more architectures.

IBKR README: Add comment to each section saying "look for evidence in table XXXX"

In the readme there is a list of tables. Good.

However, after I run respective sections of the readme, I would like to be directed to tables that can show evidence of success of the script I just ran.

So, after the readme section about getting market data htat has this bit of code:
`from ibapi.contract import Contract

import deephaven_ib as dhib

Use delayed market data if you do not have access to real-time

client.set_market_data_type(dhib.MarketDataType.DELAYED)

client.set_market_data_type(dhib.MarketDataType.REAL_TIME)

c = Contract()
c.symbol = 'AAPL'
c.secType = 'STK'
c.exchange = 'SMART'
c.currency = 'USD'

rc = client.get_registered_contract(c)
print(rc)

client.request_market_data(rc)
client.request_tick_data_realtime(rc, dhib.TickDataType.BID_ASK)
client.request_tick_data_realtime(rc, dhib.TickDataType.LAST)
client.request_tick_data_realtime(rc, dhib.TickDataType.MIDPOINT)`

Then perhaps add a comment:

"You will see market data flowing in real-time in the table ticks_price and ticks_bid_ask, for example."

I would add similar commments after the Request-News code snippet and Order-Management code snippet.

In contrast, after the "Query and do math", the script brings the user to the bars_joined table....
And after the plotting script, the plot pops up in the wed IDE as evidence.

`account_positions` table can have duplicate entries

As reported in https://deephavencommunity.slack.com/archives/C036HT73T43/p1694639036590999, it is possible for the account_positions table to have duplicate entries.

This appears to happen because position data is requested for the base account as well as all accounts.

At the same time, the raw data is aggregated using .last_by(["RequestId", "Account", "ModelCode", "ContractId"]).

Should the data be aggregated using only .last_by(["Account", "ModelCode", "ContractId"])?
Should the account for the all case be rewritten to ALL?
Which other account_* tables need a similar code change?

Supporting files:
requests.csv

positions.csv

from ibapi.contract import Contract
from ibapi.order import Order
import deephaven_ib as dhib
from deephaven import time_table
from deephaven.plot import Figure
from deephaven.plot.selectable_dataset import one_click
from deephaven.plot import PlotStyle


print("========== Create a client and connect.=========")
# Socket Port: TWS:: live:7496 paper:7497 | IB:: live:4001 paper:4002
client = dhib.IbSessionTws(host="host.docker.internal", port=4002,client_id=0, download_short_rates=False, 
                           read_only=False)
print(f"IsConnected: {client.is_connected()}")
client.connect()
print(f"IsConnected: {client.is_connected()}")

# Setup
account = "XXXXXXXXXXXXXX"
max_position_dollars = 200000.0 #per security
ema_t = "00:02:00"

errors = client.tables["errors"]
requests = client.tables["requests"]
positions = client.tables["accounts_positions"].where("Account = account")
ticks_bid_ask = client.tables["ticks_bid_ask"]
orders_submitted = client.tables["orders_submitted"].where("Account = account")
orders_status = client.tables["orders_status"]
orders_exec_details = client.tables["orders_exec_details"].where( "Account = account")
pnl = client.tables["accounts_pnl"].where("Account = account")
tmp= client.tables["accounts_summary"].where("Account=account")

`ticks_string` should map exchange codes to exchange names

The ticks_string table contains raw exchange codes, instead of decrypted exchange names.

It appears like the mapping can be obtained via reqSmartComponents.
https://interactivebrokers.github.io/tws-api/classIBApi_1_1EClient.html#aa5de1b3f68e143c6b10d5eda83d2ffe3

See: https://interactivebrokers.github.io/tws-api/smart_components.html

Class cast exception when logging

#Connect to the client
API_PORT = 7497

import deephaven_ib as dhib
client = dhib.IbSessionTws(host="host.docker.internal", port=API_PORT, read_only=False)
client.connect()

if client.is_connected():
    print('Client connected!')
else:
    raise RuntimeError("Client not connected!")

Causes:

r-Scheduler-Serial-1 | i.d.s.s.SessionState      | Internal Error 'f03f716e-a54e-40de-bad9-35db10230bfd' java.lang.RuntimeException: Error in Python interpreter:
Type: <class 'deephaven.dherror.DHError'>
Value: failed to write a row. : ValueError: cannot convert a Python 'list' to a Java 'java.lang.Object'
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ibapi-10.16.1-py3.10.egg/ibapi/client.py", line 159, in connect
    self.conn.sendMsg(msg2)
  File "/usr/local/lib/python3.10/dist-packages/ibapi-10.16.1-py3.10.egg/ibapi/connection.py", line 76, in sendMsg
    nSent = self.socket.send(msg)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/deephaven/table_factory.py", line 186, in write_row
    self._j_table_writer.logRowPermissive(values)
ValueError: cannot convert a Python 'list' to a Java 'java.lang.Object'

Line: 188
Namespace: write_row
File: /usr/local/lib/python3.10/dist-packages/deephaven/table_factory.py
Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "/usr/local/lib/python3.10/dist-packages/deephaven_ib/__init__.py", line 491, in connect
  File "/usr/local/lib/python3.10/dist-packages/deephaven_ib/_tws/tws_client.py", line 325, in connect
  File "/usr/local/lib/python3.10/dist-packages/ibapi-10.16.1-py3.10.egg/ibapi/client.py", line 199, in connect
  File "/usr/local/lib/python3.10/dist-packages/deephaven_ib/_tws/tws_client.py", line 422, in error
  File "/usr/local/lib/python3.10/dist-packages/deephaven_ib/_internal/tablewriter.py", line 90, in write_row
  File "/usr/local/lib/python3.10/dist-packages/deephaven_ib/_internal/tablewriter.py", line 83, in write_row
  File "/usr/local/lib/python3.10/dist-packages/deephaven/table_factory.py", line 188, in write_row

        at org.jpy.PyLib.executeCode(PyLib.java:-2)
        at org.jpy.PyObject.executeCode(PyObject.java:138)
        at io.deephaven.engine.util.PythonEvaluatorJpy.evalScript(PythonEvaluatorJpy.java:73)
        at io.deephaven.integrations.python.PythonDeephavenSession.lambda$evaluate$1(PythonDeephavenSession.java:186)
        at io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:49)
        at io.deephaven.integrations.python.PythonDeephavenSession.evaluate(PythonDeephavenSession.java:185)
        at io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:166)
        at io.deephaven.engine.util.DelegatingScriptSession.evaluateScript(DelegatingScriptSession.java:81)
        at io.deephaven.engine.util.ScriptSession.evaluateScript(ScriptSession.java:106)
        at io.deephaven.server.console.ConsoleServiceGrpcImpl.lambda$executeCommand$8(ConsoleServiceGrpcImpl.java:168)
        at io.deephaven.server.session.SessionState$ExportBuilder.lambda$submit$2(SessionState.java:1299)
        at io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:847)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at io.deephaven.server.runner.DeephavenApiServerModule$ThreadFactory.lambda$newThread$0(DeephavenApiServerModule.java:162)
        at java.lang.Thread.run(Thread.java:829)

README images do not render in pypi

Images in the readme do not render on pypi

Test issue

This is a test issue

IBKR README #2: '321 error' when creating connection to IB TWS #1243

I get the following 321 errors each time I make the connection.

I run this:


client = dhib.IbSessionTws(host="host.docker.internal", port=7497, read_only=False)
client.connect()

Though the connection is successful, I get these errors:

ERROR:ibapi.wrapper:ERROR 2147483647 321 Error validating request.-'ce' : cause - FA data operations ignored for non FA customers.
ERROR:ibapi.wrapper:ERROR 7 321 Error validating request.-'cq' : cause - Invalid account code

Potential deadlock in `get_registered_contract`

See #8.

I've seen cases where the code seems to deadlock in the cd = self._client.contract_registry.request_contract_details_blocking(contract) line below.

    def get_registered_contract(self, contract: Contract) -> RegisteredContract:
        """Gets a contract that has been registered in the framework.  The registered contract is confirmed to
        exist in the IB system and contains a complete description of the contract.

        Args:
            contract (Contract): contract to search for

        Returns:
            A contract that has been registered with deephaven-ib.

        Raises:
              Exception: problem executing action.
        """

        self._assert_connected()
        cd = self._client.contract_registry.request_contract_details_blocking(contract)
        return RegisteredContract(query_contract=contract, contract_details=cd)

Deadlock monitoring returns the following.

ERROR:ibapi.wrapper:ERROR 9 200 No security definition has been found for the request
r-Scheduler-Serial-1 | i.d.s.s.SessionState      | Internal Error '4936e794-be6c-49b5-8ca2-e4e51b937818' java.lang.RuntimeException: Error in Python interpreter:
Type: <class 'Exception'>
Value: ContractRegistry.request_contract_details_blocking() timed out after 120.0 sec.
Line: 181
Namespace: request_contract_details_blocking
File: /opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/contract_registry.py
Traceback (most recent call last):
  File "<string>", line 232, in <module>
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/__init__.py", line 643, in get_registered_contract
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/contract_registry.py", line 181, in request_contract_details_blocking

        at org.jpy.PyLib.executeCode(PyLib.java:-2)
        at org.jpy.PyObject.executeCode(PyObject.java:138)
        at io.deephaven.engine.util.PythonEvaluatorJpy.evalScript(PythonEvaluatorJpy.java:56)
        at io.deephaven.engine.util.PythonDeephavenSession.lambda$evaluate$1(PythonDeephavenSession.java:188)
        at io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:46)
        at io.deephaven.engine.util.PythonDeephavenSession.evaluate(PythonDeephavenSession.java:187)
        at io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:145)
        at io.deephaven.engine.util.DelegatingScriptSession.evaluateScript(DelegatingScriptSession.java:71)
        at io.deephaven.engine.util.ScriptSession.evaluateScript(ScriptSession.java:84)
        at io.deephaven.server.console.ConsoleServiceGrpcImpl.lambda$executeCommand$8(ConsoleServiceGrpcImpl.java:187)
        at io.deephaven.server.session.SessionState$ExportBuilder.lambda$submit$2(SessionState.java:1299)
        at io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:847)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at io.deephaven.server.runner.DeephavenApiServerModule$ThreadFactory.lambda$newThread$0(DeephavenApiServerModule.java:157)
        at java.lang.Thread.run(Thread.java:829)

It seems like TWS is not responding to some requests.

The contents of `ticks_efp` need to be validated

The contents of ticks_efp need to be validated. EFPs are very poorly documented by IB. It will take some research and experimentation to figure out how to get EFP data.

I suspect the data is obtained via request_market_data(contract), where contract is an EFP contract. Unfortunately, IB does not document how to create such contracts, and it is not obvious in TWS.

Register IB Error 10168 Requested market data is not subscribed. Delayed market data is not enabled.

ERROR:ibapi.wrapper:ERROR 11 10168 Requested market data is not subscribed. Delayed market data is not enabled.
ERROR:root:Unmapped value.  Please file an issue at https://github.com/deephaven-examples/deephaven-ib/issues: '10168'
  File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/client.py", line 263, in run
    self.decoder.interpret(fields)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/decoder.py", line 1294, in interpret
    self.interpretWithSignature(fields, handleInfo)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/ibapi/decoder.py", line 1275, in interpretWithSignature
    method(*args)
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/tws_client.py", line 408, in error
    [reqId, errorCode, map_values(errorCode, _error_code_message_map), errorString,
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/tablewriter.py", line 92, in map_values
    msg = f"Unmapped value.  Please file an issue at https://github.com/deephaven-examples/deephaven-ib/issues: '{value}'\n{trace_str()}\n-----"
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_internal/trace.py", line 9, in trace_str
    return "".join(traceback.format_stack())

ERROR:ibapi.wrapper:ERROR 25 200 No security definition has been found for the request

This error results during the subscription to portfolio securities. The extended error message is:

Ambiguity may occur when the contract definition provided is not unique. For some stocks that has the same Symbol, Currency and Exchange, you need to specify the IBApi.Contract.PrimaryExch attribute to avoid ambiguity. Please refer to a sample stock contract here. For futures that has multiple multipliers for the same expiration, You need to specify the IBApi.Contract.Multiplier attribute to avoid ambiguity. Please refer to a sample futures contract here.

This error is strange, because deephaven-ib is requesting contract details based upon a contract supplied by ibapi.

The error is coming fromtws_client.py -> positionMulti.

FA operation ignored error message

An FA operation ignored error message is present. Should not execute the commands that cause this, or the error should be eaten.

ERROR:ibapi.wrapper:ERROR 125 321 Error validating request.-'bW' : cause - Incorrect generic tick list of 292,456,225,232,619,165,318,293,294,233,375,295,595,236,236,104,411,106,100,105,101.  Legal ones for (STK) are: 100(Option Volume),101(Option Open Interest),105(Average Opt Volume),106(impvolat),165(Misc. Stats),221/220(Creditman Mark Price),225(Auction),232/221(Pl Price),233(RTVolume),236(inventory),258/47(Fundamentals),293(TradeCount),294(TradeRate),295(VolumeRate),318(LastRTHTrade),375(RTTrdVolume),411(rthistvol),456/59(IBDividends),460(Bond Factor Multiplier),576(EtfNavBidAsk(navbidask)),577(EtfNavLast(navlast)),578(EtfNavClose(navclose)),586(IPOHLMPRC),587(Pl Price Delayed),588(Futures Open Interest),595(Short-Term Volume X Mins),614(EtfNavMisc(high/low)),619(Creditman Slow Mark Price),623(EtfFrozenNavLast(fznavlast))

Register error code 10147

ERROR:ibapi.wrapper:ERROR 53 10147 OrderId 53 that needs to be cancelled is not found.
ERROR:root:Unmapped error code.  Please file an issue at https://github.com/deephaven-examples/deephaven-ib/issues:
	errorCode='10147'
	errorString='OrderId 53 that needs to be cancelled is not found.'
	This only impacts the error message you see and will not impact the execution of your program.

def request_account_pnl(self, account: str = "All", model_code: str = "") -> Request:
def request_account_overview(self, account: str, model_code: str = "") -> Request:
def request_account_positions(self, account: str, model_code: str = "") -> Request:

deephaven-examples / deephaven-ib Goto Github PK

deephaven-ib's People

Contributors

Stargazers

Watchers

Forkers

deephaven-ib's Issues

Use delayed market data if you do not have access to real-time

client.set_market_data_type(dhib.MarketDataType.DELAYED)

Recommend Projects

Recommend Topics

Recommend Org