KeyError / Attribute error when parsing OANDA JSON about async_v20 HOT 22 CLOSED

Here are some example errors that have occurred in the past:

from async_v20.

MarginUsed for TradeSummary is undocumented.

from async_v20.

• halfSpreadCost is undocumented for Transaction

from async_v20.

Version 2.2.4b4 contains halfSpreadCost.

Required updating TradeReduce, TradeOpen & Transaction definitions

from async_v20.

Admittedly, I'm getting frustrated about OANDA's undocumented changes to the v20 API, which keeps breaking async_v20.

I need to set up some automated API stress test that can be run on a practice account. Where ideally all endpoints and responses get polled. Placing orders and the lot! There doesn't seem to be any other way to check for consistency. :(

from async_v20.

CalculatedTradeState also has margin_used

from async_v20.

CalculatedPositionState has margin_used

from async_v20.

partialFill for Transaction is undocumented

from async_v20.

I have added check_api_methods.py to catch these changes as they occur.

Some endpoints need to be added, but it's a start

from async_v20.

OANDA has added:

• guaranteedStopLossOrderMode to Account
• resettablePLTime to Account
• guaranteedExecutionFees to Account, Position & PositionSide

from async_v20.

Just added:

• guaranteed to Order. (a bool) & corresponding version bump. 6.2.2b0
• guaranteed to Transaction.

from async_v20.

Ok, Well, I received confirmation that these extra attributes will be added to the documentation shortly. Standby!

from async_v20.

Hi there,

is this project outdated?
I noticed that there are new undescribed features (e.g. guaranteed_stop_loss_order_mutability in accountSummary).
I can repair these scalar values that are missing.

However, there seems to be an entire new class:
guaranteed_stop_loss_order_level_restriction

which is also found in the official v20 api:

class GuaranteedStopLossOrderLevelRestriction(BaseEntity):
    """
    A GuaranteedStopLossOrderLevelRestriction represents the total position
    size that can exist within a given price window for Trades with guaranteed
    Stop Loss Orders attached for a specific Instrument.
    """

    #
    # Format string used when generating a summary for this object
    #
    _summary_format = ""

    #
    # Format string used when generating a name for this object
    #
    _name_format = ""

    #
    # Property metadata for this object
    #
    _properties = spec_properties.primitives_GuaranteedStopLossOrderLevelRestriction

    def __init__(self, **kwargs):
        """
        Create a new GuaranteedStopLossOrderLevelRestriction instance
        """
        super(GuaranteedStopLossOrderLevelRestriction, self).__init__()
 
        #
        # Applies to Trades with a guaranteed Stop Loss Order attached for the
        # specified Instrument. This is the total allowed Trade volume that can
        # exist within the priceRange based on the trigger prices of the
        # guaranteed Stop Loss Orders.
        #
        self.volume = kwargs.get("volume")
 
        #
        # The price range the volume applies to. This value is in price units.
        #
        self.priceRange = kwargs.get("priceRange")

    @staticmethod
    def from_dict(data, ctx):
        """
        Instantiate a new GuaranteedStopLossOrderLevelRestriction from a dict
        (generally from loading a JSON response). The data used to instantiate
        the GuaranteedStopLossOrderLevelRestriction is a shallow copy of the
        dict passed in, with any complex child types instantiated
        appropriately.
        """

        data = data.copy()

        if data.get('volume') is not None:
            data['volume'] = ctx.convert_decimal_number(
                data.get('volume')
            )

        if data.get('priceRange') is not None:
            data['priceRange'] = ctx.convert_decimal_number(
                data.get('priceRange')
            )

        return GuaranteedStopLossOrderLevelRestriction(**data)

I am unable to solve this issue. I do not know which files and where I have to edit in async_v20 in order to include this new class.

Running python check_api_methods.py
results in:

async_v20.exceptions.InstantiationFailure: Could note create <class 'async_v20.definitions.types.Instrument'>. ARGUMENTS: ((), {'name': 'TRY_JPY', 'type': 'CURRENCY', 'display_name': 'TRY/JPY', 'pip_location': -2, 'display_precision': 3, 'trade_units_precision': 0, 'minimum_trade_size': '1', 'maximum_trailing_stop_distance': '100.000', 'minimum_trailing_stop_distance': '0.050', 'maximum_position_size': '0', 'maximum_order_units': '100000000', 'margin_rate': '0.05', 'minimum_guaranteed_stop_loss_distance': '0.200', 'guaranteed_stop_loss_order_execution_premium': '0.050', 'guaranteed_stop_loss_order_level_restriction': {'volume': '1000000', 'priceRange': '0.200'}}). __init__() got an unexpected keyword argument 'guaranteed_stop_loss_order_level_restriction'
Unclosed client session

from async_v20.

from async_v20.

Thanks great!

I will try your solution.
I was trying to reverse-engineer your code.
I can file a pull request for the few scalar features I found (but that is not much).

I was also wondering what the descriptions.yaml file is doing. Code also works without it. Some sort of serialization thing, but I guess created when you dump some instances?

from async_v20.

Many thanks.

Most of your post is about what i referred to as "scalar information". This means adding stuff to the dictionaries in "attributes.py" and adding the keywords to the constructors in "types.py".

The part where I was uncertain was how to add the actual class.
I noticed that you basically have an async_v20 class for each of Oanda's classes, but some classes in addition in primitives.py: for example "class TradeID".

i do not understand the rationale behind how you definded the classes that you used; a Class from the official Oanda v20 repository can not directlly be copy-pasted to types.py; for example, some classes in types.py contain the custom class "TradeID" (e.g. TrailingStopLossOrderTransaction), some don't; (e.g. LimitOrder); and TradeID is not an Oanda-Class.

I do not understand the logic behind your class unverse vs Oanda's class universe. Therefore, I would be unable to add the new class to types.py because I do simply not know what to write in the class body.

I guess it should be something like

class GuaranteedStopLossOrderLevelRestriction(BaseEntity):
    """
    A GuaranteedStopLossOrderLevelRestriction represents the total position
    size that can exist within a given price window for Trades with guaranteed
    Stop Loss Orders attached for a specific Instrument.
    """
    def __init__(self, volume: DecimalNumber = ..., priceRange: DecimalNumber = ...):
        Model.__init__(**locals())

But I am unsure about this. I am also unsure whether or not this class should be placed in primitives.py
and not in types.py (it appears in the primitives section on the oanda REST API).

I also noticed that you have many more classes in your primitives.py as OANDA has; for example, TransactionFilter does not appear on OANDA and I do not know its purpose and the logic behind it.

I am certainly willing to continue this project but I fear I need a few more hints from you to understand the logic behind it.

from async_v20.

from async_v20.

OK, so the first thing I have done is hopped on the API documentation: http://developer.oanda.com/rest-live-v20/primitives-df/#InstrumentName

And yes I can see that there is this new class GuaranteededStopLossOrderLevelRestriction.

There are a few steps to add it (which I haven’t done in a while) so I’ll do my best here to show you what is required.

async_v20/definitions/attributes.py contain two python dictionaries {} that convert the CamelCase class attributes OANDA sends in it’s JSON response into snake_case attributes for the python class instance.

Refer to this piece of documentation: https://async-v20.readthedocs.io/en/latest/beyond_getting_started.html#what-might-be-useful

The new class contains two attributes ‘volume’ and ‘priceRange’ we need to make sure that both of these are in the two dictionaries contained within the attributes.py file.

The first dictionary is instance_attributes this dictionary is used to parse CamelCase into snake_case. The ‘volume’ attribute is already in these dictionaries and we need not worry about it, that means that we need to add ‘priceRange’. See below:

Note that I added ‘priceRange’ twice, one with the camelCase and key and another with the ‘snake_case’ key, this allows both camelCase and snake_case keyword arguments to be passed to the Class upon instantiation.

Now we need to add the reverse, so that the instance with snake_case attributes can be sent back to OANDA in camelCase. We do this in the json_attributes dictionary.

Ok, so we have added the attributes for the class but we still need to add the class. I have one this in async_v20/definitions/types.py.

Going by the error code you sent me in the first email it looks like this class is an attribute of an Instrument class, so I have defined this new class above the Instrument class definition. Notice that each argument has a type annotation and a default value. The ellipsis ‘…’ default value is always evaluated to True in python and I have used this to determine if OANDA sent the default value or not in a json response.

The next step is to add this new class to the Instrument type definition:

Now I have just realized that guaranteed_stop_loss_order_level_restriction is an attribute of an instrument which means that it needs to be added to the attributes.py dictionaries (instance_attributes and json_attributes). So you will have to repeat the above process converting from camelCase to snake_case.

I hope this has been useful for you. Let me know how you go, also regarding the repository I am no longer interested in maintaining this and would be willing to transfer it. Keep in mind that I didn’t add any documentation in the above process. I have had previous correspondence with OANDA regarding this issue see here: #12

Also I think that yaml file was a dump I performed when I did some web scraping originally to get the api specifications from OANDAs website which I used to ‘meta’ programme async_v20 in its very early stages.

from async_v20.

from async_v20.

from async_v20.

Great! good to hear, I'm sorry they didn't come through at the start, it would have been very confusing!

To answer your question regarding why is the new Class placed in types.py and not primitives.py is basically a design choice, I had a few iterations before I ended up with this structure. In previous versions of async_v20 I had structured the modules and flies in the same manner as OANDA's online documentation, with the intention of creating an intuitive cross-reference system. This approach failed due to circular imports, that I could not resolve with out loading the whole Class hierarchy at once (This is the reason it was important you placed GuaranteedStopLossOrderLevelRestriction above the Instrument class definition. Having done that I realized that there was some structure i could preserve, which ended up being all the Classes in primitives.py essentially all classes in primitives.py are scalar values as you say. Either a string, float, integer or Pandas.Datetime, all other classes are built form combinations of these. The new GuaranteedStopLossOrderLevelRestriction class had two attributes and is there for not a single scalar value, which means it should be placed in types.py (according to my convention).

Another thing to keep in mind is that the docstring's for the classes are being used with Sphinx's autodoc allowing readthedocs to keep info updated, you may wish to keep these maintained as well.

from async_v20.

I have spent some time the last two days updating the object model incorporating changes. Some have been documented by OANDA and others have not.

I have decided to issue warnings when unexpected attributes are supplied by OANDA rather than raise UnknownValue Exceptions. This will prevent async_v20 from breaking due to small changes from OANDA, but will mean that overtime greater drift between async_v20's object model and OANDA's may occur unless maintained properly.

from async_v20.