Feature request: streaming API support about teslapy HOT 12 CLOSED

That could be done, however using my M3, the socket is closed by the remote end as soon as I subscribe. Are you able to test this feature?

from teslapy.

Tested with carson and streaming does work, but I have to be inside the car, otherwise no data is streamed.

from teslapy.

I was just playing around with carson too and also noticed it repeatedly disconnects, though only after about 10 seconds, and it does send some data before that.

from teslapy.

It looks like websocket-client and autobahn do not work (remote disconnect). I did however successfully receive streaming data using tornado and aiohttp. If I choose aiohttp, then I'll have to drop Python support for anything below 3.5. If I choose tornado, then I can still support Python 2.7. What shall I do?

from teslapy.

Whichever is easiest, I'd say. Does Tornado actually support Python 2.7 though? The page you linked to says it needs Python 3.5 or greater. In any case, given that Python 2 reached "end of life" over a year ago, Python 2 support should probably be a minor consideration. But it's your project. :)
(also, carson uses aiohttp too, but still disconnects. Are they doing something wrong?)

from teslapy.

I finally figured it out. The streaming API sends on-change data, and if there is no change in any of the data columns, then after 10 seconds a disconnect message is sent and then you have to subscribe again in order to receive data again. It is just how the streaming API works.
It does work using websocket-client and I implemented some initial support in 3af8a05.
Can you give the new code a try?

from teslapy.

It worked for a little while then stopped working for some reason.
I used cli.py and the first time it printed:

2021-06-27 10:24:49,869 - root - INFO - 2 product(s), 1 selected
Product 0:
{'timestamp': 1624814690518, 'speed': None, 'odometer': 15121.4, 'soc': 50, 'elevation': 320, 'est_heading': 354, 'est_lat': xx.xxxxxxx, 'est_lng': xx.xxxxxxx, 'power': 0, 'shift_state': 'P', 'range': 119, 'est_range': 105, 'heading': 89}
{'timestamp': 1624814701268, 'speed': None, 'odometer': 15121.4, 'soc': 50, 'elevation': 320, 'est_heading': 354, 'est_lat': xx.xxxxxxx, 'est_lng': xx.xxxxxxx, 'power': 0, 'shift_state': 'P', 'range': 119, 'est_range': 105, 'heading': 269}
2021-06-27 10:25:11,277 - teslapy - ERROR - disconnected

I'm assuming it printed two lines because the "heading" changed (not sure what that value is)
After that it would just print one line and then disconnect, e.g:

{'timestamp': 1624814785326, 'speed': None, 'odometer': 15121.4, 'soc': 50, 'elevation': 320, 'est_heading': 354, 'est_lat': xx.xxxxxxx, 'est_lng': xx.xxxxxxx, 'power': 0, 'shift_state': None, 'range': 119, 'est_range': 105, 'heading': 269}
2021-06-27 10:26:35,412 - teslapy - ERROR - disconnected

(note the shift_state though. The car was still parked at this time)
In an effort to try and keep it connected longer, I went to the car so I could shift it between "drive" and "park" while the stream was running. This is where it stopped working. It printed one line with shift_state P, then disconnected even though I put the car in drive. After that it would no longer print anything. It would just hang forever.

from teslapy.

After some more experimentation, it seems that when the car is "on" (i.e. after I put my foot on the brake), the stream stops working. The last thing it prints with debugging enabled is:

2021-06-27 11:09:42,936 - websocket - DEBUG - ++Sent decoded: fin=1 opcode=1 data=b'{"msg_type": "data:subscribe_oauth", "token": "qts-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "value": "speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading", "tag": "xxxxxxxxx"}'
2021-06-27 11:09:42,936 - websocket - DEBUG - ++Rcv raw: b'\x823{"msg_type":"control:hello","connection_timeout":0}'
2021-06-27 11:09:42,936 - websocket - DEBUG - ++Rcv decoded: fin=1 opcode=2 data=b'{"msg_type":"control:hello","connection_timeout":0}'
2021-06-27 11:09:42,937 - teslapy - DEBUG - connected
2021-06-27 11:10:42,934 - teslapy - ERROR - Connection to remote host was lost.

so when the car is "on", it just hangs for a minute and then exits.

from teslapy.

Sometimes the stream doesn't start right away, or not at all. No matter how many times I send the subscribe message. When I enable pinging, then the socket lives longer then 60 seconds and sometimes the data just starts. wsapp.run_forever(ping_interval=10)
I could put all this in a loop, to restart after a disconnect...

from teslapy.

Added support for a retry loop and indefinite loop to the stream method in ce39245

from teslapy.

Release v1.4.0 includes streaming API support

from teslapy.

I just noticed this. I was searching around to see if anyone knew of an API mimicking what green was describing here.

I could probably shed some light on the disconnect behavior. That might help explain why I wrote carson to give up (sorta) when it does instead of perpetually retry. I say 'sorta' because it will continue to loop if the API reports a user is present. But if you are tyring to create a service to stream all your trips, it's a little more involved than that.

Below is the logic, however, check for errors before you copy/paste. I inclined code from other functions simply to illustrate the flow here in one code block - I might have introduced errors when doing so. (I owe it to @timdorr and others to contribute to his Streaming page. I'll try to find time this week to do so.)

There are two aspects to consider when looping:

1. Letting the car fall asleep when it's time
2. Allowing Tesla to throttle the data feed when it's appropriate.

Letting the car fall asleep

In times when the api_version was in the single digits (I think it's up to 40 now) you risked keeping your car awake by repeatedly invoking endpoints. Even if it was simply to api/1/vehicles to see if the car is online or not, if it was, you would need to give it roughly 10-15 minutes of quiet time from that point forward before the car would fall asleep on its own.

Once asleep, you could call /api/1/vehicles as often as you'd like to see if it's out of its slumber. The car would not wake up until you called api/1/vehicles/{vehicle_id}/wake_up or it woke up due to some other stimulus. Personally, I set this sleeping interval to 10-15 seconds.

Until then, I implement a standard exponential back-off mechanism to let it go to sleep.

However...

Sometimes you don't want to back-off. Even if there is no user present, you still want a higher resolution because you want to make sure you capture a streaming session as soon as the car is being driven. This could happen in situations such as:

• It's charging
• In sentry mode in the parking lot at grocery store
• Climate-keep on, dog/fish/head mode, camping mode, etc.

Disconnects

You'll notice when the car is in motion (assuming good network connectivity, etc), you'll get a record every ~250ms. But as you roll up to a stop light, that starts to taper off. When stopped at a stop light, the stream of data ends and Tesla will eventually hang-up on your websocket. When this happens, carson will turn right around and establish a new websocket connection to await the moment you floor the accelerator.

I'm not sure what tesla-api or others do. But at first glance, it would appear to the API consumer that it's not working. Or, if you are trying to stream while your car is parked in the driveway, you are going to be disappointed because Tesla will quickly hang-up on you... there no streaming data to report!

Looping

With those two points in mind, here's the top level logic I have wrapped around carson's implementation.

async def backoff(name=None, verbose=1):

    wait_time = WAIT_TIME_BASIS
    iterations = 0
    backoff_count = 0
    credential = None

    while 1:
        iterations += 1
        logger.debug(f'==== Iteration {iterations:,} (backoff={backoff_count:,}  pid={os.getpid()})')

        credential = credential or TeslaCredential.load()
        if credential.expired:
            await credential.refresh()

        session = carson.Session(email=credential.email, access_token=credential.access_token)
        reasons_to_not_backoff = []
        try:
            car = await session.vehicles(name=name)
            if not car:
                raise Exception(f'No cars found! name={name!r}')

            logger.info(car)
            if car.state != 'online' or car.in_service:
                # We can iterate as often as we want without risking keeping the car awake.  If the
                # car is in service, we would get a 405 method not allowed if we tried to get car
                # data.
                backoff_count = 0

            else:
                # Query current data and determine if a user is present
                await car.data()
                if car.vehicle_state.is_user_present:
                    backoff_count = 0
                    queue = asyncio.Queue()

                    async def _queue_writer(waypoint):
                        queue.put_nowait(waypoint)

                    reader = asyncio.create_task(queue_reader(queue))
                    data_points = await car.stream(callback=_queue_writer)
                    logger.info(f'Data points collected on iteration: {data_points:,}.')

                    await queue.join()
                    reader.cancel()
                    try:
                        await reader
                    except asyncio.CancelledError:
                        pass
                else:
                    # Determine if we need to back off.  Cases where we do NOT back off and can
                    # sleep the short 10 seconds or whatever the configured minimum is set to:
                    #   - in sentry mode
                    #   - is charging
                    #   - dog mode
                    #   - climate protection sytem on

                    if car.charge_state.charging_state.lower() == 'charging':
                        # Known `chargin_state`s.
                        #   - Stopped
                        #   - Disconnected
                        reasons_to_not_backoff.append('charging')

                    climate_mode = car.climate_state.climate_keeper_mode
                    if climate_mode != 'off':
                        reasons_to_not_backoff.append(f'climate {climate_mode!r} mode on')
                    elif car.climate_state.is_climate_on:
                        reasons_to_not_backoff.append('climate on')

                    if car.vehicle_state.sentry_mode:
                        reasons_to_not_backoff.append('sentry mode')
                    if car.vehicle_state.valet_mode:
                        reasons_to_not_backoff.append('valet mode')
                    if not reasons_to_not_backoff:
                        backoff_count += 1

        except asyncio.CancelledError:
            credential = None
            logger.warn('Streaming stopped because task was cancelled.')
            return
        except Exception as err:
            credential = None
            logger.error('Unhandled exception while streaming.: %r', err, exc_info=True)
            if not backoff_count:
                backoff_count = 1
            else:
                backoff_count += 1
        finally:
            await session.close()

        wait_time = WAIT_TIME_BASIS
        for _ in range(backoff_count):
            jitter = random() * 0.25
            wait_time *= (2 + jitter)
        wait_time = min(1000.0, wait_time)

        if logger.isEnabledFor(DEBUG) or reasons_to_not_backoff:
            duration = timedelta(seconds=wait_time)
            until = datetime.now(tz=timezone.utc) + duration
            msg = f'Iteration {iterations:,} complete.  Sleeping {duration} until {until:%H:%M:%S}.'
            if reasons_to_not_backoff:
                msg += f'  Reasons to not backoff: {", ".join(reasons_to_not_backoff)}.'
            logger.info(msg)
        try:
            await asyncio.sleep(wait_time)
        except asyncio.CancelledError:
            return

from teslapy.