adafruit / adafruit_circuitpython_portalbase Goto Github PK

In working on a project and reading the docs for PortalBase.network I ran into a lot of confusion seeing that fetch() has headers as a list type[0].

Seeings how header params are nearly always a dictionary I decided to pass a dictionary and it worked. Also, from what I understand the underlying adafruit_requests is based on they Python Requests library, which also uses a dictionary for headers params[1].

When I try and pass a list I encounter the following traceback:

  File "code.py", line 34, in <module>
  File "code.py", line 30, in fetch_data
  File "adafruit_portalbase/network.py", line 548, in fetch_data
  File "adafruit_portalbase/network.py", line 467, in fetch
  File "adafruit_requests.py", line 684, in get
  File "adafruit_requests.py", line 567, in request
  File "adafruit_requests.py", line 565, in request
  File "adafruit_requests.py", line 494, in _send_request
  TypeError: list indices must be integers, not str

Looking into adafruit_requests it does indeed look to expect a dict[2].

[0] https://github.com/adafruit/Adafruit_CircuitPython_PortalBase/blob/main/adafruit_portalbase/network.py#L450
[1] https://docs.python-requests.org/en/master/user/quickstart/#custom-headers
[2] https://github.com/adafruit/Adafruit_CircuitPython_Requests/blob/master/adafruit_requests.py#L533

Somehow due to the timing of the implementation, I think this may have been overlooked and only TimeAlarm was added.

The data’s there in the Adafruit IO time response but is currently discarded. This could either be returned by the function (in which case, programs that don’t need it can just ignore) or stored in the class somewhere (requiring a name or getter function and a few bytes of RAM).

This is rare, but some things do want a valid UTC offset…anything using the MET Norway sun/moon API for example (Moon phase clock, LED shadow box), maybe others. The current workaround is to follow a call to get_local_time() with a second “manual” query from Adafruit IO that requests just the UTC string. Doing this once on startup won’t suffice, since it changes twice a year. During the periodic time syncs that a lot of projects do anyway seems preferable.

I’m not too concerned with the representation. Adafruit IO response is a string 'sHHMM' (sign, hours, minutes), MET Norway wants 'sHH:MM', but it’s a trivial conversion, so I’d maybe just keep the original response string (e.g. don’t separate into integer tuple, because reasons).

CircuitPython version

Adafruit CircuitPython 9.0.0-alpha.2-8-g7715ff09cc on 2023-10-31; Adafruit PyPortal Titano with samd51j20
>>> [D
... 
>>>

Code/REPL

# SPDX-FileCopyrightText: 2020 Richard Albritton for Adafruit Industries
#
# SPDX-License-Identifier: MIT

import time
import board
import microcontroller
import displayio
import busio
from analogio import AnalogIn
import neopixel
import adafruit_adt7410
from adafruit_bitmap_font import bitmap_font
from adafruit_display_text.label import Label
from adafruit_button import Button
import adafruit_touchscreen
from adafruit_pyportal import PyPortal

# ------------- Constants ------------- #
# Sound Effects
soundDemo = "/sounds/sound.wav"
soundBeep = "/sounds/beep.wav"
soundTab = "/sounds/tab.wav"

# Hex Colors
WHITE = 0xFFFFFF
RED = 0xFF0000
YELLOW = 0xFFFF00
GREEN = 0x00FF00
BLUE = 0x0000FF
PURPLE = 0xFF00FF
BLACK = 0x000000

# Default Label styling
TABS_X = 0
TABS_Y = 15

# Default button styling:
BUTTON_HEIGHT = 40
BUTTON_WIDTH = 80

# Default State
view_live = 1
icon = 1
icon_name = "Ruby"
button_mode = 1
switch_state = 0

# ------------- Functions ------------- #
# Backlight function
# Value between 0 and 1 where 0 is OFF, 0.5 is 50% and 1 is 100% brightness.
def set_backlight(val):
    val = max(0, min(1.0, val))
    try:
        board.DISPLAY.auto_brightness = False
    except AttributeError:
        pass
    board.DISPLAY.brightness = val


# Helper for cycling through a number set of 1 to x.
def numberUP(num, max_val):
    num += 1
    if num <= max_val:
        return num
    else:
        return 1


# Set visibility of layer
def layerVisibility(state, layer, target):
    try:
        if state == "show":
            time.sleep(0.1)
            layer.append(target)
        elif state == "hide":
            layer.remove(target)
    except ValueError:
        pass


# This will handle switching Images and Icons
def set_image(group, filename):
    """Set the image file for a given goup for display.
    This is most useful for Icons or image slideshows.
        :param group: The chosen group
        :param filename: The filename of the chosen image
    """
    print("Set image to ", filename)
    if group:
        group.pop()

    if not filename:
        return  # we're done, no icon desired

    # CircuitPython 6 & 7 compatible
    image_file = open(filename, "rb")
    image = displayio.OnDiskBitmap(image_file)
    image_sprite = displayio.TileGrid(
        image, pixel_shader=getattr(image, "pixel_shader", displayio.ColorConverter())
    )

    # # CircuitPython 7+ compatible
    # image = displayio.OnDiskBitmap(filename)
    # image_sprite = displayio.TileGrid(image, pixel_shader=image.pixel_shader)

    group.append(image_sprite)


# return a reformatted string with word wrapping using PyPortal.wrap_nicely
def text_box(target, top, string, max_chars):
    text = pyportal.wrap_nicely(string, max_chars)
    new_text = ""
    test = ""

    for w in text:
        new_text += "\n" + w
        test += "M\n"

    text_height = Label(font, text="M", color=0x03AD31)
    text_height.text = test  # Odd things happen without this
    glyph_box = text_height.bounding_box
    target.text = ""  # Odd things happen without this
    target.y = int(glyph_box[3] / 2) + top
    target.text = new_text


def get_Temperature(source):
    if source:  # Only if we have the temperature sensor
        celsius = source.temperature
    else:  # No temperature sensor
        celsius = microcontroller.cpu.temperature
    return (celsius * 1.8) + 32


# ------------- Inputs and Outputs Setup ------------- #
light_sensor = AnalogIn(board.LIGHT)
try:
    # attempt to init. the temperature sensor
    i2c_bus = busio.I2C(board.SCL, board.SDA)
    adt = adafruit_adt7410.ADT7410(i2c_bus, address=0x48)
    adt.high_resolution = True
except ValueError:
    # Did not find ADT7410. Probably running on Titano or Pynt
    adt = None

# ------------- Screen Setup ------------- #
pyportal = PyPortal()
pyportal.set_background("/images/loading.bmp")  # Display an image until the loop starts
pixel = neopixel.NeoPixel(board.NEOPIXEL, 1, brightness=1)

# Touchscreen setup  [ Rotate 270 ]
display = board.DISPLAY
display.rotation = 270

if board.board_id == "pyportal_titano":
    screen_width = 320
    screen_height = 480
    set_backlight(
        1
    )  # 0.3 brightness does not cause the display to be visible on the Titano
else:
    screen_width = 240
    screen_height = 320
    set_backlight(0.3)

# We want three buttons across the top of the screen
TAB_BUTTON_Y = 0
TAB_BUTTON_HEIGHT = 40
TAB_BUTTON_WIDTH = int(screen_width / 3)

# We want two big buttons at the bottom of the screen
BIG_BUTTON_HEIGHT = int(screen_height / 3.2)
BIG_BUTTON_WIDTH = int(screen_width / 2)
BIG_BUTTON_Y = int(screen_height - BIG_BUTTON_HEIGHT)

# Initializes the display touch screen area
ts = adafruit_touchscreen.Touchscreen(
    board.TOUCH_YD,
    board.TOUCH_YU,
    board.TOUCH_XR,
    board.TOUCH_XL,
    calibration=((5200, 59000), (5800, 57000)),
    size=(screen_width, screen_height),
)

# ------------- Display Groups ------------- #
splash = displayio.Group()  # The Main Display Group
view1 = displayio.Group()  # Group for View 1 objects
view2 = displayio.Group()  # Group for View 2 objects
view3 = displayio.Group()  # Group for View 3 objects

# ------------- Setup for Images ------------- #
bg_group = displayio.Group()
splash.append(bg_group)
set_image(bg_group, "/images/BGimage.bmp")

icon_group = displayio.Group()
icon_group.x = 180
icon_group.y = 120
icon_group.scale = 1
view2.append(icon_group)

# ---------- Text Boxes ------------- #
# Set the font and preload letters
font = bitmap_font.load_font("/fonts/Helvetica-Bold-16.bdf")
font.load_glyphs(b"abcdefghjiklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890- ()")

# Text Label Objects
feed1_label = Label(font, text="Text Window 1", color=0xE39300)
feed1_label.x = TABS_X
feed1_label.y = TABS_Y
view1.append(feed1_label)

feed2_label = Label(font, text="Text Window 2", color=0xFFFFFF)
feed2_label.x = TABS_X
feed2_label.y = TABS_Y
view2.append(feed2_label)

sensors_label = Label(font, text="Data View", color=0x03AD31)
sensors_label.x = TABS_X
sensors_label.y = TABS_Y
view3.append(sensors_label)

sensor_data = Label(font, text="Data View", color=0x03AD31)
sensor_data.x = TABS_X + 16  # Indents the text layout
sensor_data.y = 150
view3.append(sensor_data)

# ---------- Display Buttons ------------- #
# This group will make it easy for us to read a button press later.
buttons = []

# Main User Interface Buttons
button_view1 = Button(
    x=0,  # Start at furthest left
    y=0,  # Start at top
    width=TAB_BUTTON_WIDTH,  # Calculated width
    height=TAB_BUTTON_HEIGHT,  # Static height
    label="View 1",
    label_font=font,
    label_color=0xFF7E00,
    fill_color=0x5C5B5C,
    outline_color=0x767676,
    selected_fill=0x1A1A1A,
    selected_outline=0x2E2E2E,
    selected_label=0x525252,
)
buttons.append(button_view1)  # adding this button to the buttons group

button_view2 = Button(
    x=TAB_BUTTON_WIDTH,  # Start after width of a button
    y=0,
    width=TAB_BUTTON_WIDTH,
    height=TAB_BUTTON_HEIGHT,
    label="View 2",
    label_font=font,
    label_color=0xFF7E00,
    fill_color=0x5C5B5C,
    outline_color=0x767676,
    selected_fill=0x1A1A1A,
    selected_outline=0x2E2E2E,
    selected_label=0x525252,
)
buttons.append(button_view2)  # adding this button to the buttons group

button_view3 = Button(
    x=TAB_BUTTON_WIDTH * 2,  # Start after width of 2 buttons
    y=0,
    width=TAB_BUTTON_WIDTH,
    height=TAB_BUTTON_HEIGHT,
    label="View 3",
    label_font=font,
    label_color=0xFF7E00,
    fill_color=0x5C5B5C,
    outline_color=0x767676,
    selected_fill=0x1A1A1A,
    selected_outline=0x2E2E2E,
    selected_label=0x525252,
)
buttons.append(button_view3)  # adding this button to the buttons group

button_switch = Button(
    x=0,  # Start at furthest left
    y=BIG_BUTTON_Y,
    width=BIG_BUTTON_WIDTH,
    height=BIG_BUTTON_HEIGHT,
    label="Light Switch",
    label_font=font,
    label_color=0xFF7E00,
    fill_color=0x5C5B5C,
    outline_color=0x767676,
    selected_fill=0x1A1A1A,
    selected_outline=0x2E2E2E,
    selected_label=0x525252,
)
buttons.append(button_switch)  # adding this button to the buttons group

button_2 = Button(
    x=BIG_BUTTON_WIDTH,  # Starts just after button 1 width
    y=BIG_BUTTON_Y,
    width=BIG_BUTTON_WIDTH,
    height=BIG_BUTTON_HEIGHT,
    label="Light Color",
    label_font=font,
    label_color=0xFF7E00,
    fill_color=0x5C5B5C,
    outline_color=0x767676,
    selected_fill=0x1A1A1A,
    selected_outline=0x2E2E2E,
    selected_label=0x525252,
)
buttons.append(button_2)  # adding this button to the buttons group

# Add all of the main buttons to the splash Group
for b in buttons:
    splash.append(b)

# Make a button to change the icon image on view2
button_icon = Button(
    x=150,
    y=60,
    width=BUTTON_WIDTH,
    height=BUTTON_HEIGHT,
    label="Icon",
    label_font=font,
    label_color=0xFFFFFF,
    fill_color=0x8900FF,
    outline_color=0xBC55FD,
    selected_fill=0x5A5A5A,
    selected_outline=0xFF6600,
    selected_label=0x525252,
    style=Button.ROUNDRECT,
)
buttons.append(button_icon)  # adding this button to the buttons group

# Add this button to view2 Group
view2.append(button_icon)

# Make a button to play a sound on view2
button_sound = Button(
    x=150,
    y=170,
    width=BUTTON_WIDTH,
    height=BUTTON_HEIGHT,
    label="Sound",
    label_font=font,
    label_color=0xFFFFFF,
    fill_color=0x8900FF,
    outline_color=0xBC55FD,
    selected_fill=0x5A5A5A,
    selected_outline=0xFF6600,
    selected_label=0x525252,
    style=Button.ROUNDRECT,
)
buttons.append(button_sound)  # adding this button to the buttons group

# Add this button to view2 Group
view3.append(button_sound)

# pylint: disable=global-statement
def switch_view(what_view):
    global view_live
    if what_view == 1:
        button_view1.selected = False
        button_view2.selected = True
        button_view3.selected = True
        layerVisibility("hide", splash, view2)
        layerVisibility("hide", splash, view3)
        layerVisibility("show", splash, view1)
    elif what_view == 2:
        # global icon
        button_view1.selected = True
        button_view2.selected = False
        button_view3.selected = True
        layerVisibility("hide", splash, view1)
        layerVisibility("hide", splash, view3)
        layerVisibility("show", splash, view2)
    else:
        button_view1.selected = True
        button_view2.selected = True
        button_view3.selected = False
        layerVisibility("hide", splash, view1)
        layerVisibility("hide", splash, view2)
        layerVisibility("show", splash, view3)

    # Set global button state
    view_live = what_view
    print("View {view_num:.0f} On".format(view_num=what_view))


# pylint: enable=global-statement

# Set veriables and startup states
button_view1.selected = False
button_view2.selected = True
button_view3.selected = True
button_switch.label = "OFF"
button_switch.selected = True

layerVisibility("show", splash, view1)
layerVisibility("hide", splash, view2)
layerVisibility("hide", splash, view3)

# Update out Labels with display text.
text_box(
    feed1_label,
    TABS_Y,
    "The text on this screen is wrapped so that all of it fits nicely into a "
    "text box that is {} x {}.".format(
        feed1_label.bounding_box[2], feed1_label.bounding_box[3] * 2
    ),
    30,
)

text_box(feed2_label, TABS_Y, "Tap on the Icon button to meet a new friend.", 18)

text_box(
    sensors_label,
    TABS_Y,
    "This screen can display sensor readings and tap Sound to play a WAV file.",
    28,
)

board.DISPLAY.show(splash)


# ------------- Code Loop ------------- #
while True:
    touch = ts.touch_point
    light = light_sensor.value
    sensor_data.text = "Touch: {}\nLight: {}\nTemp: {:.0f}°F".format(
        touch, light, get_Temperature(adt)
    )

    # Will also cause screen to dim when hand is blocking sensor to touch screen
    #    # Adjust backlight
    #    if light < 1500:
    #        set_backlight(0.1)
    #    elif light < 3000:
    #        set_backlight(0.5)
    #    else:
    #        set_backlight(1)

    # ------------- Handle Button Press Detection  ------------- #
    if touch:  # Only do this if the screen is touched
        # loop with buttons using enumerate() to number each button group as i
        for i, b in enumerate(buttons):
            if b.contains(touch):  # Test each button to see if it was pressed
                print("button{} pressed".format(i))
                if i == 0 and view_live != 1:  # only if view1 is visible
                    pyportal.play_file(soundTab)
                    switch_view(1)
                    while ts.touch_point:
                        pass
                if i == 1 and view_live != 2:  # only if view2 is visible
                    pyportal.play_file(soundTab)
                    switch_view(2)
                    while ts.touch_point:
                        pass
                if i == 2 and view_live != 3:  # only if view3 is visible
                    pyportal.play_file(soundTab)
                    switch_view(3)
                    while ts.touch_point:
                        pass
                if i == 3:
                    pyportal.play_file(soundBeep)
                    # Toggle switch button type
                    if switch_state == 0:
                        switch_state = 1
                        b.label = "ON"
                        b.selected = False
                        pixel.fill(WHITE)
                        print("Switch ON")
                    else:
                        switch_state = 0
                        b.label = "OFF"
                        b.selected = True
                        pixel.fill(BLACK)
                        print("Switch OFF")
                    # for debounce
                    while ts.touch_point:
                        pass
                    print("Switch Pressed")
                if i == 4:
                    pyportal.play_file(soundBeep)
                    # Momentary button type
                    b.selected = True
                    print("Button Pressed")
                    button_mode = numberUP(button_mode, 5)
                    if button_mode == 1:
                        pixel.fill(RED)
                    elif button_mode == 2:
                        pixel.fill(YELLOW)
                    elif button_mode == 3:
                        pixel.fill(GREEN)
                    elif button_mode == 4:
                        pixel.fill(BLUE)
                    elif button_mode == 5:
                        pixel.fill(PURPLE)
                    switch_state = 1
                    button_switch.label = "ON"
                    button_switch.selected = False
                    # for debounce
                    while ts.touch_point:
                        pass
                    print("Button released")
                    b.selected = False
                if i == 5 and view_live == 2:  # only if view2 is visible
                    pyportal.play_file(soundBeep)
                    b.selected = True
                    while ts.touch_point:
                        pass
                    print("Icon Button Pressed")
                    icon = numberUP(icon, 3)
                    if icon == 1:
                        icon_name = "Ruby"
                    elif icon == 2:
                        icon_name = "Gus"
                    elif icon == 3:
                        icon_name = "Billie"
                    b.selected = False
                    text_box(
                        feed2_label,
                        TABS_Y,
                        "Every time you tap the Icon button the icon image will "
                        "change. Say hi to {}!".format(icon_name),
                        18,
                    )
                    set_image(icon_group, "/images/" + icon_name + ".bmp")
                if i == 6 and view_live == 3:  # only if view3 is visible
                    b.selected = True
                    while ts.touch_point:
                        pass
                    print("Sound Button Pressed")
                    pyportal.play_file(soundDemo)
                    b.selected = False

Behavior

soft reboot

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
code.py output:
Traceback (most recent call last):
File "code.py", line 148, in
File "adafruit_pyportal/init.py", line 128, in init
File "adafruit_pyportal/graphics.py", line 44, in init
File "adafruit_portalbase/graphics.py", line 60, in init
AttributeError: .show(x) removed. Use .root_group = x

Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

Description

No response

Additional information

No response

Hi,
I noticed something during my project with a PyPortal: when I use the fetch method from a PyPortal object, if the connection with the Wifi cannot be established, the program execution will be stopped there forever. So I decided to have a look at the code of the fetch method in the PortalBase class and I saw that the fetch method supports a timeout parameter, which is even set to 10 by default. So I was wondering why the connection tries would keep going forever.
The timeout parameter is indeed used by the call to the method self._wifi.requests.get in the fetch method of adafruit_portalbase/network.py. But before that, there is a call to the method connect in the same file. And that's where the problem appears to be: there is a loop which becomes infinite:

        while not self._wifi.is_connected:
            ...

There is not an alternative exit point for that loop: either it connects, or it connects. No maximum number of retries, no timeout, just nothing.
I believe that this is a bug or a least a serious candidate for improvement. By understanding this I have implemented this workaround:

try:
    pyportal.network._wifi.connect(pyportal.network._secrets["ssid"], pyportal.network._secrets["password"])
except RuntimeError as error:
    print("Error connecting to WiFi - ", error)
    # Do whatever you want to do with this situation

Only if I don't get an error by this connect attempt, I will call the fetch method, otherwise I won't

The secrets.py looks suspiciously like a wpa_supplicant.conf file but we can only put one wifi in there is there a way to put multiple as I would love my tag to work at home and office

Sometimes when getting the time, the following error occurs:

Hardware is PyPortal Adafruit Product ID: 4116

------ START: Error Message ------
File "adafruit_portalbase/network.py", line 208, in get_strftime
AttributeError: 'NoneType' object has no attribute 'get'
------ END: Error Message ------

Line 208: response = self._wifi.requests.get(api_url, timeout=10)

Not certain why this happens but it is not every time. Perhaps one in 5 connection attempts.

My workaround has been to catch all exceptions when attempting to get the time.

def getTime(lastRefreshedTime, secondsToUpdate):
    if pyportal is None:
        return None
    if (not lastRefreshedTime) or (time.monotonic() - lastRefreshedTime) > secondsToUpdate:
        eventWindow.statusDateTime.text = "Updating Time"
        try:
            print("INFO: Getting time from internet!")
            if (not lastRefreshedTime):
                eventWindow.changeBackground(
                    eventWindow.IMG_FILE_CONNECTING_BACKGROUND, False)

            start = time.time()
            pyportal.get_local_time()  # pyportal.get_local_time(false) non-blocking?
            print(f"INFO: Getting time took {time.time() - start} seconds.")
            lastRefreshedTime = time.monotonic()
            print("INFO: Success the time has been set.")
        except RuntimeError as e:
            print(f"WARN: Some error occured, retrying!\r\n{e}")
            if (not lastRefreshedTime):
                eventWindow.changeBackground(
                    eventWindow.IMG_FILE_CONNECT_FAILED_BACKGROUND, False)
        # Use try catch here as there is an error in adafruit_portalbase
        #   File "adafruit_portalbase/network.py", line 208, in get_strftime
        #   AttributeError: 'NoneType' object has no attribute 'get'
        except Exception as e:
            print(f"ERROR: Failed to getTime\r\n{e}")
            eventWindow.changeBackground(
                eventWindow.IMG_FILE_CONNECT_FAILED_BACKGROUND, False)

    removePastEvents()

    return lastRefreshedTime

Full code with the workaround is available at https://github.com/richteel/PyPortal_Events. Direct link to the code.py file is https://github.com/richteel/PyPortal_Events/blob/main/PyPortal/code.py.

CI failed to build, but should pass after adafruit/Adafruit_CircuitPython_AdafruitIO#85 is merged and it's re-run.

Sooo, PortalBase was never optimized. Thus it has gotten a bit bloated. It was designed with compatibility of old code in mind, and to that end it succeeded, but the library could be revamped to be smaller and more functional. I actually didn't think it would be used as heavily when I originally made a version for the MatrixPortal and then the other boards.

One of the major problems with the bloat is that it pushes many of the boards to their memory limits and I currently have a PR in place to freeze the library into CircuitPython. However, by making it smaller, this would free up some room on there.

Some of optimizing would be doing things like stripping out debug which is really unnecessary at this point and was a holdover from the original PyPortal library. Other things would be stripping out things like the "init everything at once approach" that was also a holdover from the original PyPortal library. It would mostly be writing the code to be more efficient and would likely be largely the same to the average user. I do really like the approach that was made with most of the boards in terms of adding text and would likely stick with that for the most part.

Before doing anything with this, I would like to hear other people's opinions.

On the Matrix Display & Matrix portal from Adabox 016 I get an error running the Custom Scrolling Quote Board Matrix Display example from [https://learn.adafruit.com/aio-quote-board-matrix-display].

With a feed set up with quotes:

"Gratitude is the art of seeing a gift in everything" -Unknown (Added on:2020-12-03)
"I see little of more importance to the future of our country & our civilization than full recognition of the place of the artist" -John F. Kennedy (Added on:2021-01-12)
"...save us from succumbing to the tragic temptation of becoming cynical" -Martin Luther King Jr. (Added on:2021-01-17)'

and using two colors, I get the error within a minute or two. The error occurs both when connected to MU and also after unplugging/replugging the display.

In PortalBase init.py if I replace

from adafruit_display_text.label import Label

with

from adafruit_display_text.bitmap_label import Label

the error no longer occurs.

Circuit Python 6.1.0
Libraries adafruit-circuitpython-bundle-6.x-mpy-20210225

Running Adafruit CircuitPython 6.2.0-beta.3 on 2021-03-04; Adafruit MagTag with ESP32S2 & 20210315 Libs distro, Project Selector gives 'pystack exhausted' error on json call ... Error occurs with 20210315 & 20210311 Portal_Base lib ... Error does NOT occur with 20210304 Portal_Base lib ... Code is modified SpaceX code from project ...

3WzRocket.py.zip

I noticed I was getting an infinite loop when trying to pull data from a particular json source on the MatrixPortal that I wasn't getting on the MagTag. Is this expected behavior? Why isn't it just raising the MemoryError?

Here is my test code which is just a modified version of the examples/matrixportal_simpletest.py from Adafruit_CircuitPython_MatrixPortal

import time
import board
import terminalio
from adafruit_matrixportal.matrixportal import MatrixPortal

# Set up where we'll be fetching data from
DATA_SOURCE = "https://covid.cdc.gov/covid-data-tracker/COVIDData/getAjaxData?id=vaccination_data"

# 63 is USA
LOCATION_NUM = 63

DATE_LOCATION = ["vaccination_data", LOCATION_NUM, 'Date']
NAME_LOCATION = ["vaccination_data", LOCATION_NUM, 'LongName']
CENSUS_LOACTION = ["vaccination_data", LOCATION_NUM, 'Census2019']
ADMINISTER_1_LOCATION = ["vaccination_data", LOCATION_NUM, 'Administered_Dose1']
ADMINISTER_2_LOCATION = ["vaccination_data", LOCATION_NUM, 'Administered_Dose2']

matrixportal = MatrixPortal(
    url=DATA_SOURCE,
    json_path=(DATE_LOCATION, NAME_LOCATION,
               CENSUS_LOACTION, ADMINISTER_1_LOCATION,
               ADMINISTER_2_LOCATION),
)


while True:
    try:
        value = matrixportal.fetch()
        print("Response is", value)
    except (ValueError, RuntimeError) as e:
        print("Some error occured, retrying! -", e)

    time.sleep(3 * 60)  # wait 3 minutes

Some useful REST endpoints don't return with a content-type of application/json while still returning valid JSON, preventing the use of the library's built-in JSON parsing/processing.

I ran into this while working with the freely available US National Weather Service forecast API, which can only respond with GeoJSON (application/geo+json) or JSON LD (application/ld+json).

As a workaround in my project I've just called directly into the MagTag object's network.fetch_data(url) to get a response string to parse and wrangle but it'd be nice to do it in a more elegant way with these types of endpoints.

When trying to use code from https://learn.adafruit.com/matrix-portal-new-guide-scroller/code-the-matrix-portal
I hit the following exception:

code.py output:
Preloading font glyphs: 0123456789
obtaining time from adafruit.io server...
Connecting to AP ffiot
Getting time from IP address
New Hour, fetching new data...
Retrieving data...Obtaining guide info for guide 0...
Guide Title Adafruit Neo Trinkey
Traceback (most recent call last):
  File "code.py", line 104, in <module>
  File "code.py", line 77, in get_guide_info
  File "adafruit_portalbase/__init__.py", line 310, in set_text_color
AttributeError: 'NoneType' object cannot assign attribute 'color'

That is happening because self._text[index]["label"] may be None and code is not accounting for that.

Hello!

I am not sure of the initial intentions of the code, or the wider implications of changing it, so please do correct me if I am mistaken.

I ran into an issue where the data I was trying to pull has a content-type of application/geo+json. And so according to the logic here, because application/geo+json is not a "supported" type, the response is flagged as text.

Wanting to make 0 changes to the MatrixPortal code to get this to work, I then attempted to use text_transform to load the JSON myself and parse out what was needed. I added this to my code:

def text_transform(t):
    j = json.loads(t)
    return j['properties']['periods'][0]['shortForecast']

What I ran into here though was that the value of t was only the opening curly brace of the JSON, and so I could not actually load the data. After looking a bit into the code, I found that the problem lies here:

• network.py#L517
• matrixportal.py#L430

Because the response.text is returned as is from network.py , the call to apply a transformation to it in matrixportal.py is explicitly only passing along the first element of values, which in the case of it being text is just the first character.

I changed network.py#L517 to valeus = [response.text], and the text is passed along properly.

Is this a bug in the library, or am I missing something/misunderstanding something? I can open a PR if this change is deemed appropriate.

Thanks!

As stated in https://forums.adafruit.com/viewtopic.php?f=60&t=192584

The user was running into a 500 server error. It seems to be dying inside of network.py. I'm wondering if there's a better way to handle this or is it an error that we should leave as it is so the user can try/catch it?

This guide:
https://learn.adafruit.com/pyportal-discord-online-count/overview
uses code:
https://github.com/adafruit/Adafruit_Learning_System_Guides/blob/main/PyPortal_Discord/code.py
that parses the XML found in a downloaded SVG using regex. That requires a return type of CONTENT_TEXT from check_response() for the PyPortal library to work as expected:
https://github.com/adafruit/Adafruit_CircuitPython_PyPortal/blob/8031c95bbb209a2e4ad5bec7628fd02a4614da40/adafruit_pyportal/__init__.py#L310

However, this library will end up returning CONTENT_IMAGE since the fetch headers will contain content-type: image/svg_html.

That breaks the code for the above guide.

Opening issue here, but could maybe be fixed in one of the higher level libraries, like PyPortal?

For some reason json_transform(json_data) gets called on the second call to PyPortal.fetch() and it fails with MemoryError every time that happens. The first call to PyPortal.fetch() never fails, and all calls should get about the same response.

My PyPortal project is published on GitHub and this branch is the version running when reporting this issue:
https://github.com/blog-eivindgl-com/netatmo-pyportal-display/tree/report_issue_to_adafruit

The response doesn't contain much json data, and I don't specify any JSON path to be processed when initializing the PyPortal object.

Calls to pyportal.get_local_time(secrets["timezone"]) fail with error:

  File "/lib/adafruit_portalbase/__init__.py", line 403, in get_local_time
  File "/lib/adafruit_portalbase/network.py", line 233, in get_local_time
  File "/lib/adafruit_portalbase/network.py", line 216, in get_strftime
  File "/lib/adafruit_portalbase/network.py", line 202, in get_strftime
AttributeError: 'NoneType' object has no attribute 'get'

Printing on L198 (https://github.com/adafruit/Adafruit_CircuitPython_PortalBase/blob/main/adafruit_portalbase/network.py#L198):

            print(self._wifi)
            print(dir(self._wifi))
            print(self._wifi.requests)

Yields a None type for self._wifi.requests.

Getting time for timezone Etc/UTC
<WiFi object at 20003fe0>
['__class__', '__dict__', '__init__', '__module__', '__qualname__', 'connect', 'enabled', 'requests', 'esp', 'is_connected', 'neo_status', 'manager', 'neopix', '_manager']
None

Example code for where this occurs: https://learn.adafruit.com/pyportal-google-calendar-event-display/code-usage

I successfully connected to the network repeatedly, and then on a refresh, received two different errors on two different resets - one unknown network error and one authentication error. Each time, it failed to the error and the code stopped running, and it required a hard reset to begin running the code again.

Through discussion with @makermelissa, we both agree it should retry a network connection on a failure. She is currently working on a refactor, and this could be included.

Running Adafruit CircuitPython 6.3.0 on 2021-06-01; Adafruit PyPortal with samd51j20 with libraries from adafruit-circuitpython-bundle-6.x-mpy-20210717.

When pushing data to an Adafruit IO feed, including the precision parameter with floating point data ala pyportal.push_to_io(key, value, precision=x) fails with an error. pyportal.network.push_to_io(key, value, precision=x) (which is what pyportal.push_to_io appears to be a wrapper for) works, however.

Is this intentional or an oversight?

Using Blinka to run NASA image of the day example on a pitft results in the following error:
response = self.network.fetch(self.url, timeout=timeout) File "/home/pi/.local/lib/python3.7/site-packages/adafruit_portalbase/network.py", line 429, in fetch print("Free mem: ", gc.mem_free()) # pylint: disable=no-member AttributeError: module 'gc' has no attribute 'mem_free'

I think because gc.mem_free() is a circuitpython specific attribute and not present in Cpython gc module

I'm noticing we're getting some duplicates such as the MatrixPortal and PyPortal having essentially the same WiFi code and the MagTag and FunHouse boards both use the ESP32-S2 WiFi. I think there may need to be some modification of the use of the DotStar vs NeoPixel though.

Howdy,

This is really more of a question than an issue.

I was working on a project with a MagTag and I wanted to display a QR code MECARD as part of a nametag. When I attempted to including anything more than name, email address and a short note the code would fail. I tracked it down to this bit of code in PortalBase

        # generate the QR code
        for qrtype in range(1, 5):
            try:
                qrcode = adafruit_miniqr.QRCode(qr_type=qrtype)
                qrcode.add_data(qr_data)
                qrcode.make()
                break
            except RuntimeError:
                pass
                # print("Trying with larger code")
        else:
            raise RuntimeError("Could not make QR code")

I changed it to for qrtype in range(1, 6): and it worked just like I wanted so I'm all good. Along the way I got to learn how to turn a .py file into an .mpy file so I consider all of this time well spent.

So I'm wondering how the particular range was selected? Maybe a memory limitation on certain boards?

Thanks!

colin j.

There are missing type annotations for some functions in this library.

The typing module does not exist on CircuitPython devices so the import needs to be wrapped in try/except to catch the error for missing import. There is an example of how that is done here:

try:
    from typing import List, Tuple
except ImportError:
    pass

Once imported the typing annotations for the argument type(s), and return type(s) can be added to the function signature. Here is an example of a function that has had this done already:

def wrap_text_to_pixels(
    string: str, max_width: int, font=None, indent0: str = "", indent1: str = ""
) -> List[str]:

If you are new to Git or Github we have a guide about contributing to our projects here: https://learn.adafruit.com/contribute-to-circuitpython-with-git-and-github

There is also a guide that covers our CI utilities and how to run them locally to ensure they will pass in Github Actions here: https://learn.adafruit.com/creating-and-sharing-a-circuitpython-library/check-your-code In particular the pages: Sharing docs on ReadTheDocs and Check your code with pre-commit contain the tools to install and commands to run locally to run the checks.

If you are attempting to resolve this issue and need help, you can post a comment on this issue and tag both @FoamyGuy and @kattni or reach out to us on Discord: https://adafru.it/discord in the #circuitpython-dev channel.

The following locations are reported by mypy to be missing type annotations:

• adafruit_portalbase/network.py:87
• adafruit_portalbase/network.py:117
• adafruit_portalbase/network.py:126
• adafruit_portalbase/network.py:149
• adafruit_portalbase/network.py:161
• adafruit_portalbase/network.py:170
• adafruit_portalbase/network.py:223
• adafruit_portalbase/network.py:251
• adafruit_portalbase/network.py:319
• adafruit_portalbase/network.py:375
• adafruit_portalbase/network.py:409
• adafruit_portalbase/network.py:426
• adafruit_portalbase/network.py:442
• adafruit_portalbase/network.py:458
• adafruit_portalbase/network.py:484
• adafruit_portalbase/network.py:494
• adafruit_portalbase/network.py:503
• adafruit_portalbase/network.py:534
• adafruit_portalbase/network.py:541
• adafruit_portalbase/network.py:563
• adafruit_portalbase/network.py:611
• adafruit_portalbase/network.py:630
• adafruit_portalbase/wifi_esp32s2.py:42
• adafruit_portalbase/wifi_esp32s2.py:54
• adafruit_portalbase/wifi_esp32s2.py:69
• adafruit_portalbase/wifi_esp32s2.py:103
• adafruit_portalbase/wifi_coprocessor.py:45
• adafruit_portalbase/wifi_coprocessor.py:78
• adafruit_portalbase/wifi_coprocessor.py:85
• adafruit_portalbase/wifi_coprocessor.py:94
• adafruit_portalbase/graphics.py:44
• adafruit_portalbase/graphics.py:66
• adafruit_portalbase/graphics.py:112
• adafruit_portalbase/__init__.py:59
• adafruit_portalbase/__init__.py:100
• adafruit_portalbase/__init__.py:117
• adafruit_portalbase/__init__.py:130
• adafruit_portalbase/__init__.py:140
• adafruit_portalbase/__init__.py:216
• adafruit_portalbase/__init__.py:272
• adafruit_portalbase/__init__.py:286
• adafruit_portalbase/__init__.py:294
• adafruit_portalbase/__init__.py:302
• adafruit_portalbase/__init__.py:315
• adafruit_portalbase/__init__.py:336
• adafruit_portalbase/__init__.py:356
• adafruit_portalbase/__init__.py:359
• adafruit_portalbase/__init__.py:390
• adafruit_portalbase/__init__.py:409
• adafruit_portalbase/__init__.py:413
• adafruit_portalbase/__init__.py:423
• adafruit_portalbase/__init__.py:432
• adafruit_portalbase/__init__.py:441
• adafruit_portalbase/__init__.py:458

According to https://learn.adafruit.com/adafruit-pyportal/pyportal-hardware-faq#faq-3023990, the lights should act as follows:

Red = not connected to WiFi
Blue = connected to WiFi
Yellow = fetching data
Blue = got data
Cyan = file opening

The status colors are changeable at: https://github.com/adafruit/Adafruit_CircuitPython_PortalBase/blob/main/adafruit_portalbase/network.py#L59-L65

I see several issues here:

1. STATUS_CONNECTED is never used.
2. It looks like the get_strftime() function in network is not setting the status colors.
3. It appears the connected color is currently set to green.
4. It appears the Got Data is blue, though the original comments from the PyPortal library suggest it should have been green (https://github.com/adafruit/Adafruit_CircuitPython_PortalBase/blob/main/adafruit_portalbase/network.py#L281).
5. The value for STATUS_FETCHING suggests more of an orange color rather than yellow.

The splash attribute of PortalBase is not documented. The splash attribute holds the root displayio.Group() object.

Reference: PortalBase Source Code

The splash attribute is used directly in this project: MagTag Weather Project

It appears that display attribute is also not documented.

I'm using PyPortal 7.X to create a weather station display for Netatmo. I have an on-prem ASP.NET 6 proxy against the Netatmo API, so PyPortal qeries my proxy, which queries Netatmo and prepares the response in a json structure that's easy to display as widgets on the PyPortal.

I have the following program loop:

while True:
    # only query the online time once per hour (and on first run)
    if (not localtile_refresh) or (time.monotonic() - localtile_refresh) > 3600:
        try:
            print("Getting time from internet!")
            pyportal.get_local_time()
            localtile_refresh = time.monotonic()
        except RuntimeError as e:
            print("Some error occured, retrying! -", e)
            continue

    # only query the weather every 10 minutes (and on first run)
    if (not weather_refresh) or (time.monotonic() - weather_refresh) > 600:
        try:
            value = pyportal.fetch()
            gfx.draw_display(value)
            weather_refresh = time.monotonic()
        except RuntimeError as re:
            print("Some error occured, retrying! -", re)
            gfx.draw_error()
            continue
        except TimeoutError as te:
            print("Timeout - ", te)
            gfx.draw_error()
            continue

    gfx.draw_time()
    updateTime = .5
    if mode == "weekday":
        updateTime = 30

    time.sleep(updateTime)  # wait X seconds before updating anything again

I experience unexpected behavior when pyportal.get_local_time() executes the second time (and all times after). First time the Adafruit API at http://io.adafruit.com is called as expected. But the second time, even though the URL looks correct, my local ASP.NET proxy is called instead. That's the same IP as pyportal.fetch() calls.

This is a flowchart describing what happens:

I added Azure Application Insights to my ASP.NET proxy to figure out what's going on, and was surprised to see requests for the domain io.adafruit.com hitting my on-prem proxy. I've documented proof of the traffic in my own project in this issue:
blog-eivindgl-com/netatmo-pyportal-display#9

I don't know why this happens, but I suspect there's some kind of local DNS issue on the PyPortal that incorrectly maps io.adafruit.com to the same IP address as the URL of the fetch() call.

Seems to get asked a lot. How to remove a text label added with magtag.add_text()?

As far as I can tell, PR #65 broke the MagTag() constructor.

From the "Simple test" example in the docs, you are instructed to use the MagTag constructor to get an instance to the MagTag class, which you need to access MagTag-specific APIs:

from adafruit_magtag.magtag import MagTag

magtag = MagTag()
magtag.add_text(
    text_position=(
        50,
        (magtag.graphics.display.height // 2) - 1,
    ),
    text_scale=3,
)

that constructor ends up calling the Network class, which then ends up calling the constructor from PortalBase' Network:

https://github.com/adafruit/Adafruit_CircuitPython_MagTag/blob/32ec2d95131c5dea8ba33093cb3c5ad92133c9d5/adafruit_magtag/network.py#L70-L74

This is called without any secrets, and that's fine because PortalBase defaults to None. However, if you don't pass None, this piece of code will still be executed:

secrets here is the global secrets import. This code assumes that ssid and password will always be defined at the root level of the secrets. However, this assumption isn't really documented anywhere, and I've been previously fine with having a secrets structure of, for example,

secrets = {
  "api_root": "https://example.com",
  "wifi": {
    "ssid": "example",
    "password": "example"
  }
}

If you have a structure like mine, the aforementioned access of self._secrets["ssid"] and self._secrets["password"] will fail with a KeyError.

In my very humble opinion, this whole code should make sure that the secrets are actually there, and if not, it probably should just not attempt to connect to a network. Users can connect manually via wifi.radio.connect anyway, and provide the credentials there. Moreover, even if "you need to provide the SSID and password in the secrets root if you want to use networking" would be a acceptable compromise, this would still break the library for users who want to use MagTag-specific things, but aren't actually interested in networking. So either way, a bit of extra protection probably doesn't hurt. :)

I would submit a PR, but unfortunately I'm not set up to build CircuitPython myself, and it looks like it's quite a task to get that done... :(

/cc @tekktrik

It looks like adafruit_bitmap_font (repo) is used as a convenience function to _load_font:

This could be turned into an inline import. For example:

 if font not in self._fonts:
    from adafruit_bitmap_font import bitmap_font  # <-- Inline import 
    self._fonts[font] = bitmap_font.load_font(font)

Turning this into an inline import would make this a soft dependency. If it's not used it's not required. I was working up a small demo on the MagTag but not using anything about the display or Wi-Fi. I noticed that I still needed quite a few libraries. Some of these, like adafruit_fakerequests, aren't used even if I was using the full capabilities on the device. For example, this line could also use an inline import:

Making these inline imports would reduce the number of hard dependencies here. This would also provide some space savings and improve the speed at which the code starts up. This also shouldn't functionally impact any existing example code which makes use of functions requiring these libraries. They would still require the library to be installed.

Is this worth submitting a PR for?

I encountered a few MagTag and MatrixPortal projects that failed on me except if I create the proper secrets.py. And this even if I create a file settings.toml.

Initially I believed those library were not updated, but I found code here to support both the old and the new way is define in the code, but despite that I fail to make it work and I always get secrets.py related message.

Could someone have a look?

I know my settings.toml is OK because using the todbot trick below, I have the expected result: my IP addr: 192.168.192.153

https://github.com/todbot/circuitpython-tricks#what-the-heck-is-settingstoml

import os, wifi
print("connecting...")
wifi.radio.connect(ssid=os.getenv('CIRCUITPY_WIFI_SSID'),
                   password=os.getenv('CIRCUITPY_WIFI_PASSWORD'))
print("my IP addr:", wifi.radio.ipv4_address)

From a user on the forum:

*** USING LOCALFILE FOR DATA - NOT INTERNET!!! ***
Traceback (meest recente call laatst):
  Bestand "code.py", regel 36, in <module>
  Bestand "adafruit_pyportal/__init__.py", regel 310, in fetch
  Bestand "adafruit_portalbase/network.py", regel 509, in check_response
  Bestand "adafruit_portalbase/network.py", regel 535, in _get_headers
AttributeError: 'Fake_Requests' object heeft geen attribuut 'headers'

Guessing this is just something related to changes in CP.

Was using this BMP for testing:
https://github.com/adafruit/Adafruit_Learning_System_Guides/blob/main/MagTag_Weather/bmps/weather_bg.bmp

And then this:

Adafruit CircuitPython 7.0.0 on 2021-09-20; Adafruit MagTag with ESP32S2
>>> from adafruit_magtag.magtag import MagTag
>>> magtag = MagTag()
>>> magtag.graphics.set_background("/bmps/weather_bg.bmp")
>>> magtag.display.refresh()
>>>

results in all dark display.

It seems to be MagTag library related. This example does work:

Adafruit CircuitPython 7.0.0 on 2021-09-20; Adafruit MagTag with ESP32S2
>>> import board
>>> import displayio
>>> display = board.DISPLAY
>>> bitmap = displayio.OnDiskBitmap("/bmps/weather_bg.bmp")
>>> tile_grid = displayio.TileGrid(bitmap, pixel_shader=bitmap.pixel_shader)
>>> group = displayio.Group()
>>> group.append(tile_grid)
>>> display.show(group)
>>> display.refresh()
>>>