The Engine currently contains functionality for cycling to the previous_animation and next_animation (shown below). However, in the 2020-06-23 meeting we realized these are responsibilities of the application rather than the rendering engine. Therefore, we should move this code out of Engine, leaving the engine with the play_animation function for the application to tell the engine which animation should currently be playing.

https://github.com/ElixirSeattle/xebow/blob/e5589fdec01278b23f043fea198c164ce97eed01/lib/xebow/engine.ex#L31-L45

Nerves out of the box uses a built-in kernel module CONFIG_USB_ETH=y. This is widely compatible across operating systems, but lacks the HID device we need for a keyboard.

The conventional wisdom to create composite USB devices is to use Linux's ConfigFS, which is what xebow does. (It creates a composite NCM Ethernet + RNDIS Ethernet + HID device, using ConfigFS: https://github.com/ElixirSeattle/xebow/blob/master/lib/xebow/hid_gadget.ex). The problem seems to be that we can't create an Ethernet device through ConfigFS that mimics the portability and robustness of the built-in kernel module... the NCM + RNDIS devices are a hack (one works on windows (RNDIS), the other works on linux and macOS (NCM)).

We need to either try harder to mimic the same behavior as the built-in module through configFS, or scrap configFS altogether, and maybe write our own composite HID + Ethernet kernel module based on the one Nerves uses out of the box.

I consider this problem "very hard".

We would like to restyle the options list for the select element away from the browser default and to a style that matches the Xebow look and feel. Tailwind claims this requires a considerable amount of JavaScript, so waiting for their library seems reasonable unless someone feels passionate about writing this custom code.

Custom select controls like this require a considerable amount of JS to implement from scratch. We're planning to build some low-level libraries to make this easier with popular frameworks like React, Vue, and even Alpine.js in the near future, but in the mean time we recommend these reference guides when building your implementation:

https://www.w3.org/TR/wai-aria-practices/#Listbox
https://www.w3.org/TR/wai-aria-practices/examples/listbox/listbox-collapsible.html

^link

Design

Figma

In the live view, there are instances where an Option field type will not show the correct value for an animation.

This happens because the live view doesn't check what the current value for an option is when building the <select> option list.

Minimum reproduction steps:

1. Delete settings directory: rm -rf priv/settings
2. Run locally: iex -S mix phx.server
3. Open web interface, "Next Animation" to Hue Wave, and change the direction to "Up"
4. Menu -> Animations, turn off any of the animations except Hue Wave
5. Menu -> Dashboard, switch back to Hue Wave, and observe that Direction shows "Right", but the animation still plays as if it were "Up".

Note: The AFK API may change.

We would like to have a tutorial for Xebow integrators to be able to learn how to listen and respond to key events (press/release) in their applications. Part of this tutorial could include an explanation of keys.ex.

We would like to have a color picker widget in the web UI for users to be able to pick the colors they want to set LEDs to.

We could start out with a basic interface that supports HSV & RGB (and maybe the hex value) and lets a user pick a color by either adjusting the sliders or typing in numbers for each property's value. A square could show a preview of the color to the user. We could use the existing dependency chameleon to help with this task.

Mockup:

There are a couple TODO notes for refactoring the casting logic in RGBMatrix.Animation.Config. @amclain, I think you had a suggestion for how we could do this, didn't you?

• TODO: better key casting (lib/rgb_matrix/animation/config.ex#L77)
• TODO: better value casting (lib/rgb_matrix/animation/config.ex#L80)

May overlap with: #43

See comment: #54 (comment)

There was another comment somewhere about not coupling Keys to LEDs and looking up by coords to feed the event to interact. Can't find the comment.

There's also this comment, which might be more about layouts than about interactions, but I feel this is all tied together: #54 (comment)

I noticed this isn't working right and will look into it.

The animations have multiple places where they need to be touched up.

SolidReactive module has three TODOs, one FIXME, and one Credo-suggested refactor to reduce nesting depth:

• Nesting depth refactor (lib/rgb_matrix/animation/solid_reactive.ex#L72-L87)
• FIXME: leaves color 1 away from base (lib/rgb_matrix/animation/solid_reactive.ex#L89)
• TODO: configurable base color (lib/rgb_matrix/animation/solid_reactive.ex#L51)
• TODO: take speed into account (lib/rgb_matrix/animation/solid_reactive.ex#L76)
• TODO: we can optimize this by rewriting the above instead of filtering here (lib/rgb_matrix/animation/solid_reactive.ex#L90)

SolidColor has a TODO:

• TODO: configurable base color (lib/rgb_matrix/animation/solid_color.ex#L23)

Breathing:

• TODO: configurable base color (lib/rgb_matrix/animation/breathing.ex#L25)

HueWave:

• TODO: fixme (lib/rgb_matrix/animation/hue_wave.ex#L62)

This issue was double-submitted. See #39.

At least it should provide a default implementation for interact since not all animations need to implement this.

Description

If all of the animations are disabled, attempting to cycle to the next/previous animation causes a crash. Based on the error message, it appears that the config is trying to be retrieved for a non-existent animation.

** (UndefinedFunctionError) function nil.config/0 is undefined
    nil.config()
    (xebow 0.1.0) lib/rgb_matrix/animation.ex:105: RGBMatrix.Animation.get_config/1

Expected behavior

Nothing happens when attempting to cycle the animation (no-op).

Steps to reproduce

Disable all active animations on the animations page.

Navigate back to the dashboard and cycle to the next or previous animation.

The following error is printed in the console.

[error] GenServer RGBMatrix.Engine terminating
** (UndefinedFunctionError) function nil.config/0 is undefined
    nil.config()
    (xebow 0.1.0) lib/rgb_matrix/animation.ex:105: RGBMatrix.Animation.get_config/1
    (xebow 0.1.0) lib/rgb_matrix/engine.ex:233: RGBMatrix.Engine.inform_configurables/1
    (xebow 0.1.0) lib/rgb_matrix/engine.ex:182: RGBMatrix.Engine.handle_cast/2
    (stdlib 3.13) gen_server.erl:680: :gen_server.try_dispatch/4
    (stdlib 3.13) gen_server.erl:756: :gen_server.handle_msg/6
    (stdlib 3.13) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: {:"$gen_cast", {:set_animation, nil}}
State: %RGBMatrix.Engine.State{animation: %RGBMatrix.Animation{config:
%RGBMatrix.Animation.SolidReactive.Config{direction: :random, distance: 180,
speed: 4}, state: %RGBMatrix.Animation.SolidReactive.State{color:
%Chameleon.HSV{h: 190, s: 100, v: 100}, first_render: false, hits: %{}, leds:
[%Layout.LED{id: :l001, x: 0, y: 0}, %Layout.LED{id: :l002, x: 1, y: 0},
%Layout.LED{id: :l003, x: 2, y: 0}, %Layout.LED{id: :l004, x: 0, y: 1},
%Layout.LED{id: :l005, x: 1, y: 1}, %Layout.LED{id: :l006, x: 2, y: 1},
%Layout.LED{id: :l007, x: 0, y: 2}, %Layout.LED{id: :l008, x: 1, y: 2},
%Layout.LED{id: :l009, x: 2, y: 2}, %Layout.LED{id: :l010, x: 0, y: 3},
%Layout.LED{id: :l011, x: 1, y: 3}, %Layout.LED{id: :l012, x: 2, y: 3}],
paused: true, tick: 0}, type: RGBMatrix.Animation.SolidReactive}, configurables:
#MapSet<[#Function<5.66497960/1 in XebowWeb.MatrixLive.register_with_engine!/0>]>,
last_frame: %{l001: %Chameleon.HSV{h: 190, s: 100, v: 100},
l002: %Chameleon.HSV{h: 190, s: 100, v: 100}, l003: %Chameleon.HSV{h: 190, s: 100, v: 100},
l004: %Chameleon.HSV{h: 190, s: 100, v: 100}, l005: %Chameleon.HSV{h: 190, s: 100, v: 100},
l006: %Chameleon.HSV{h: 190, s: 100, v: 100}, l007: %Chameleon.HSV{h: 190, s: 100, v: 100},
l008: %Chameleon.HSV{h: 190, s: 100, v: 100}, l009: %Chameleon.HSV{h: 190, s: 100, v: 100},
l010: %Chameleon.HSV{h: 190, s: 100, v: 100}, l011: %Chameleon.HSV{h: 190, s: 100, v: 100},
l012: %Chameleon.HSV{h: 190, s: 100, v: 100}}, leds: [%Layout.LED{id: :l001, x: 0, y: 0},
%Layout.LED{id: :l002, x: 1, y: 0}, %Layout.LED{id: :l003, x: 2, y: 0},
%Layout.LED{id: :l004, x: 0, y: 1}, %Layout.LED{id: :l005, x: 1, y: 1},
%Layout.LED{id: :l006, x: 2, y: 1}, %Layout.LED{id: :l007, x: 0, y: 2},
%Layout.LED{id: :l008, x: 1, y: 2}, %Layout.LED{id: :l009, x: 2, y: 2},
%Layout.LED{id: :l010, x: 0, y: 3}, %Layout.LED{id: :l011, x: 1, y: 3},
%Layout.LED{id: :l012, x: 2, y: 3}], paintables:
#MapSet<[#Function<4.66497960/1 in XebowWeb.MatrixLive.register_with_engine!/0>]>,
timer: nil}

Originally I had the phoenix live-view responsible for casting values coming from the client, strictly requiring that all values were correct before being passed in to Config.update. This is obviously not that great... it would be much nicer if Config.update returned either :ok, or {:error, :cast_error} or something like that. This would be a nicer developer experience. As of right now, it will crash with a pattern match error, which is not ideal.

• TODO: better key casting (lib/rgb_matrix/animation/config.ex#L77)
• TODO: better value casting (lib/rgb_matrix/animation/config.ex#L80)

We would like to add the ability for a user to change the mode of the USB Ethernet endpoint that Xebow exposes to the host.

Xebow currently exposes two Ethernet endpoints, RNDIS and NCM. This is because neither driver is fully compatible with all operating systems (see #91). If Xebow is only being connected to a single operating system type, a user may want to disable the other unused Ethernet endpoint which will show up as an invalid device, and may also be periodically rediscovered by the O/S and cause notifications to pop up.

Note that disabling one type of Ethernet endpoint may result in the device being unusable if it is connected to another type of operating system. Due to this, we also need to expose a recovery mechanism if the user encounters this state.

Acceptance Criteria

• On the settings page, allow the user to select between the following USB Ethernet modes:
  • Auto - Both RNDIS and NCM endpoints are available as connection options.
    • It is OK for one of these endpoints to be disabled after the other connects successfully, as long as they both become available if neither has a connection (eliminates the need for the network bridge).
  • RNDIS - Only the RNDIS endpoint is active.
  • NCM - Only the NCM endpoint is active.
• The user can hold a key sequence on device boot to change this setting without the web UI (recovery mechanism).
  • Holding keys 1 & 2 changes the setting to auto.

Design

https://www.figma.com/file/RMYWHeXJoDErWii1jm8C7k/Xebow?node-id=45%3A44

Depends on #43.

We would like to have documentation that shows a new animation designer how to write an animation for Xebow.

This may include topics like:

• Where the built-in animation files are stored (here)
• How to implement (use) Animation
• Properties in %Animation{} that are available to an animation designer, including how to use opaque state
• How to use the field macro
• Understanding the layout of pixels (#43)
• Any important details for an animation designer about how Engine renders animations

I commented them out in my PR 'cause 😅

Currently, animations are hard-coded in the Animation module. We should think about some way of populating the list of types and the list of animations dynamically, based on the animations which exist at startup.

Good topic for the next planning meetup.

https://github.com/ElixirSeattle/xebow/blob/abb857f8d76ed5ac2b969de1e81911972b2e8c63/lib/xebow/animation.ex#L39-L55

We would like Engine to be able to play multiple animations at the same time, layering them on top of each other. Animations may either stop naturally after they have reached their number of loops played, or forcefully by instructing the engine to stop an animation that is currently in progress. The latter is especially useful for infinitely running animations.

Right now, there are no friendly ways to control the animation. There's no standardized means of telling the application to play the next/previous animation, or to change the animation's configuration.

This is especially apparent in lib/xebow_web/live/matrix_live.ex on L71, where there's logic for switching animations duplicated between the live view module and lib/xebow/keyboard.ex.

We'll need to figure out a way to do this. I'm pretty sure it'll involve some refactoring, and will likely touch a couple other issues.

As discussed in the 2020-06-23 meeting, we need to design a system to lay out pixels that don't conform to a perfect grid (like the Keybow does). We need to be able to accommodate the zig-zag layout of keys and pixels like on a full size keyboard.

Keybow's grid:

Zig-zag of a full size keyboard:

@doughsay & @dhedlund: Considering this project doesn't have unit tests yet, how would you feel if I implemented ESpec and configured CI? I've enjoyed using this library on other projects, and its mocks are a nice way to work around physical hardware components that aren't available during the CI run. I've also found the ESpec style of testing (inspired by RSpec) pleasant to read as well, but I realize that's subjective.

What are your thoughts?

Need to have a discussion about this, a lot of us feel like something's wrong here...

Migrate the work done in doughsay/rgb_matrix_simulator into Xebow. This allows the Xebow to be rendered inside of a LiveView, and animation settings can be adjusted here as well.

Conceptual screenshot:

Related: #9
Related: 96c4fa8#diff-6023be6004fce4718dad3dafb576d258L6

@doughsay I noticed after PR #9 was merged the hardware target name was changed from keybow to kebow (without the y). Just want to point out that Keybow, the product, does have a y in it. Or are you going for this Kebow? 😜

It just threw me off when building the firmware, so thought I'd point it out. Am I missing something about why the name ended up the way it is now?

Currently, the typespecs are not consistent with each other. Sometimes they're declared with a parameter name, sometimes just the type. We should pick one style and try to stick to it, and probably before there are a bunch of people writing code for this. It will help lower the barrier to entry for newcomers.

For instance, I find it much more readable to have literal notation whenever possible. For example, ✔️[any()] instead of ❌list(any()), or ✔️{atom(), binary()} instead of ❌tuple(atom(), binary()), except for when the functional version is more readable or conveys better meaning, like ✔️map() instead of ❌%{optional(any) => any}. I also go for functional types over bare names (:heavy_check_mark:any() vs ❌any). (This last point is just me, so if anybody thinks it's extra visual noise, I'm just fine with bare names.)

Parameterized types should be reserved for when they add to typespec clarity, like when there are multiple identical types or when the return type directly uses a given parameter. Honestly, I like having lots of typespecs, so the case of multiple identical types should be abstracted to named types. (E.g. @spec foo(integer(), integer(), integer()) ➡️ @spec foo(hours(), minutes(), seconds()) or similar, where hours(), minutes(), and seconds() are defined types of integer().)

I like the way ExDoc styles the types, and the way the Erlang docs do their specs (usually). It's highly legible. If we followed mostly the same style, then it would make it a lot easier to jump from documentation to code. This is personal preference and I'm open to any other style, as long as it's fairly consistent. AFAIK, ExDoc normalizes things when it creates the docs, so the published versions are good, but consistency is nice for people editing and reading the code, too.

After pressing arbitrary keys for a while, every now and then one of the key presses ends up triggering twice. It doesn't matter if the keys are pressed fast or slow, and I have also tried varying the timing between different key presses. It feels like the issue happens more often after a small pause, but that may just be coincidence. The first few occurrences in the sequence in the screenshot seem to happen just after 50 characters, but that may also be coincidence. In the 123, 456, 789 sequence it feels like 1, 7, 5 trigger the most often. I have tried combinations of keys that are not in that group, like 2, 6, 8, and see the same issue of keys triggering twice. I have also tried the keybow on a couple different surfaces to try to eliminate mechanical bouncing of the keys if that was potentially an issue.

Is anyone else able to reproduce this?

We would like to add a page for a user to be able to choose which functionality to bind to which keys. We can start out by getting the list of scan codes from AFK and displaying them in a list for the user to select from.

This is the first iteration of this design and will be expanded on in the future.

Acceptance Criteria

• Layers can be added and removed.
• (Optional) Layers can be renamed.
• A representation of the keyboard shows the mapped keys.
• Selecting a layer shows the key mapping for that layer.
• The user can select a key on the keyboard, and then select the AFK scan code from a list. The code will then be mapped to the key.
• (Optional) A search box narrows down the list of codes.
  • The text in the search box can be cleared by clicking an icon next to it.

Any optional items that are deferred should be tracked in a new ticket.

Design

https://www.figma.com/file/RMYWHeXJoDErWii1jm8C7k/Xebow?node-id=59%3A82

Depends on #43.

As mentioned in the 2020-06-23 meeting, the Keybow's physical pixels are rendered based on a position-dependent list, @pixels, in utils.ex. In this list the elements have to be in the same order that the pixels are arranged on the Keybow on the SPI bus. We would like hardware layer that controls the pixels (RGBMatrix) to be responsible for translating the pixels in a Frame to the correct physical representation.

TBD: The strategy we choose to refactor this will most likely be dependent on how we decide to design pixel layouts in #43.

The physical keys are given IDs in the code that both AFK and RGBMatrix are (or will be) using. The IDs are atoms like :k001, :k002, etc.

The keys are currently arranged like this:

+-----+-----+-----+
|  1  |  5  |  9  |
+-----+-----+-----+
|  2  |  6  |  10 |
+-----+-----+-----+
|  3  |  7  |  11 |
+-----+-----+-----+
|  4  |  8  |  12 |
+-----+-----+-----+

whereas they should be arranged like this:

+-----+-----+-----+
|  1  |  2  |  3  |
+-----+-----+-----+
|  4  |  5  |  6  |
+-----+-----+-----+
|  7  |  8  |  9  |
+-----+-----+-----+
|  10 |  11 |  12 |
+-----+-----+-----+

This follows the conventions of the wider mechanical keyboarding community better and removes keybow specific hardware coupling.

If a user bypasses the mix setup step because they don't need to run the web UI locally, the workflow to burn the firmware does not include the npm install step. This results in the UI not rendering correctly when hosted from the hardware due to the missing assets.

A new major release of nerves_system_rpi0 is on the way and is currently in rc0. Adopting this release will require a small change to validating the firmware. For Xebow, we should be ok running Nerves.Runtime.validate_firmware() when the application starts.

Our existing users will also need to run the firmware validation before upgrading.

Excerpt from the release announcement:

Each rpi* system now does a simple firmware validation check before loading the rootfs. [...] This change requires the firmware to be marked as valid at runtime.

Can I update existing devices?
Nerves rpi* systems < v2.0.0 don’t have this UBoot variable set, so any attempt to update via fwup (such as SSH or upload.sh script) will fail [...] so before updating, you’ll need to manually validate the running firmware with:
Nerves.Runtime.validate_firmware()

https://elixirforum.com/t/nerves-raspberry-pi-systems-v2-0-0-release-candidates/33446

https://hostiledeveloper.com/2020/06/19/organizing-liveview-logic-with-presentation-models.html

The live-view assigns are hard to manage, this refactor will clean that up.

We need a second pass at animation configuration:

• Make it so use Animation is all you need and can use the field macros directly.
• Clean up the DSL macros so they work right (i.e. so you can use things like module attributes when using them)
• Somehow have definable documentation per field that the live-view UI can use

We would like to include backup & restore functionality so that a user can save their settings off of the device for safe keeping, and later restore their settings if the device becomes misconfigured.

One option we discussed is to save all of the settings files in a specific directory, then zip/overwrite that entire directory on backup/restore.

Acceptance Criteria

Backup

• The settings page (/settings) provides a button to backup the configuration.
• The backup process allows the user to download a single file (like a zip archive) to their computer.

Restore

• The settings page provides a button to restore the configuration.
• The restore process prompts the user for a singe file to restore the configuration from, which is uploaded from their computer to the device.

Design

https://www.figma.com/file/RMYWHeXJoDErWii1jm8C7k/Xebow?node-id=45%3A44

Xebow should be able to start up the BEAM on the host, without being dependent on running on the target hardware. This will help with working on the web interface, designing animations, and being able to run commands like iex -S mix to experiment with the code without flashing the hardware.

We would like to redesign the main page of the Xebow Studio UI.

Notes

• The design is available in Figma for margins, colors, fonts, etc.
• The following components were inspired by tailwindcss, which can be included as a dependency. (fomantic-ui is another option if we want more pre-built components.)
• Icons are from the fontawesome free set.
• We need to try to load assets from the hardware rather than from the web, as the user may not have an internet connection when using the device. If this is not possible, reasonable alternatives should be used when the user is offline.
• The speedometer icon is intended to link to the telemetry dashboard at /telemetry. This should open in a new tab, as the page does not contain a way to navigate back to Xebow Studio.
• The hamburger icon is intended for menu items, like device settings, which might not exist in the first iteration. This is ok and the icon can be omitted.
• The design does not need to be pixel-perfect if another implementation looks better.

Design

nerves_system_keybow needs to be updated to the latest nerves system, rpi0 v1.12.2, and this project's dependency needs to be updated.

We would like to replace the UI functionality of cycling to the next/previous animation with a dropdown menu that allows the user to pick the specific animation they want to play from a list.

Acceptance Criteria

• The animation dropdown shows the name of the currently playing animation as its current value.
• The animation dropdown shows a list of active animations (configured from the Animations page) in the dropdown options list.
• Selecting a dropdown option starts playing that animation.

Design

https://www.figma.com/file/RMYWHeXJoDErWii1jm8C7k/Xebow?node-id=1%3A3

The only way we've found to get the Xebow's networking to work on macOS Catalina is to use https://github.com/jwise/HoRNDIS

And specifically, HoRNDIS also doesn't work out of the box on Cataline, so we had to use this package someone provided in the comments: jwise/HoRNDIS#102 (comment)

This should all be mentioned in the README.

We would like to configure the USB gadget device before the BEAM boots so that in the future we can have access to it from a separate application that can send simple HID keyboard reports, like pressing a key to get into the host's BIOS. Due to this we also need to migrate off of usb_gadget since we won't have access to Elixir at this stage.

There's something wrong with the type, or with the module's usage later on. Dialyzer throws fits.

https://github.com/ElixirSeattle/xebow/blob/626c7cb7ac5c818f51856e5b216df7f79351e459/lib/layout.ex#L8-L14

This fixes live-view showing black when an animation is not actively animating

nerves_system_br v1.12.0 has been released. ^link
We need to update ElixirSeattle/nerves_system_keybow and bump the nerves_system_keybow dependency of this project.

With the use Animation refactor, we can look at making the UI render the description and name using the metadata provided by the doc option given in field definitions.

Because doc is optional, we should keep our current method as a fallback for when no name is provided.

When building the firmware, the Phoenix assets need to first be fetched, installed, and deployed. While this is easy enough, mix firmware doesn't check or know when the Phoenix assets are missing, and it displays no errors or warnings, so the Live View fails to work correctly.

We should make the mix firmware (and mix firmware.burn) task ask if a user wants to install the Phoenix deps, similar to how mix phx.new does.

Phoenix does it like this: https://github.com/phoenixframework/phoenix/blob/1f0040a513c25a0ffee78931357fde44ac5d0583/installer/lib/mix/tasks/phx.new.ex#L150

Xebow might not need to be this involved. It would be really cool if we could make it more automagic and somehow detect whether the Phoenix assets need to be installed, but it's probably not strictly necessary.

Bare minimum would be a couple sentences in the readme about how to do this.

new and interact both return {render_in, state} and render returns {render_in, colors, state}. They are so close to being the same, let's see if we can come up with a good refactor here that makes these all a shared struct definition instead of "almost the same" tuples.

We would like to expose each of the animation types on a page where the user can toggle each one on or off. This will include or exclude the respective animation in the list of animations that can be selected (cycled through) on the dashboard.

This is the first iteration of this design and will be expanded on in the future.

Acceptance Criteria

• A page at /animations displays the animation types.
• Each animation type can be toggled on or off, which makes it either available or unavailable when the user cycles animations.
  • This works when cycling animations on the dashboard page.
  • This works when cycling animations with the hardware keypad.
• A menu item exists to navigate to the animations page.

Design

https://www.figma.com/file/RMYWHeXJoDErWii1jm8C7k/Xebow?node-id=41%3A0