Gamepad Inputs about scenic HOT 6 CLOSED

I think this is a great idea, but I'm on the side of it not being in core. I work on embedded devices, and I am often trying to slim down builds and code. I would prefer to need to pull these things into the project. Many times our devices have a screen but don't need an input device like a gaming controller.

Having external scenic libraries can make things a little harder for people to get started or to make sure that things are staying up to date. As long as the external libraries have good dep specs, this shouldn't be a problem.

The other thing that I like is that something in x scenic library doesn't have to slow down the core if there are improvements for others not using controllers.

from scenic.

I'm happy to take a crack at this with at least the Xbox 360 controller, though it ultimately comes down to what GLFW supports.

To do this, we're probably going to have to:

• Figure out/list what actual events we want to support ("joysticks" cover a crazy-large range of things...I'm happy to use the XB360 as the first use case since it's so common)
• Add support for the events in the scenic GLFW driver
• Add support for the events in scenic proper

I figure @boydm might have some opinions on this, given his prior experience. ;)

from scenic.

Before either of us spends any time writing code, here are a few open ended questions. Answering these questions is part of why it wasn't in the original release...

1. Does it need to work on Nerves devices? i.e. is connecting a game controller to a Nerves device an interesting scenario? I tend to think yes, but am not convinced. This makes the problem much harder as GLFW is not used in that driver. So the design needs to take that into account.

By the way, if you want to play with Scenic on Nerves, it is already open. Just not advertised yet. I'm still fretting over perf. Frank Hunleth is on the case helping to figure out if we can make it better. Perf is ok, right up until it is sent to the graphics chip for rendering. Even then it isn't bad, just isn't great. You can play with it yourself by looking at the rep nerves_scenic_example_app. Ignore the warnings I put in the readme to keep people away for now...

2. Which controllers are interesting? Yes, the xbox controller is common. Others are interesting too. Want to see a decent discussion here.

3. What about custom input devices? In a dedicated device the controller equivalent may be custom hardware. Would be good to design this with an eye toward a standard way of building new devices. There should be some standard guidance on how to do this.

That sort of thing. Lets have a good discussion here....

from scenic.

Another open question: Should this go in the core Scenic package. I'm wary of loading up the core scenic library with code that is only applicable to a subset of apps. Maybe this makes the bar, maybe it goes into an optional package. Open to thoughts here too.

For example, I'm tempted to pull all the components out of the core lib into a separate package.

from scenic.

So, the places that are kind of informative about how the input systems work are:

For Nerves (plus touch of some variety), see:

• https://github.com/boydm/scenic_driver_nerves_touch/blob/master/lib/scenic_driver_nerves_touch.ex

For normal scenic GLFW, see:

• https://github.com/boydm/scenic_driver_glfw/blob/master/lib/glfw/input.ex#L229
• https://github.com/boydm/scenic_driver_glfw/blob/master/c_src/comms.c#L333

And for how input actually gets dispatched and used by components, text field is an informative example:

• https://github.com/boydm/scenic/blob/master/lib/scenic/component/input/text_field.ex#L267

It looks to me (and this could be me misinterpreting what I'm seeing!) like input events are parsed from the driver binary format (which is per driver, presumably) and then translated into input tuples. The things that this makes me wonder are:

• Why are keycodes given as strings instead of atoms? Comparisons with atoms are faster, no?
• Are we really wanting components to be responsible for knowing how to parse the raw input tuples that bubble up to them?
• Do we want a way for components/drivers to be able to say "Yo, I'm a color picker, but you don't have any mouse/touch/pointing-like HIDs attached...I'm not gonna work right."?
• Is there a way of providing structs (or something with better tooling) than just raw tuples? Records would've been a nice fit I think on the performance curve, but they've been deprecated Because Reasons(tm). Maps I think would be too heavy-weight.

Now, I'll give some thoughts on @boydm 's questions:

Does it need to work on Nerves devices?

I would imagine so, since presumably a neat case would be setting up a Pi running Nerves and like an arcade ROM or something. We shouldn't do anything that precludes Nerves from being able to use it--that said, looking at various instructions (being unfamiliar with how the kernel and device drivers are setup for Nerves) it looks like it'll take end-users a little effort to make it work.

Which controllers are interesting? Yes, the xbox controller is common. Others are interesting too. Want to see a decent discussion here.

The XB360 controller is, according to the Steam hardware survey, the most-used controller. But, yeah, we would want to be able to support other devices like the PS4 or Steam controllers.

But, like, we might also want to properly support things like thumsticks, sliders, discrete random buttons, and other types of weird gaming peripherals. We might also want to support input devices like thermometers, GPS or IMU units, or other "input" devices.

I don't think the current architecture prevents any of that, because of how it passes input tuples around. It also doesn't provide any real special support either though. :(

What about custom input devices?

See above.

There should be some standard guidance on how to do this.

Yep. Honestly, a simple tutorial for adding an "input device" powered by the magic of /proc/uptime or some simple source of system data might be in order...showing just those parts of the messaging.

The standard guidance is also a lot simpler if we document the current input events so people can write components to a standard interface. Maybe we need like a central registry of these things?

I'm tempted to pull all the components out of the core lib into a separate package.

I can 100% see the argument for doing that. You'd want a way of components to declare their version or caps though (same as for drivers), right?

from scenic.

Another open question: Should this go in the core Scenic package. I'm wary of loading up the core scenic library with code that is only applicable to a subset of apps. Maybe this makes the bar, maybe it goes into an optional package. Open to thoughts here too.

Maybe there's an abstraction of non-standard (not keyboard or mouse) input that covers gamepads and expected input devices in for embedded systems (buttons/switches/dials). Game controller modules outside of the core scope could implement the @behavior of input.

As a developer, I want to instantiate a game controller process and subscribe to updates from it. .

Something like

{:ok, pid} = GenServer.start_link(GameController, %{subscribers: [self()]})

def handle_call({:scenic_input, input, _GameController, state) do
   %{:controller1 => %{:buttons => ... }, ...}
end

A specially built embedded device might have a unique driver that implements the input behavior and a unique pattern for the update eg. %{:button1 => :pressed, :dial => 0.3, :temperature => 40}

Regarding "maps being too heavyweight" -- this might have only been in context of the existing code. However as a developer, maps would be easier to parse -- though I think leaving the atual Term coming from the game controller module open ended makes sense (could vary from device to device).

Speaking more concretely about game controllers specifically, I have noticed some oddities with HTML gamepad controllers relating to unplugging and replugging devices in. So perhaps :controller1 is a bad key name and better of to use a pid or port number, then require the developer to track which controllers map to which user input functions eg. <PID 123> => User1 and/or User1 => . If that controller is disconnected then re-connected, the developer's project should detect that <PID 123> is not included in the GameController call and unassign the relationship between User1 and <PID 123> so that they can reassign the new process to User1. All of this I imagine taking place in the developer's code. A GameController module should just calling in with the controllers connected and their input values.

from scenic.