[RFC] Add proper support for dynamic channel creation for things about openhab-core HOT 10 OPEN

1. Click + on things file
2. Select astro binding
3. Select "astro:sun"
4. Edit location
5. Click "Create Thing"
6. navigate to the Thing in MainUI, open the code tab
7. ...

That includes a bunch of unnecessary steps that are only really necessary only if something went wrong.

0. If you already have an Astro thing skip to step 5
1. Click the Inbox
2. If Astro was just installed, you will already have a local-sun and local-moon Thing in the Inbox. Select local-sun and go to step 5
3. If you've previously ignored the local-sun Thing, click "show ignored". Select the local-sun Thing and go to step
4. Only if something went wrong do you need to click + and go through all the steps to create the Thing. Note the location can be 0,0 for all it matters. You're going to edit that in the .things file anyway.
5. Navigate to the Thing in MainUI, open the code tab ...

But that's getting off topic.

To summarize:

1. Standardization is good. Individual bindings shouldn't need to invent a new way to store information about discovered Things.
2. I don't understand how/why you do not see the Chanels in the Code tab even for discovered Things. That is IMO a bug, maybe worthy of a separate issue. You should always be able to see the Chanels in the code tab and it definitely shouldn't be the case where it's different between different users.
3. I think a distinction needs to be made between truly automatically discovered Channels and hard coded Channels. Astro and Network binding Things always have the same Channels. These are not auto discovered, they are hard coded. I don't think it makes sense to treat them the same as the Channels of the Shelly or Zwave binding (for example).
4. I have no opinion on the rest.

from openhab-core.

Thus the discovered channels should also be shown in the Code tab.

I don't have any opinions on this RFC except that at least for Zwave and Zigbee, the discovered channels are shown in the Code tab. If they are not shown for other bindings (Shelly), maybe that part at least is a problem with the binding implementation instead of core?

but currently it does not work for textually defined things.

Isn't it the whole point of textually defined things to textually define everything and not do discovery? It seems very weird to me to have a textually defined Thing but not include the Channels in the textual definition. I'm totally on board with standardization of this sort of thing.

from openhab-core.

... at least for Zwave and Zigbee, the discovered channels are shown in the Code tab.

This is what gets shown for a fully discovered Smart Color Button (Z-Wave) with multiple channels.

UID: zwave:device:controller:node33
label: "Philio PSR04 (Node 33): Smart Color Button"
thingTypeUID: zwave:philio_psr04_00_000
configuration:
  config_1_1: 0
  config_2_1: 99
  config_10_1: 12
  wakeup_node: 1
  wakeup_interval: 86400
  group_1:
    - controller
  config_25_1: 0
  config_26_1: 1
  group_2:
    - controller
  node_id: 33
bridgeUID: zwave:serial_zstick:controller

For me it does not show the channels there (OH 4.0.0). Do you see something different?

It seems very weird to me to have a textually defined Thing but not include the Channels in the textual definition.

Since the docs are always lacking the idea is to just define the plain thing and still have auto discovery / automatic channel creation. Once the device channels are properly discovered one can than edit the textual thing config to include the channel definition if desired (which then would be available from the code tab).
Basically discover what you use and then "freeze" that.

Another idea would be to have an additional thing state TARGET_INITIALISATION with the status detail CHANNEL_DISCOVERYand REGISTERING_OPENHAB.
That way the device states would clearly be decoupled from the thing initialization states which IMHO makes sense

from openhab-core.

This is what gets shown for a fully discovered Smart Color Button (Z-Wave) with multiple channels.

That's weird as all of my Zwave and Zigbee (and Shelly for that matter) Things have all their Channels listed in the Code tab.

Here are a few but I've looked at them all.

UID: zwave:device:zw_controller:node7
label: Home Energy Meter
thingTypeUID: zwave:aeon_dsb09_00_000
configuration:
  config_10_1: 5
  config_102_4_00000100: 0
  group_1:
    - controller
  config_102_4_00002000: 0
  config_101_4_00000008: 1
  config_101_4_00000800: 1
  config_101_4_00000004: 1
  config_101_4_00000400: 0
  config_112_4: 720
  config_101_4_00000001: 0
  config_101_4_00000002: 1
  config_101_4_00000200: 1
  config_103_4_00002000: 0
  config_103_4_00000100: 0
  config_255_4_wo: 0
  config_102_4_00001000: 0
  config_1_2: 110
  config_101_4_00001000: 1
  config_7_2: 25
  config_9_1: 5
  config_5_2: 25
  config_102_4_00000008: 0
  config_102_4_00000800: 0
  config_103_4_00001000: 0
  config_11_1: 5
  config_102_4_00000004: 0
  config_102_4_00000400: 0
  config_102_4_00000002: 0
  config_102_4_00000200: 0
  config_102_4_00000001: 0
  config_110_4_wo: 0
  config_113_4: 720
  config_101_4_00000100: 1
  config_111_4: 720
  config_101_4_00002000: 0
  config_103_4_00000001: 0
  config_103_4_00000002: 0
  config_103_4_00000200: 0
  config_103_4_00000004: 0
  config_103_4_00000400: 0
  config_3_1_00000008: 0
  config_103_4_00000008: 0
  config_103_4_00000800: 0
  config_3_1_00000004: 0
  config_3_1_00000002: 0
  config_3_1_00000001: 0
  config_100_4_wo: 0
  config_6_2: 25
  config_8_1: 5
  config_4_2: 25
  node_id: 7
bridgeUID: zwave:serial_zstick:zw_controller
location: Global
channels:
  - id: meter_watts
    channelTypeUID: zwave:meter_watts
    label: Electric meter (watts)
    description: Indicates the instantaneous power consumption
    configuration: {}
  - id: meter_kwh3
    channelTypeUID: zwave:meter_kwh
    label: Electric meter (kWh) 3
    description: Indicates the energy consumption (kWh)
    configuration: {}
  - id: meter_kwh2
    channelTypeUID: zwave:meter_kwh
    label: Electric meter (kWh) 2
    description: Indicates the energy consumption (kWh)
    configuration: {}
  - id: battery-level
    channelTypeUID: system:battery-level
    label: Battery Level
    description: null
    configuration: {}
  - id: meter_kwh
    channelTypeUID: zwave:meter_kwh
    label: Electric meter (kWh)
    description: Indicates the energy consumption (kWh)
    configuration: {}
  - id: meter_reset
    channelTypeUID: zwave:meter_reset
    label: Reset Meter
    description: Reset the meter
    configuration: {}
  - id: meter_kwh1
    channelTypeUID: zwave:meter_kwh
    label: Electric meter (kWh) 1
    description: Indicates the energy consumption (kWh)
    configuration: {}
  - id: meter_watts1
    channelTypeUID: zwave:meter_watts
    label: Electric meter (watts) 1
    description: Indicates the instantaneous power consumption
    configuration: {}
  - id: meter_watts2
    channelTypeUID: zwave:meter_watts
    label: Electric meter (watts) 2
    description: Indicates the instantaneous power consumption
    configuration: {}
  - id: sensor_power2
    channelTypeUID: zwave:sensor_power
    label: Sensor (power) 2
    description: Indicates the energy consumption (kWh)
    configuration: {}
  - id: sensor_power
    channelTypeUID: zwave:sensor_power
    label: Sensor (power)
    description: Indicates the energy consumption (kWh)
    configuration: {}
  - id: sensor_power3
    channelTypeUID: zwave:sensor_power
    label: Sensor (power) 3
    description: Indicates the energy consumption (kWh)
    configuration: {}
  - id: meter_watts3
    channelTypeUID: zwave:meter_watts
    label: Electric meter (watts) 3
    description: Indicates the instantaneous power consumption
    configuration: {}
  - id: sensor_power1
    channelTypeUID: zwave:sensor_power
    label: Sensor (power) 1
    description: Indicates the energy consumption (kWh)
    configuration: {}


UID: zwave:device:zw_controller:node4
label: Family Room Outlet
thingTypeUID: zwave:ge_45605_00_000
configuration:
  switchall_mode: 255
  nodename_location: ""
  config_3_1: 0
  config_4_1: 0
  nodename_name: FamilyRoomLamp
  node_id: 4
bridgeUID: zwave:serial_zstick:zw_controller
location: Family Room
channels:
  - id: config_decimal_param3
    channelTypeUID: zwave:ge_45605_00_000_config_decimal_param3
    label: Night Light
    description: Defines the behavior of the blue LED.
    configuration: {}
  - id: switch_binary
    channelTypeUID: zwave:switch_binary
    label: Switch
    description: Switch the power on and off
    configuration: {}

UID: zigbee:device:zg_coordinator:0022a3000040d061
label: Front Porch Light Switch
thingTypeUID: zigbee:device
configuration:
  zigbee_macaddress: 0022A3000040D061
bridgeUID: zigbee:coordinator_ember:zg_coordinator
location: Front Yard
channels:
  - id: 0022A3000040D061_1_switch
    channelTypeUID: zigbee:switch_onoff
    label: "ON_OFF_LIGHT: Switch"
    description: null
    configuration:
      zigbee_reporting_polling: 900
  - id: 0022A3000040D061_3_switch
    channelTypeUID: zigbee:switch_onoff
    label: "ON_OFF_LIGHT_SWITCH: Switch"
    description: null
    configuration:
      zigbee_reporting_polling: 2147483647

UID: zigbee:device:zg_coordinator:282c02bfffe8f86f
label: Rich's Button
thingTypeUID: zigbee:device
configuration:
  zigbee_macaddress: 282C02BFFFE8F86F
bridgeUID: zigbee:coordinator_ember:zg_coordinator
channels:
  - id: 282C02BFFFE8F86F_1_batterylevel
    channelTypeUID: system:battery-level
    label: Battery Level
    description: null
    configuration:
      zigbee_reporting_polling: 7200
  - id: 282C02BFFFE8F86F_1_dimmer
    channelTypeUID: zigbee:switch_level
    label: "REMOTE_CONTROL: Level Control"
    description: null
    configuration:
      zigbee_reporting_polling: 2147483647
  - id: 282C02BFFFE8F86F_1_batteryalarm
    channelTypeUID: zigbee:battery_alarm
    label: Battery Alarm
    description: null
    configuration:
      zigbee_reporting_polling: 7200
  - id: 282C02BFFFE8F86F_1_batteryvoltage
    channelTypeUID: zigbee:battery_voltage
    label: Battery Voltage
    description: null
    configuration:
      zigbee_reporting_polling: 7200

The last Thing shown above was added last week so it's not just the old Things I have hanging around.

I wonder what the difference is. Maybe you are doing a mixture of .things files and looking in MainUI?

I cannot find a single one of my Zwave or Zigbee Things that does not list all the Channels in the Code tab. In truth I cannot figure out how you can even have a Thing with Channels showing on the Channels tab without them showing on the Code tab. Both the Channels tab and the Code tab are rendered from the same JSON so the UI would have to go out of it's way to not show the Channels in the Code tab and I can't think of a good reason to do that.

I'm on 4.1 but I don't recall ever not seeing the Channels in the code tab for any Thing that has Channels. All of my things are auto discovered where that's possible.

Do you see Channels in the Channels tab? In the RAW JSON from the REST API?

Since the docs are always lacking the idea is to just define the plain thing and still have auto discovery / automatic channel creation. Once the device channels are properly discovered one can than edit the textual thing config to include the channel definition if desired (which then would be available from the code tab).
Basically discover what you use and then "freeze" that.

Wouldn't all that become moot when the YAML config files become a thing? Then the user can discover the Thing and, if desired, export the whole lot to a text file in one step, possibly even through an export function. Is there a case where the Thing has to be manually defined but the Channels are discovered? All my experience is that either the Thing is completely discovered or the Thing is completely manually defined.

from openhab-core.

Do you see Channels in the Channels tab? In the RAW JSON from the REST API?

Yes - the rest response for the thing correctly returns the channels and the gui shows them correctly in the channels tab

But we digress ...

Wouldn't all that become moot when the YAML config files become a thing?

I don't think so. If I have to add 10 rollershutters it's very quickly to do so through a textual config (e.g. set the IP, proper name, and label). But I am not sure about the channels so I want to use auto discovery in a first step and only in a second step define the channel configuration.
The thing type is typically very clear from the openHAB docs but the channel types typically aren't.
Also if we talk backwards compatibility:
All my shelly things are currently defined in *.things files however the channels have to be auto discovered.
Additionally it will only become moot if we finally have a separation between thing settings and device settings (see other issue) because until then it's not possible to change device settings if the thing is textually defined.

Btw. I just looked at my things again one flood alarm and one smoke alarm is still not working after over two weeks because it's stuck in CONFIG PENDING.

from openhab-core.

I don't think so. If I have to add 10 rollershutters it's very quickly to do so through a textual config (e.g. set the IP, proper name, and label). But I am not sure about the channels so I want to use auto discovery in a first step and only in a second step define the channel configuration.

That seems like even more work than discovering the Thing in the first place, exporting the YAML complete with Channels, and then copy/paste/editing from there. I don't see any benefit to starting with a file defined Thing and only then going to the UI to pull the Channels and then needing to modify your already created Thing definition with the Channels. It just seems like more work.

from openhab-core.

You assume that all devices are fully discoverable by auto discovery in the first place which is not always the case. Sometimes the thing discovery also does not work or only works after providing some form of data (e.g. I want to create 10 astro things which require the coordinates). Also the thing type discovery could be faulty so it's good to have an option to set the type.
Last but not least this would be downwards compatible with the current implementation in .things file so everyone could benefit from the new things

from openhab-core.

You assume that all devices are fully discoverable by auto discovery in the first place which is not always the case.

Are there cases where the Thing is manually created but the Channels are discovered? I know of places where the Things are wholly manually configured and other cases where the Things are wholly discoverable. I know of no cases where one has to manually create the Thing but the Channels are discovered.

Is there a binding that works like this? Or is it s theoretical concern?

I'm not talking about Bridges though, that's a different use case.

Sometimes the thing discovery also does not work or only works after providing some form of data (e.g. I want to create 10 astro things which require the coordinates).

I don't see any difference in this use case. Astro does autodiscover a Thing (two Things actually, one for sun and one for moon) based on the regional settings coordinates.

And this use case too seems to be more work to:

1. define the Thing in a .things file
2. navigate to the Thing in MainUI, open the code tab
3. copy the Channels
4. incorprate the Channels into the .things file
5. copy/paste/edit to create new Things

compared to

1. discover the Thing from the inbox
2. navigate to the Thing in MainUI, open the code tab
3. copy the full Thing definition and paste into a .things file
4. copy/paste/edit to create new Things

Even if for some reason the Thing cannot be discovered (assuming the theoretical case), the fact that you don't have to jump between the editor to MainUI and then back to the editor is an improvement in workflow. At least for cases where the initial configuration of the Thing is simple like with Astro.

Also the thing type discovery could be faulty so it's good to have an option to set the type.

Seems like that should be caught and fixed instead of hidden and/or worked around. Do we have lots of cases of this? I've seen some stuff around Shelly but haven't followed that closely.

from openhab-core.

Are there cases where the Thing is manually created but the Channels are discovered? I know of places where the Things are wholly manually configured and other cases where the Things are wholly discoverable. I know of no cases where one has to manually create the Thing but the Channels are discovered.

Many bindings do not support auto discovery because it's just not possible. And almost all the time when the user currently supplies the thing via text file the channels are automatically created or discovered (since from the user perspective this is the same I'm not going to differentiate between these two).

• Network Binding
• PV inverter binding
• Shelly Binding
  If the shellies are in a different subnet discovery will not work (which is also true for almost all other bindings which do ip based discovery)

As you can see the docs provide the thing definition but never the channel definition.

And this use case too seems to be more work to:

1. define the Thing in a .things file
2. navigate to the Thing in MainUI, open the code tab
3. copy the Channels
4. incorprate the Channels into the .things file
5. copy/paste/edit to create new Things

compared to

1. discover the Thing from the inbox
2. navigate to the Thing in MainUI, open the code tab
3. copy the full Thing definition and paste into a .things file
4. copy/paste/edit to create new Things

For me it's like this (since I want an astro thing for a different location).

1. Copy past from binding docs
2. Edit location and save
3. navigate to the Thing in MainUI, open the code tab
4. ...

compared to

1. Click + on things file
2. Select astro binding
3. Select "astro:sun"
4. Edit location
5. Click "Create Thing"
6. navigate to the Thing in MainUI, open the code tab
7. ...

However the discussion which one is faster is futile since obviously you are faster with the gui and I am faster with text files.
Since I am going to create the thing file anyway it's easier for me to start that way.
Even if I am/would be slower for me it's easier to work that way.
Channel discovery always comes after thing creation so there is no reason to prevent automatic channel discovery even when the thing was created through text.

Seems like that should be caught and fixed instead of hidden and/or worked around. Do we have lots of cases of this? I've seen some stuff around Shelly but haven't followed that closely.

The problem with these issues is that they are really hard to identify, harder to debug and even harder to reproduce. I consider myself an above average openHAB user and it took me very long to find the cause of my Shelly issues. And even with the help of the binding developer we were not able to pin the issue.
I know for a fact of two other openHAB users who face the same issues, one even stopped using the Shelly binding completely because it was so unstable and switched to MQTT (which the Shellies luckily support out of the box).
Additionally I've read a few forum posts that seem to be related to that issue so I would definitely say it's an issue but I can't quantify how big it is. I guess many users just restart openHAB once or twice which most of the time fixes the issue so they never bother to report it. Or in case of a smoke alarm or a flood alarm they just don't realize it because how often do you have a fire or a flood.

Since I've found more and more bindings which do the automatic channel generation (e.g. not yet released esphome binding, too) it's necessary to address these systematic issues because all bindings which do dynamic channel discovery are affected by this.

from openhab-core.

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/get-data-from-webserver-provided-by-esp8266/154112/17

from openhab-core.