hwhw / zigbee-lua Goto Github PK

View Code? Open in Web Editor NEW

29.0 10.0 4.0 199 KB

Zigbee coordinator and tools for LuaJIT

License: MIT License

Lua 99.27% Shell 0.73%

zigbee lua cc2530 cc2531 cc-znp zcl luajit mqtt etrx3

zigbee-lua's Introduction

Zigbee-lua

A Zigbee control framework written in Lua.

Requirements

LuaJIT (uses ffi, bitop)
libmicrohttpd
mosquitto

Supported devices

Status: It's operational. You can use this to run a Zigbee coordinator and control its communication. Via the TCP interface, you can control its behaviour and e.g. send adhoc custom packages. Best results are currently archieved using a CC2538 device. CC2530/31 are working, too - they have difficulties dealing with larger networks due to their extremely limited CPU and memory resources, though.

Controllers:

CC253x via ZNP. Status: CC253x: testing is done using the CC253x firmware from zigbee2mqtt project.
ETRX3 series (probably broken ATM). Status: interfacing using the AT command set, tested with firmware R309, possibly broken at the moment until latest set of changes are implemented.

Devices:

Features:

start up a Zigbee coordinator
maintain a simple device database
TCP interface for injecting code into the running instance
MQTT client that allows for publishing/subscribing (still rough around the edges, see mqtt-environment.lua for a very basic example)
HTTP server via libmicrohttpd
ZCL abstraction to build/parse data packages
ZLL touchlink factory reset (implemented on CC253x for now, still problems with sweeping over many channels - but working successfully for my Hue lightbulbs)

Description of software structure

Core (in lib/):
- ctx.lua is a general application context. It is supposed to be the single instance of its kind. It implements:
  - coroutine based tasks integrated with
  - messaging between these tasks
- util.lua is a collection of small utility functions, e.g. logging, table copies and more
- srv-epoll.lua is a epoll-based (and thus: Linux specific) wrapper around socket management. This would have to be reimplemented on other platforms. It also implements a main loop which is used from ctx.lua.
- serial.lua is a ljsyscall based wrapper for serial/UART communication
- codec.lua is a generic binary protocol codec, configured by simple Lua data structures.
Interfaces (in interfaces/):
- These will get loaded depending on configuration (in config.lua, or whatever you include in its place)
- Noteworthy interface is the "zigbee" interface. There is a try to separate the protocol stuff (interfaces/zigbee.lua, interfaces/zigbee/zcl.lua) from device-/cluster-specific code
- The CC253x ZNP interface is in interfaces/zigbee/devices/dongle-cc253x.lua, the ZNP protocol definition (using codec.lua) can be found in interfaces/zigbee/cc-znp.lua
- The ETRX3 interface is in interfaces/zigbee/devices/dongle-etrx3.lua

Usage

The following description assumes that you want to use this software to build a home automation infrastructure with Zigbee devices - which is what drove the development of this software in the first place. But note that you can probably use this software for other tasks, too.

Prepare software

check out the git submodules. You do not need luarocks or similar lua package management. Just do:
```
$ git submodule init
$ git submodule update
```

Prepare hardware

CC253x: flash with appropriate firmware. See firmware remarks for a very subjective view on what you should use.
ETRX3 based hardware: should already have firmware

optional: configure USB serial interface permissions This might be your chance to look and see what udev is and can do for you
edit/adapt config.lua or integrate it into your own "environment" definition
Run an "environment" that will itself call out to lib/ctx.lua

Then:

peruse scripts in contrib/ to show list of known devices, permit network joins, name devices, ...
create an "environment" by writing event handling functions that trigger actions. Have a look at the example environments in the project's root directory.

Similar projects:

When I started this whole smarthome project of mine, I had a loooong look at https://github.com/Koenkk/zigbee2mqtt - and I used it for a short amount of time. It is good for what it does. I have to admit though, that I have a certain dislike for Javascript in general and the nodejs ecosystem in particular. Zigbee2mqtt pulled LOTS of other packages and there is no way to get me motivated enough to go and see what they all do. Yes, there is some amount of "not invented here" syndrome, too. Zigbee2mqtt mostly stands on the shoulders of https://github.com/zigbeer/zigbee-shepherd.

I was not really content with the behaviour of these applications. A lot of development seems to be on the "hacky" side, which is generally fine. However, these levels of hacks are accumulating...

Also, I found some really nicely written software:

https://github.com/Frans-Willem/AqaraHub/ is really nicely abstracted, very concise modern C++. ZNP and ZCL are cleanly abstracted. Its abstractions use futures, which makes for easily read state machines - a value in its own accord for a state machine heavy task. This software, like Zigbee2mqtt, focuses on providing an MQTT interface for controlling the Zigbee devices.

https://github.com/Tropicao/zigbridge/ is also really nicely abstracted. It is written in a very nicely styled C, from the enlightenment.org school of programming. It does NOT provide an MQTT interface and has some rudimentary implementation of other interfaces.

Making up my mind, I came to the point where I wanted to have my own implementation. Central design points were:

not focused on MQTT only (or even at all?)
must allow to script the "home automation" logic
must allow for access to as many Zigbee (ZDP/ZCL) features as possible
must not try to be too clever

While Languages like C++ and C have their appeal (and I spent a few days thinking about continuing development by reusing corresponding projects), I am quite proficient with Lua and Lua/C interop using (LuaJIT) FFI. So I settled on this. Result is a very compact (in terms of lines of code) core that allows quite some flexibility.

Roadmap:

Something to store/analyze sensor data (temperature & so on)
Better abstracted Zigbee device classes (rather than the "any.lua" which does a bit of everything)
Better examples
Tests

zigbee-lua's People

Contributors

Stargazers

Watchers

Forkers

wenij chiroip xuchuanxin abwebc

zigbee-lua's Issues

device_database.json

Where is this file located?

Please unbundle dependencies

Hi, It is possible to unbundle all dependencies from this repo?

I mean all external libraries, including those ffi_* than have separate repos. All libraries should go into their place in system. This will ease package maintenance.

Also, json-lua could be replaced with cjson, which is available almost in every distro, even in RHEL ;).

I've added some of them in my gentoo overlay (https://github.com/Perlovka/portage-overlay/tree/master/dev-lua), but still using a "hack" to unbundle them from code (https://github.com/Perlovka/portage-overlay/blob/30641bcd9310ec0134f3a16f1d268ef99b29ca15/dev-lua/zigbee-lua/zigbee-lua-0.1_pre20190529.ebuild#L47)

Thanks.

interface:209

traceback:

./interfaces/zigbee.lua:209: attempt to call method 'set' (a nil value), stack traceback:
./interfaces/zigbee.lua: in function <./interfaces/zigbee.lua:206>
./luajit: ./lib/srv-epoll.lua:132: ./lib/ctx.lua:212: ./interfaces/zigbee.lua:209: attempt to call method 'set' (a nil value)
stack traceback:
[C]: in function 'char_reader'
./lib/srv-epoll.lua:132: in function 'on_readable'
./lib/srv-epoll.lua:82: in function 'run'
m.lua:6: in main chunk
[C]: at 0x0000c370

code & loc

      if d then
        self.devices:set(ieeeaddr, d) -- line 209
        self.devices:save()
      end

maybe

"self.device:set(...)",
self or devices is null, or devices is not an valid object.
or the device has some error.

env

arch: arm71 on i.MX6UL
luajit: 2.05
mqtt: 1.6.2

btw. thanks for good work.

device database

Hello!
Let me first thank you for sharing this exciting project!
I'm trying to run zigbee-lua on a PI/raspbian now but in the long term I hope this can be something I could run on a openwrt router as there is no hope I could get zibee2mqtt running on a MIPS router even with 128MB RAM. I just don't want another PI running 24/7 just as a zigbee<->mqtt bridge.

I'm completely new to lua therefore please excuse my dumb questions...

First of all I'm not sure what is the best way to get the dependent lua libraries. So far I've tried luarocks and installed:
luarocks install inspect
luarocks install ljsyscall
luarocks install luabitop
luarocks install json-lua
There was a problem with the json libary - had to move it to a place expected by luajit, but expect this is just a little problem with raspbian and luarocks.
Please let me know if there is a better way to set up lua libraries.

Then, I'm not sure if the zigbee-lua server is supposed to create its device database or should I do something manually. On start I get:
Zigbee: cannot open database file device_database.json, error: device_database.json: No such file or directory

It would seem I've managed to pair a Xiaomi Aquara WXKG11LM switch because I do get plenty of debugging output each time I press the button. The list-devs.sh script does not return any devices though which I guess is a result of the missing device_database.json file.
The same switch was paired on zigbee2mqtt on the same CC2531 previously... not sure if CC2531 just 'remembers' it or the pairing actually worked (sorry, lost the logs with all the excitement).

Also, sometimes it crashes:
`[2019-01-29 15:16:46] srv: got event on fd 4
[2019-01-29 15:16:46] dongle-cc2530//dev/ttyACM0: SOF found
[2019-01-29 15:16:46] dongle-cc2530//dev/ttyACM0|mt: got MT command: type AREQ (2), subsystem ZDO (0x05), command id 0x84, data:
45 84 9D D6 00 9D D6 10 01 04 01 03 01 01 03 00 E...............
00 12 00 03 00 01 00 00 ........

[2019-01-29 15:16:46] dongle-cc2530//dev/ttyACM0|mt: MT command AREQ_ZDO_SIMPLE_DESC_RSP, payload:
{
DeviceId = 259,
DeviceVersion = 1,
Endpoint = 1,
InClusterList = { 0, 18, 3 },
Len = 16,
NwkAddr = 54941,
OutClusterList = { 0 },
ProfileId = 260,
SrcAddr = 54941,
Status = 0
}
[2019-01-29 15:16:46] srv: removed FD 5 from epoll group
[2019-01-29 15:16:46] ctx|task: task zigbee_provisioning resuming
[2019-01-29 15:16:46] ctx|task: task zigbee_provisioning finished
[2019-01-29 15:16:46] ctx|task: task zigbee_provisioning aborted with error: /usr/local/share/lua/5.1/lib/json-lua/json.lua:706: JSON:encode must be called in method format, stack traceback:
[C]: in function 'assert'
/usr/local/share/lua/5.1/lib/json-lua/json.lua:706: in function 'onEncodeError'
/usr/local/share/lua/5.1/lib/json-lua/json.lua:1396: in function 'encode'
./interfaces/zigbee.lua:32: in function 'save'
./interfaces/zigbee.lua:109: in function <./interfaces/zigbee.lua:105>
luajit: ./lib/srv-epoll.lua:114: ./lib/ctx.lua:212: /usr/local/share/lua/5.1/lib/json-lua/json.lua:706: JSON:encode must be called in method format
stack traceback:
[C]: in function 'char_reader'
./lib/srv-epoll.lua:114: in function 'on_readable'
./lib/srv-epoll.lua:68: in function 'run'
./main.lua:4: in main chunk
[C]: at 0x00013bf8
`
Could it be I've installed wrong json-lua library?

Appreciate your help!

mqtt_client: random segfaults

Hi,

I've tried to create testing env, but app mostly segfaults at random point.
I have two Aqara LED bulbs, the goal was to control them via mqtt.

Sometimes it start sucessfully, and works as expected, e.g. it reports status from bulbs

mosquitto_sub -t /zigbee-lua/event/Lamp
{"state":1,"bright":255,"temp":370}

and I could get devices via

mosquitto_pub -t /zigbee-lua/devices/get -n

and could control color temp with

mosquitto_pub -t /zigbee-lua/devices/Lamp2/set -m '{ "ctemp": 270}'

Here is an environment file:

local ctx = require"lib.ctx"
local U = require"lib.util"
local D = require"interfaces.zigbee.any"
local json = require"lib.json-lua.json"
local M = ctx.interfaces.mqtt_client[1]

local E = U.object:new()

local devices = ctx.interfaces.zigbee[1].devices:names()
for _,dev in ipairs(devices) do
    E[dev] = D:new{id=dev}
---[[
    E[dev]:on_aqara_report(function (d)
        if d.ReportAttributes then
            local state = { lq = d.linkquality }
            for _, r in ipairs(d.ReportAttributes) do
                if r.AttributeIdentifier == 100 then
                    state.state = r.Attribute.Value
                end
                if r.AttributeIdentifier == 101 then
                    state.bright = r.Attribute.Value
                end
                if r.AttributeIdentifier == 102 then
                    state.temp = r.Attribute.Value
                end
            end
            local ok, msg_json = pcall(json.encode, state)
            if ok then M:publish(string.format("/zigbee-lua/event/%s", dev), msg_json) end
        end
    end)

    M:subscribe(string.format("/zigbee-lua/devices/%s/set", dev))

    ctx.task{function()
        for ok, msg in ctx:wait_all({"mqtt_client", "message"}, function(msg) return msg.topic == string.format("/zigbee-lua/devices/%s/set",dev) end) do
            local ok, t = pcall(json.decode, msg.payload)
            if ok then
                for k,v in pairs(t) do
                    if k == 'ctemp' then E[dev]:ctemp(v)
                    U.DEBUG("env", "payload: %d", v)
                    end
                end
            end
            M:publish(string.format("/zigbee-lua/event/%s", dev), string.format("%s",t.ctemp))
        end
    end}
--]]
end

-- MQTT subscribing: We use this to open the network for joining
M:subscribe("/zigbee-lua/permit_join")
ctx.task{name="mqtt_permit_join",function()
  for ok, msg in ctx:wait_all({"mqtt_client", "message"}, function(msg) return msg.topic == "/zigbee-lua/permit_join" end) do
    U.DEBUG("mqtt_environment", "got permit_join message via MQTT, opening network for devices")
    -- TODO: timeout handling etc
    ctx:fire({"Zigbee","permit_join"},{include={0xfffc}})
  end
end}

---[[
M:subscribe("/zigbee-lua/devices/get")
ctx.task{name="mqtt_list_devices",function()
  for ok, msg in ctx:wait_all({"mqtt_client", "message"}, function(msg) return msg.topic == "/zigbee-lua/devices/get" end) do
    U.DEBUG("mqtt_environment", "got list message via MQTT")

    local ok, msg_json = pcall(json.encode, {type="list", event="devices", data = ctx.interfaces.zigbee[1].devices})
    if ok then M:publish("/zigbee-lua/devices", msg_json) end
  end
end}
--]]
return E

Running as a service

Hi,

After refactoring, my zigbee-lua setup is broken ;) . I've used main.lua code as service binary (https://github.com/Perlovka/portage-overlay/blob/master/dev-lua/zigbee-lua/files/zigbee-lua), which has been launched by init script.
And environment file has been set in config.lua:

return {
  srv_implementation = "lib.srv-epoll",
  systemd = false,
  environment = "test-env",
  interfaces = {

That was very handy, but now I'm required to run an environment file itself, and this is not so flexible as before.

Could we revert to the old behavior? 😉

Or even better:

Create some lua script that will act as system service binary.
It should read config file and run appropriate environment.
It should search config file either in place, provided by command line (like zigbee-lua -c /etc/zigbee-lua/config.json) or in current directory.
Config file should not be a lua code (because otherwise you should add it's path to lua package.path). Easiest option - json format, it can be easy converted into lua table.

How do you think about it?