Giter VIP home page Giter VIP logo

web-of-things-framework's Introduction

Web of Things Framework for NodeJS

This is an experimental implementation of the Web of Things Framework, including a combined HTTP and Web Sockets server, as well as a web page library for demoing access to "things" from within web page scripts. This work has been done in support of the W3C Web of Things Interest Group.

A hundred billion IoT devices are expected to be deployed over the next ten years. There are many IoT platforms, and an increasing number of IoT technologies. However, the IoT products are currently beset by silos and a lack of interoperability, which blocks the benefits of the network effect from taking hold. At W3C, we are exploring the potential for bridging IoT platforms and devices through the World Wide Web via a new class of Web servers that are connected through a common framework, and available in a variety of scales from microcontrollers, to smart phones and home/office hubs, and cloud-based server farms.

This framework involves virtual objects ("things") as proxies for physical and abstract entities. These things are modelled in terms of metadata, events, properties and actions, with bindings to scripting APIs and a variety of protocols, since no one protocol will fulfil all needs.

This project explores the use of JSON-LD as the basis for describing things. The thing description language (TDL) declares the metadata, events, properties and actions, as well as the relationship that this "thing" has to other things. Complementary informal descriptions allow for thingsonomies. A web server implementing the web of things framework provides an interface for instantiating local objects as a proxy for the thing they represent, starting from the URI for that thing's description. Servers also provide an interface to registering new "things" along with their description and implementation. Such things could interface to IoT sensors and actuators, or play the role of agents that implement a service in terms of other "things".

The scripting languages available will depend upon the server. This particular project supports JavaScript courtesy of NodeJS. To start with, only bindings to Web Sockets are provided. Over time, the aim is to expand this to other protocols, e.g. HTTP, CoAP, MQTT, XMPP and AMQP. In addition, the aim is to illustrate how to support a variety of IoT devices over Bluetooth, ZigBee and so forth. The challenges will include the security and communications metadata. Security metadata covers such things access control and the terms and conditions under which a data owner permits others to access this data. Communications metadata covers such as aspects as scheduling time slots for communicating with low power devices that spend most of the time sleeping, and for bulk data transfers, where the focus is on a continuous log of sensor data. The framework allows applications to indicate when a property has been updated but not yet applied as can happen when waiting for an IoT device to wake up and open its receive window.

Technical Recap.

The Web of Things starts with URIs for “thing” descriptions as Linked Data expressed in JSON-LD. Things are modelled in terms of their events, properties and actions. Scripts use this URI to ask the server to create a local proxy object in the script’s execution space for the designated thing. The server takes care of the protocol details, and can use which protocol best suits its needs, e.g. HTTP, WebSockets, CoAP, MQTT, XMPP or AMQP. The thing metadata and the target server metadata enable the server hosting the proxy to figure out which protocol to use for the messaging between the proxy and the thing it proxies.

You can have chains of proxies across different servers, which can range from microcontrollers to cloud-based server farms. Whilst REST based messages are used, the URI paths are really a matter for each server (just as in HTTP). Discovery involves a range of techniques: mDNS and UPnP on local networks, Bluetooth and BLE beacons, NFC, barcodes, cloud based device registries, social networks of people and things, and by following links between things from the dependencies stated in their descriptions. Servers may also expose a list of the things they host. Privacy will be based upon access control, and terms & conditions that enable data owners to control who can access their data and for what purpose.

Prerequisites

  1. Git
  2. node.js and npm
  3. websockets node module (ws)

Installation

The starting point is to install Git, see:

http://git-scm.com/book/en/v2/Getting-Started-Installing-Git

Then install node.js and npm from https://nodejs.org/download/

Next create a copy of this directory and change to it

  git clone https://github.com/w3c/web-of-things-framework
  cd web-of-things-framework

Install the node web socket module (and other dependencies):

  npm install

To start the server use the following command from a terminal shell:

  cd examples\coap_demo
  node demo.js

to run the CoAP demo

  cd examples\http_demo
  node demo.js

to run the HTTP REST demo

Both for the CoAP and HTTP demo point your web browser to load the demo web page at

http://localhost:8888/

and open the browser console to view the log.

Click on the Things menu item in the browser to view the loaded things.

Open other browser instance at http://localhost:8888/ and invoke actions and writable properties, the framework should send data to both browser sessions.

  cd examples\p2p_demo
  node demo.js

to run the P2P Kademlia (DHT) and peer to peer demo.

  1. http://www.w3.org/blog/2015/05/building-the-web-of-things
  2. http://www.w3.org/2015/05/wot-framework.pdf
  3. http://www.w3.org/WoT/IG/

Contributing

Gitter

We welcome contributions. If you find a bug in the source code or a mistake in the documentation, you can help us by submitting an issue to our GitHub repository, and likewise if you have suggestions for new features. Even better you can submit a Pull Request with a fix. We also have a Gitter chat channel, see above link.

Note: we are using js-beautify with its default settings for normalising the use of whitespace in JavaScript source code.

We encourage you to join the W3C Web of Things Community Group where contribution and discussions happen. Anyone can join and there are no fees.

The amount of time you contribute and the areas in which you contribute is up to you.

We need plenty of help with developing open source software for the Web of Things, including servers and test frameworks and test suites. I have started a NodeJS based server, and we also are looking for help with servers on microcontrollers, on smart phones/tablets, and for highly scalable cloud based systems.

There are opportunities to help with embedded systems, especially when it comes to embedded web of things servers, and device drivers. We will need a way to describe the communications patterns used by particular device, e.g. how to communicate with devices that spend most of their time asleep, and the way in which data is buffered or multiplexed (e.g. when aggregating data from multiple sensors).

There are also plenty of areas where you could contribute that don’t involve programming, for example, gathering use cases and requirements, the details of the bindings to protocols, and the architectural considerations. Security and privacy are areas where we need to better understand what’s practical. Real-time control e.g. of manufacturing robots is one of the potential application domains where the architectural considerations are especially challenging. When it comes to programming, you would have an opportunity to take on the role of an IoT application developer who writes the application scripts. We will be looking of people who can try things out and share the experiences with others as a way of building interest in the wider community.

Please see the other project documentation (*.md), e.g. protocol bindings and also the list of project issues where we are looking for help, including design topics and bugs.

Acknowledgements

This work has been funded in part (through October 2015) by the European Union's 7th Research Framework Programme (FP7/ 2013-2015) under grant agreement nº317862 - Compose.

Related Projects

These are not as far advanced as the NodeJS project, and your help is welcomed.

License

(The MIT License)

Copyright (c) 2015 Dave Raggett <[email protected]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

web-of-things-framework's People

Contributors

apexskier avatar barthanssens avatar bubbafat avatar draggett avatar gadgetnet avatar jollen avatar mlasak avatar mpetyx avatar zsoltpardi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

web-of-things-framework's Issues

Web of things servers for microcontrollers

This is to track proposals for implementing open source Web of Things servers for microcontrollers. I would expect these to be separate GitHub projects, but to share a common test framework.

The idea is to create resource constrained servers that can run on microcontrollers for use as IoT devices directly connected to sensors or actuators, or as gateways to IoT technologies such as Bluetooth or ZigBee. It would make sense to start from existing open source servers. Some questions to be addressed include

  • What open source projects to start from?
  • Which programming language to use, e.g. C, C++, Lua or Python?
  • Which scripting languages to support if any, e.g. JavaScript, Lua, Python, etc.?
  • Which protocols to support, e.g. CoAP or MQTT?
  • What security mechanisms it supports?
  • Which binary encodings to support, e.g. CBOR or EXI?
  • Whether it supports remote updates, e.g. for security fixes or capability enhancements?

A resource constrained IoT device may just support a fixed set of "things" without any dependencies on other "things", for example a temperature and humidity sensor such as the DHT11, or a switch that operates a light. This allows for a really stripped down Web of Things server, where the logic of the things it hosts is coded in the same language as the server, e.g. C.

A more powerful device may support a scripting language and allow scripts for things to be uploaded to it subject to appropriate security mechanisms. One possibility would be a server written in C + Lua. Such a device may also play a role in discovery, and allow things with dependencies on other things that are hosted on other devices.

If you are starting such a project please post a response to this issue telling us about your project and including a link to it.

Web of things servers for the cloud

This is to track proposals for implementing open source Web of Things servers for the cloud. I would expect these to be separate GitHub projects, but to share a common test framework.

The idea is to create highly scalable, lightweight and fast servers. It would make sense to start from existing open source servers. Some questions to be addressed include

  • What open source projects to start from?
  • Which programming language to use, e.g. C, C++, Java, Go, C#, etc.?
  • Which scripting languages to support, e.g. JavaScript, Lua, Python, .etc.?
  • Which protocols to support in addition to HTTP, e.g. Web Sockets, CoAP, MQTT, XMPP, AMQP?

If you are starting such a project please post a response to this issue telling us about your project and including a link to it.

Use editorconfig for consistent coding styles between editors

Checkout current coding styles in master branch are as the following.

  • indent_style is space
  • indent_size is 4
  • quote_type is set to auto while there are single and also double in the code
  • spaces_around_operators is set to true
  • space_after_control_statements is set to true
  • spaces_in_brackets is almost false in current codebase

Discussion: splitting some of this up into modules

First of all, there are lot's of great things happening here, but arguably it would be much more useful if we could talk about how to start splitting it up into more reusable npm modules.

As a starting point I've mocked up an example of the type of modules I would like to see, checkout Thing.js. It has a small scope: a simple class for constructing and working with Thing objects that contain metadata, events, properties, and actions.

It would be great to start seeing similarly scoped modules for things like secure device configuration, discovery, etc. There are many parts to an IoT framework, as such it might be helpful to start discussing how we can break this framework up into it's core components.

Thoughts?

Registering a proxy on one server for a thing on another server

This is very much work in progress, and needs further testing/debugging. I am marking this as a bug until the tests show that it is working. I reckon that it is pretty close to working, but I ran out of time today. The work involves changing the example of calling wot.proxy in demo.js to point to the second server.

Persistent Object State

Things objects could be initialized (registered) from a data store. (for example memory caching like memcached). The client can manage the IoT device states. For example, the previous states of backdoor light from a memcached is on. Thus, the WoT server could restart without reseting all of the device controlling states.

Use Case

  1. Backdoor light device is power on.
  2. The WoT server shows backdoor light is off
  3. My mom turns light on. The WoT server should show backdoor light is on
  4. The WoT server process is terminated abnormally.
  5. The server process is respawn (restarted).
  6. The WoT server initialized things from data store and server shows backdoor light is on.

ReferenceError: proxy is not defined

When trying to start node demo.js (after installing ws), an error is thrown referring to this line:

https://github.com/w3c/web-of-things-framework/blob/master/framework.js#L453
seems like the variable "proxy" is defined in a function scope

Node: v0.10.25 (latest on Ubuntu LTS)

Error Message:

started http server on port 8888
started web sockets server on port 8080

/.../web-of-things-framework/framework.js:453
exports.proxy = proxy;
                ^
ReferenceError: proxy is not defined
    at Object.<anonymous> (/.../web-of-things-framework/framework.js:453:17)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/.../web-of-things-framework/demo.js:1:73)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)

P2P Demo problem

Hello I was trying the P2P demo but it say:
error: uncaughtException: Cannot find module '../../../libs/message/wotmsg' date=Tue Nov 22 2016 13:54:14 GMT+0100 (CET), pid=36005, uid=501, gid=20, cwd=/Users/user/Desktop/web-of-things-framework/examples/p2p_demo, execPath=/usr/local/bin/node, version=v6.9.1, argv=[/usr/local/bin/node, /Users/user/Desktop/web-of-things-framework/examples/p2p_demo/demo.js], rss=44527616, heapTotal=23744512, heapUsed=13480936, loadavg=[1.625, 1.73095703125, 1.8681640625], uptime=494009, trace=[column=15, file=module.js, function=Function.Module._resolveFilename, line=469, method=Module._resolveFilename, native=false, column=25, file=module.js, function=Function.Module._load, line=417, method=Module._load, native=false, column=17, file=module.js, function=Module.require, line=497, method=require, native=false, column=19, file=internal/module.js, function=require, line=20, method=null, native=false, column=14, file=/Users/user/Desktop/web-of-things-framework/libs/transport/p2p/wotkad/lib/node.js, function=, line=40, method=null, native=false, column=32, file=module.js, function=Module._compile, line=570, method=_compile, native=false, column=10, file=module.js, function=Object.Module._extensions..js, line=579, method=Module._extensions..js, native=false, column=32, file=module.js, function=Module.load, line=487, method=load, native=false, column=12, file=module.js, function=tryModuleLoad, line=446, method=null, native=false, column=3, file=module.js, function=Function.Module._load, line=438, method=Module._load, native=false], stack=[Error: Cannot find module '../../../libs/message/wotmsg', at Function.Module._resolveFilename (module.js:469:15), at Function.Module._load (module.js:417:25), at Module.require (module.js:497:17), at require (internal/module.js:20:19), at Object.<anonymous> (/Users/user/Desktop/web-of-things-framework/libs/transport/p2p/wotkad/lib/node.js:40:14), at Module._compile (module.js:570:32), at Object.Module._extensions..js (module.js:579:10), at Module.load (module.js:487:32), at tryModuleLoad (module.js:446:12), at Function.Module._load (module.js:438:3)]

Wiki

Hello @draggett and architects,
can you please use Wiki pages here: https://github.com/w3c/web-of-things-framework/wiki to describe the project goals, roadmap and architecture. Currently we have only one document (presentation slides) and this is not enough for developers to understand this project goals and structure well.

Add a write stream logger with winston

  • There are log functions in existing logger.js implementation.
  • Propose to export a write streamer in logger.js module.
  • The write streamer can be used by other logger system. For example, Morgan can be used in combination with Winston's write stream.

Dropping @dependencies in favour of using @properties?

@Dependencies is essentially syntactic sugar for defining properties whose values are things. We could just drop @Dependencies and use @properties along with an associative array with metadata field names including "type" and "uri", e.g.

{
  @properties {
      door: { type: "thing", "uri": "door12" },
      light: { type: "thing", "uri": "switch12" }
   }
}

where "uri" gives the URI for the thing's description. This syntax is a little more verbose, but would win when it comes to providing additional metadata for the things this thing depends upon. What do you all think about this change?

Loop closures causing several types of bugs

There is an issue that appears in several parts of the node server where improper loop closures are causing bugs. For example if I define this thing model:

wot.thing("twoprops", {
    "@properties": {
        "property1": "string",
        "property2": "string"
    }
}, {
    start: function(thing) {
        console.log("setting property 1");
        thing.property1 = "property 1";

        console.log("setting property 2");
        thing.property2 = "property 2";
    },
    stop: function(thing) {},
});

When start is called you would expect framework.js (or wot.js) to send the patch message for both property1 and property2 - however it does not. It sends property2 twice. I created a mocha test to reproduce this - the output is:

started http server on port 8888
started web sockets server on port 8080
creating: twoprops
registering http://localhost:8888/wot/twoprops
starting1 twoprops
setting property 1
setting twoprops.property2 = property 1
setting property 2
setting twoprops.property2 = property 2

Notice:

setting twoprops.property2 = property 1

The bug is the classic loop closure issue (described in many places - but here is a good explanation)

The problem can be seen here in framework.js - the loop modifies the value of prop but also uses it in a closure. So in the end all the defined properties use the same property - the last in the loop.

A fix is to wrap this in an anonymous function such as this:

for (var prop in properties) {
    if (properties.hasOwnProperty(prop)) {
        thing._properties[prop] = null;

        (function(property) {
            Object.defineProperty(thing, property, {
                get: function () {
                    return thing._values[property];
                },

                set: function (value) {
                    thing._values[property] = value;
                    var message = {
                        uri: thing._uri,
                        patch: property,
                        data: value
                    };

                    ws.send(JSON.stringify(message));
                }
            });
        })(prop);
    }
}

This bug occurs in several parts of the code. I have submitted a pull request that fixes them but it was reverted - I believe because of whitespace issues (please let me know if for some other reason).

I will submit another pull request referencing this issue.

Bindings (MQTT, Firebase, etc) - question about scope.

Apologies if this belongs in another form - please redirect me if there is a more appropriate place.

I want to make sure I'm understanding the intent and scope of the bindings work.

At a high level, I think the goal is define the JSON protocol used to communicate between endpoints. How functions are called, events fired, etc.

Creating a reference implementation (i.e., WOT Node.js), and as many bindings as possible, is to help flush out the requirements, find gaps, prove ideas, etc.

Assuming that is a correct assumption, I'll propose a sample scenario and then ask some questions.

Let's say I work for IFTTT.com (I do not) - and I want to create a set of channels (an IFTTT concept) for IoT devices. I might start by creating an IoT channel for MQTT because there is a smart door that uses MQTT as it's protocol. I then create a channel for Firebase.io because there is a smart thermostat that uses Firebase.io as it's protocol.

Now a customer of IFTTT wants to create a rule that says "When the smart door is unlocked from the outside, set the inside temperature to 74 degrees."

To get going quickly, IFTTT forks the Node.js WOT server and enables the protocol bindings for MQTT and Firebase.io. They create a proxy to the smart door "thing" on MQTT and a proxy to the thermostat thing on Firebase.

When the smart door proxy "unlocked" event fires, they call the "setTemperature" function on the thermostat proxy.

This event and function operate using well-defined JSON (which is the same for both bindings).

This leaves me with some questions about the bindings being created for the WOT implementations.

Specifically about how generic the proxies are.

When I have a proxy that communicates over MQTT, do I need to be aware of that after creation? For example:

  • Do I need to understand that MQTT has a (potential) multi-phase ACK process (or does the proxy handle all that for me?)
  • Do I need to know about DUP messages or care about how RETAIN messages might be sent on connection?
  • Does the proxy handle the necessary authentication?
  • Is the Javascript API for a pub/sub store like MQTT or Firebase going to be the same as for something fundamentally different (e.g., a HTTP RESTful system based on disconnected polling rather than long-polling/web sockets)?

As an example, we might have something like:

Configuration file: devices.json

{
    "protocol": "mqtt",
    "connection": {
        "uri": "mosquito.example.org:1883",
        "user": "username",
        "password": "my password",
        "qosLevel": "1",
        "dup_action": "ignore",
        "retain_action": "pass-through"
    },
    "device": "door18729873"
}
{
    "protocol": "firebase",
    "connection": {
        "uri": "https://s-example-iot.firebaseio.com",
        "token": "JWTs authentication token",
        "heartbeat_timeout": "300",
    },
    "device": "thermostat2874981"
}

This defines a pair of bindings using two different protocols, each with unique settings dependent on the protocol. Then perhaps there would be something like (imaginary syntax - just communicating the idea, not an API):

// this finds the binding for each device and creates
// a protocol-specific proxy that transparently manages
// the authentication and communication needs for each device.
var thermostateProxy = factory.get("door18729873");
var doorProxy = factory.get("thermostat2874981");

doorProxy.onUnlocked((function(thing) {
    // Either Fahrenheit or very uncomfortable
    thermostatProxy.set(74);
}));

So my two questions:

  1. Am I understanding the scope of the bindings work correctly?
  2. My code samples show things like a device binding registry and a proxy factory - are these in-scope for the WOT Node server? If so (and it seems like they are), do we need to define that as a separate effort before creating more bindings?

[Refactor] Use URL router to decouple the API

Introduction

  • Reuse of service component. Use epxress.Router() to allow WoT server to completely decouple the server API (URL routing) from it's server logic (application).
  • Easy integration. One service component (API implementation) can be integrated (mounted) to various WoT server implementations.
  • Consider examples/http_demo/sumulator.js, the refactored programming style is as the following.
var simulator = function (thing, port) {
    logger.debug("starting HTTP device simulator for " + thing.name);

    // create express instance
    var app = express();

    // keep reference to thing model
    app.model = things.model;

    // keep reference to config
    app.config = config;

    // mount thing's URL router to express application
    app.use('/', require('service'));

    // create HTTP server and listen incoming request
    var server = http.createServer(app);

    server.listen(port, function () {
        logger.info('HTTP device simulator listener for thing ' + thing.name + ' started on port ' + port);
    });
}

exports.start = function start() {
    logger.debug('Start device simulator to communicate with WoT via the HTTP protocol');
    var door_device = new simulator(door, 8890);
    door.model.properties.battery_value();
    door.model.events.bell();
}

Develops can also mount things URL router to their existing Node.js application. For example,

var app = express();

// Create my 'ParseServer' instance.
var api = new ParseServer({
  ...
});
// Mount my 'ParseServer' URL router.
app.use('/parse', api);

// Mount my WoT thing's URL router under the relative path '/wot'
app.use('/wot', require('service'));

API implementation is stored in JavaScript module file service.js, can is exported via express.Router(). For example,

// Filename: service.js
var create = function(req, res, next) {
 ...
}

router.post('/', create);

module.exports = {
  router: router
};

Implementation

zsoltpardi's comment (#65) with regards to separate demos.

The original demo "http_demo" was kept. A new demo was created "http_express_demo". The "http_express_demo" demonstrate the Express based design and implementation. And it also demonstrates of how to use Express router to decouple the API implementation from the WoT server logic. The API router can then be reused by similar device WoT servers, said "http_express_demo2".

Test

A simple CLI test case using curl:

$ curl -X POST http://localhost:8890/switch12/property_get

Response:

{"thing":"switch12"}

URI Styling

zsoltpardi's comment (#65) with regards to URI style.

The current implementation put the thing name before action. If the URI is in the syntax GET /username/status, the 'username' is considered as an instance (resource name), and 'status' is its property. It shall be more REST style.

Security README images

This issue is just a work around to store images at Github for the security readme file. I don't want to store readme images in the source repository and trying to solve the problem of storing images using this add issue feature.

crypt_table

mipsyears_compare

JVM implementation

I saw the docs and the current implementations, is there any plan to have a JVM implementation of the framework?

I would suggest:

  • Scala
  • Finagle
  • Zipkin
  • Swagger
  • Cassandra
  • Spark

It's ok to do a PR as a sub project?

Web of Things server test framework?

We need common test framework to verify interoperability of servers by ensuring that they pass a common test suite.

The tests will need to cover at least:

  • The basic model of things in terms of metadata, events, properties and actions
  • Dependencies between things hosted on the same or different servers
  • Security and privacy related features, e.g. authentication and access control
  • Support for multiple protocols, e.g. HTTP, WebSockets, CoAP, MQTT, XMPP and AMQP
  • Support for different data formats and encodings

We will need a declarative format for tests, and a mechanism for adding, documenting and noting the status of tests, e.g. (un)reviewed, what bugs this test has, the status of code that implements the test, and so forth. Likewise, it would be very desirable to have a registry of test results across servers.

One suggestion is that it may be worth considering writing a test harness in NodeJS given the availability of node modules for many protocols. Please respond to this issue if you would like to contribute to work on the test framework in any way.

Interdependencies between "things"

Web of Things servers create objects for things based upon the models in their descriptions. The implementation's start method for thing is called when all of its dependencies on other things have been resolved. The resolution of dependencies happens asynchronously, see the init_dependencies() method in framework.js. This handles dependencies on locally hosted things and on proxies for remotely hosted things. A registry is used to share the same object when several things depending on the same proxy. This approach allows for circular dependencies where one thing depends directly or indirectly on another thing that in turn depends on the first thing. The code in a thing's start method may be executed before the start methods of the thing's it depends upon, and this is inevitable in the case of circular dependencies. For proxied things, this is handled by the target server queuing messages for a thing it hosts until that thing starts. For locally hosted things, this is handled by the virtual object for the thing's implementation.

Note that an error will occur if the server is unable to set up a proxy for a remote thing. This may occur when the target server hostname isn't resolvable to an IP address, if that IP address is unreachable, if the this server isn't authorised to access the target server, or if this server is unable to open a Web Socket connection with the target server (or likewise for whichever protocol is to be used for messaging). An open question is whether to set a platform dependent timeout for resolving dependencies, and how to expose such timeouts to scripts.

Further study is needed to flesh out the details for the lifecycle of things, e.g. when a thing is unregistered then its proxies and the things that depend on it need to be notified by raising the corresponding event.

Security, privacy and resilience for the Web of Things

Security, privacy and resilience are very important topics for the Web of Things. We need to build upon best practices for the underlying protocols and for end to end systems. Some of the considerations include:

  • Use of encryption to defend against pervasive monitoring
  • Software updates to fix security flaws
  • Mutual authentication and identity management
  • Credentials and trust management
  • Access control mechanisms and policies
  • Terms and conditions for data consumers
  • Resilience to software and hardware faults
  • Resilience to cyber attacks

We need your help to survey existing approaches, to identify gaps and potential solutions.

Swagger Interface

Hello,
I know it may be too early for such an idea, but I think it would be really helpful to have a Swagger interface for this framework.
It would made it possible from day one to develop clients in other languages, as well as make it more clear what it is the major development goal of this framework.
I would be more than glad to contribute on that and my ultimate goal is to actually reuse the Hydra Vocabulary.
Regards,
Michael.

ps:
The idea came to my mind because I followed the readme tutorial and everything was working fine, but then I was thinking.
What can I build on top of this, and most importantly, how..?

Bug: use server metadata to determine which protocols/uris/encodings to use with it

The current code assumes that the server supports WebSockets on the following URI:

 'ws://'  + hostname  + ':8080/webofthings'

This needs to be replaced by querying the server to determine which protocols, URIs and encodings it supports. This process starts with a URI for a thing on the server. That URI is then transformed to access a well known path:

 /.well-known/protocols

A GET on this should return a JSON object with the desired info. This still needs to be defined!

Note: URI schemes for thing descriptions are limited to protocols supporting GET, e.g. HTTP and CoAP.

How to use the project?

I just forked this project, but there is not even a hardware demo available. It is unclear how to use the project with a hardware.

The Readme.md states
"This work has been funded in part (through October 2015) by the European Union's 7th Research Framework Programme (FP7/ 2013-2015) under grant agreement nº317862 - Compose."

It's not my business how much you get for this from the public purse, but could you include at least a hardware demo please?

Add thing into http registry

Hello,
I think the line #33 of framework.js:

registry[uri] = { model: model, thing: this };

should be replaced by

registry[name] = { model: model, thing: this };

because "name" is the thing's id and http server search it into registry by id.

Regards
Simone Guerra

Work on the develop branch

I am opening this issue to discuss the latest changes at the develop branch.

tl;dr
There are some changes at the develop branch. To review the changes please get the latest from the develop branch, (you need to do the usual npm install to get the npm modules) open a browser session at http://localhost:8888 and click on the "Things" menu item, this will be client "A" session. Open an other browser session at http://localhost:8888 and click on the "Things" menu item, this will be client "B" session. The property values such as "is_open" or "battery_value" and "power_consumption" for the door and switch respectively should appear on both browser windows. Click on the Invoke button next to the "unlock" action and the value of "is_open" should change in both browser sessions as well as the Invoke button next the the "lock" action should close the door and change the "is_open" property.
There are the trace and debug logs in the browser development console, the Nodejs window and log files.

In more details:
Finally, I could find some times and changed a few things with the aim of having a bit better structured and more maintainable application.
I kept all original ideas and concept terms of models, events, property and action handling.
I assume there will be many iterations until we find the correct design and implementation, so this is mainly a brain storming exercise.

The changes are for the latest commit

  • There were an excellent community suggestion with regards to separating the UI from the back-in. I agree, such separation is really necessary and in first step I have created a "servers" directory which later can be easily moved into a separate repository. The http server related files and directories are moved from the root into the servers directory. The idea is (if I understood correctly) that there will be different servers and interfaces exposed to the clients, such as HTTP server (with web sockets), a REST server to provide mainly API based query/command interaction, or perhaps CoAP and MQTT. I am referring to clients who connect to their devices via the WoT framework, i.e. manage a door lock via the WoT framework. I imagine the majority of the client connections will be via HTTP server using web sockets, for instance the client open the door via the HTML5 application using the mobile enabled UI or via the REST application programming interface.
    There are sub directories in the "servers" directory, http, restapi, coap, etc. directories that refer to the implementation. I have moved the Express web server and related files into the "http" directory.
    The start and configuration of the servers is based on a config settings. Probably most implementations will be using a database to store the configuration settings. Currently we get it from the config.js file, the server settings is in the config.servers variable.
    The \servers\server_handler.js is responsible to manage the server sessions. The idea is to decouple the server management task from the main application file so we can write tests for this as well as probably it will be easier to maintain the application if not everything is embedded in the main application file (framework.js) like as it was before.
  • Added a "dbs" directory to store database related implementations. I believe most of clients will be using database to store configuration settings and model definition of the things. Under the dbs directory will be stored the different database implementations such as postgresql, mongodb, mssql, nedb, etc.
    Currently the application pulls the model settings (which was before hard coded in the demo.js file) from a local db file (under the dbs/file directory), but the find_thing and things_list function returns asynchronous results just like the databases will be, so this hopefully gives an idea how the database based resources will be populated.
    The changes are about separation of concerns. I think the objects that are responsible for the business logic should not include configuration and settings resources, but in a local repository file or in a database repository.
  • I tried to start breaking down the previous files that embed transport and application logic together into objects. I haven't got as far as implementing bubbafat's excellent suggestion about the factory design pattern, but I created a thing object as well as a thing handler. Previously everything was included in the framework.js and the wsd.js files, and I felt some decoupling is really necessary to have a cleaner design. The idea is that for example the transport layer such as the web socket handler should be responsible for the transport and the thing_handler object manages the things. So now the thing_handler under libs/thing/ is responsible for managing the things instead of the web socket file which was before performing basically all application logic.
    I believe we clearly need to change that as there will be other transport than web sockets and we should have some common objects which the transports can use to interact with the things.
    If this is the right direction then the next step could be to implement the factory design pattern.
    Please note the dependencies are commented out. I will wire them up if we think this is the way to go forward.
  • In order to have a decoupled design I believe we should utilize the event emitter of NodeJs. I like to use the event emitter, it is efficient, quick and allows a decoupled event driven design. Therefore I started to use it in this project as well. For example there are no asynchronous callbacks nor synchronous returns between the thing and the transport (web socket) but the thing is emitting an event upon property changes and any device events. The web socket transport object listen on the event (at the end of the servers\http\wsd.js file) and then submits it to the UI. Using the event driven design allows that all type of protocols, ws, REST, etc. listen on the very same events in a decoupled manner and then submit them to their connected clients.
  • Now there are database based settings and asynchronous calls to populate the UI. Previously the settings were hard coded in the wot.js file at the UI. To address high availability and scalability requirements I believe all real world, load balanced applications will be based on database settings or at least some server side configuration files, and therefore the committed design pulls the UI related resources such as list of things asynchronously from a DB file (currently it is a local file but I will wire up mongodb and postgresql schemas soon). The \servers\http\routes\api_routes.js pulls the model from a local DB file.
    I understand WoT is mainly a back-in server solution, but I think it is quite important to provide users with ideas how the WoT server can wired up to desktop and mobile UIs and how the WoT framework can be actually used - that's why I put some effort into the HTTP server design.
    Later it would be nice to roll out a proper MVC design using controllers and proper models for things, users, etc.
    Also, to have a different type of server we can add later node-restify which was another excellent suggestion from drasko.
  • There are many changes in the UI and these I believe are pretty standard changes. There is a new main js file at \servers\http\public\js\wot\wotapp.js which introduces name space, base viewmodel, dialog window handling based on bootstrap, etc. I started to integrate a flexible Model-View-ViewModel pattern javascript framework. I am mainly familiar with knockout.js so that's what I am using here as well, but based on the current design it will be straight forward to include Angular or other frameworks.
    This early stage UI design hopefully gives some idea about what can be done via the UI. For example device management using some edit templates. I assume the properties, events, actions eventually will be configured via the UI.
  • Please note, currently the remote proxy stuff is not implemented as I was not sure what is the best way to handle that. Probably it will be not very hard to add, most likely by extending the "thing" object (so the thing will be interacting with a remote WoT server instead of with a local device), but I am not sure at this stage.

Please let me know if these changes are sort of all right and then I will write tests and continue rolling out new files and device simulators. Alternatively, please let me know that in your opinion how to go forward with this WoT design.

events.js:72 throw er; // Unhandled 'error' event

Vanilla install. Cloned repo, installed dependencies and upon node demo.js receive the following error:

events.js:72
throw er; // Unhandled 'error' event
^
Error: listen EADDRINUSE
at errnoException (net.js:905:11)
at Server._listen2 (net.js:1043:14)
at listen (net.js:1065:10)
at net.js:1147:9
at dns.js:72:18
at process._tickCallback (node.js:448:13)
at Function.Module.runMain (module.js:499:11)
at startup (node.js:119:16)
at node.js:935:3

Testing for local versus remote URI?

When a creating a locally hosted thing, we need to resolve its dependencies on other things. We can use the registry to see if the dependee is ready registered, otherwise we note the dependency and if it is local, we wait for it to be registered, if it is remote we retrieve the dependee thing's model and then create the proxy and register it and resolve is pending dependents.

This requires a robust means to distinguish between URIs for things on this server, and URIs for things on other servers. Unfortunately, there are a multiplicity of ways that local URIs can be written. For instance, using the domain "localhost", or the IPv4 address 127.0.0.1, or the corresponding IPv6 address, or the external IPv4 or IPv6 address, and there could be multiple network interfaces on this device compounding the problem. So how do we robustly distinguish local vs remote URIs?

Exception when log file doesn't yet exist

I get the following expected error:

error: couldn't load http://localhost:9999/wot/switch12, error: Error: connect ECONNREFUSED

This is expected since there is no server running on port 9999. However, this is followed by the exception:

error: uncaughtException: ENOENT, open '/Users/dsr/Projects/webofthings/logs/application.log' date=Tue Aug 04 2015 10:32:11 GMT+0100 (BST), pid=80159, uid=501, gid=20, cwd=/Users/dsr/Projects/webofthings, execPath=/usr/local/bin/node, version=v0.12.6, argv=[node, /Users/dsr/Projects/webofthings/demo.js], rss=41631744, heapTotal=31598080, heapUsed=16440920, loadavg=[1.99853515625, 1.947265625, 1.9375], uptime=1188314, trace=[column=null, file=null, function=Error, line=null, method=null, native=true], stack=[Error: ENOENT, open '/Users/dsr/Projects/webofthings/logs/application.log', at Error (native)]

This could be caused by a failure of the logger to create the log file if it doesn't already exist

Consider using Typescript

Typescript is a super-set of javascript which allows you to use upcoming javascript features (es 6,7) now. It's compatible with existing javascript code and adds features like better type support (classes/interfaces). It also enables better intellisense, refactoring, compile time error checking, etc.

It is open-source, started by Microsoft, but even Google is using it to develop Angular2.

More info:
http://www.typescriptlang.org/

Do we need frontend?

I can see that PR #37 introduced frontend, making project into webapp with GUI. I thing that this repo should be strictly backend, and the code must be kept simple and tight.

I see it something like Meshblu: https://github.com/octoblu/meshblu, so backend only.

If frontend is needed, we can make a separate repo and think how we can do the integration.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.