Giter VIP home page Giter VIP logo

metacom's Introduction

Metacom Communication Protocol for Metarhia

ci status snyk npm version npm downloads/month npm downloads

Metacom protocol specification: https://github.com/metarhia/Contracts/blob/master/doc/Metacom.md

import { Metacom } from 'metacom';
// const { Metacom } = require('metacom'); // for backend

const metacom = Metacom.create('ws://domainname.com:8000');
const { api } = metacom;
try {
  await metacom.load('auth'); // Load `auth` interface
  await api.auth.status(); // Check session status
} catch (err) {
  await api.auth.signIn({ login: 'marcus', password: 'marcus' });
}
await metacom.load('example'); // Load `example` interface
const result = api.example.methodName({ arg1, arg2 });

Streams over Websocket

Example: big file upload

Create uploadFile function on the client:

const metacom = Metacom.create('ws://example.com/api');

const uploadFile = async (file) => {
  // createBlobUploader creates streamId and inits file reader for convenience
  const uploader = metacom.createBlobUploader(file);
  // Prepare backend file consumer
  await metacom.api.files.upload({
    streamId: uploader.streamId,
    name: file.name,
  });
  // Start uploading stream and wait for its end
  await uploader.upload();
  return { uploadedFile: file };
};

Create API method to init file destination:

// api/files/upload.js
async ({ streamId, name }) => {
  const filePath = `./application/resources/${name}`;
  // Get incoming stream by streamId sent from client
  const readable = context.client.getStream(streamId);
  // Create nodejs stream to write file on server
  const writable = node.fs.createWriteStream(filePath);
  // Pipe metacom readable to nodejs writable
  readable.pipe(writable);
  return { result: 'Stream initialized' };
};

Example: big file download

Create downloadFile function on the client:

const metacom = Metacom.create('ws://example.com/api');

const downloadFile = async (name) => {
  // Init backend file producer to get streamId
  const { streamId } = await metacom.api.files.download({ name });
  // Get metacom readable stream
  const readable = await metacom.getStream(streamId);
  // Convert stream to blob to make a file on the client
  const blob = await readable.toBlob();
  return new File([blob], name);
};

Create API method to init file source:

// api/files/download.js
async ({ name }) => {
  const filePath = `./application/resources/${name}`;
  // Create nodejs readable stream to read a file
  const readable = node.fs.createReadStream(filePath);
  // Get file size
  const { size } = await node.fsp.stat(filePath);
  // Create metacom writable stream
  const writable = context.client.createStream(name, size);
  // Pipe nodejs readable to metacom writable
  readable.pipe(writable);
  return { streamId: writable.streamId };
};

License & Contributors

Copyright (c) 2018-2024 Metarhia contributors. Metacom is MIT licensed.
Metacom is a part of Metarhia technology stack.

metacom's People

Contributors

aliendrew avatar aliusdiemorietur avatar baristanko avatar dependabot[bot] avatar georgolden avatar juliagerasymenko avatar klarpen avatar lundibundi avatar marhiievhe avatar mille-nium avatar mprudnik avatar nechaido avatar o-rumiantsev avatar pgnedoy avatar pom4h avatar rohiievych avatar slavakaderkin avatar timursevimli avatar tshemsedinov avatar vladyslavry141 avatar zaitsev-oleksii avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

metacom's Issues

Tests and CI

Unittests and integration tests, for both client-side and server-side metacom implementation running together, establishing communication and executing example scenario.

Protocol parser

Network transmission structures:

Handshake

- version: 2b
- status: 1b
  - 0 = new connection (from client)
  - 1 = restore connection (from client)
  - 2 = accept connection (from server)
  - 3 = reject connection (from server)
- reserved: 1b
- token: 32b (optional), token contains id (bit prefix)

Parcel

- structType: 1b
  - parcel = 0
- parcelId: 4b
  - client++, server--
- parcelType: 1b
  - ping = 0, pong = 1, call = 2, callback = 3,
  - event = 4, stream = 5
- compression: 1b
  - no = 0, gzip = 1
- encoding: 1b
  - binary = 0, jstp = 1, json = 2, bson = 3, v8 = 4
- length: 8b

Chunk

- structType: 1b
  - chunk = 1
- parcelId: 4b
- chunkId: 4b
  - client++, server--
- flags: 1b
  - more = 1
  - stop = 2
  - pause = 4
  - resume = 8
- length: 2b
- payload: length

Client/Server dialog example

C: 0001 00 (handshake, new session request)
S: 0001 00 D16A...2A75 (handshake, token)
C: 00 00000000 00 00 00 (ping parcel)
S: 00 00000000 01 00 00 (pong parcel)
C: 00 00000001 02 00 02 0000000000000035 (call parcel)
C: 01 00000001 00000000 01 001C ["applicationName","methodNa (chunk)
C: 01 00000001 00000001 00 0019 me","par1","par2","par3"] (chunk)
S: 00 00000001 03 00 02 000000000000000A (callback parcel)
S: 01 00000001 00000002 00 00 0A ["result"] (chunk)

Parsed from chunks
["applicationName","methodName","par1","par2","par3"]

RPC call and callback contract

Call object:

{
  key:  string, 8 byte key generated with `common.generateKey()` from ALPHA_DIGIT character set,  
  appName: string, name of the application,
  appVersion: string, version of the application,
  apiName: string, name of the interface,
  method: string, name of the method,
  args: array, arguments to pass to the method
}

Callback object:

{
  key: string, key from appropriate `Call` object,
  error: string | null, error message,
  result: remote procedure result
}

Implement Metacom events

Browser class Metacom need following events (will be inherited from EventEmitter):

  • metacom.on('open')
  • metacom.on('close')
  • metacom.on('connected')
  • metacom.on('disconnected')
  • metacom.on('reconnect')

Bug: Client instance `connection` property is null

Description

13:49:14  W3   error   TypeError: Cannot read property 'send' of null
  Client.emit (/node_modules/metacom/lib/channel.js:38:21)
  Timeout._onTimeout (/api/example.1/subscribe.js:4:20)
  listOnTimeout (internal/timers.js:551:17)
  processTimers (internal/timers.js:494:7)

To Reproduce

  • Install metarhia/Example with metacom v1.0.0-alpha.0
  • Run browser application and close it
  • Wait 2-3 minutes

Websocket contract

Websocket contract is different from Net one. So we need to wrap websocket server and client to unify websocket and net interfaces

Pass errors if no http status code

Error messages should be serialized and passed to client-side (without stack) if no http status code provided to Channel.prototype.error(code, err, callId)

Refs: #67

Implement RPC queue

Why we need queue?
In disconnected state RPC calls waiting for timeout and fails. But if metacom state changes within timeout we can resend request. It would be easy to automate with queue (we will use queue just in disconnected mode).

Introspection: listMethosd

Method listMethods will be available after selectApplication and returns API definition with method names, parameters and description.

Remove channels from collection

Memory leak

We have channels collections in lib/channel.js but we need to remove keys from collections when http response ends or websocket closes.

  • Support keep-alive timeout for sessions
  • Support http keep-alive

To Reproduce

  • metacom version 1.2.0
  • impress version 2.0.4

Internal methods: signIn, signOut

Methods:

  • signIn(name, password) where name is encrypted by token and password is hashed by token
  • signOut() will leave session id (connection token) but disassociate auth info with current token

Token format

Token is a 32 bytes (256 bit) token. We use it in protocol #3 to restore connection #15. But it should contain device id (server prefix + client prefix). Need to specify it here in details.

Library classes structure

  • Connection - abstract connection
  • Server extends Connection
  • Client extends Connection - client-side implementation
  • ClientConnection extends Connection - server-side implementation
  • Transport - abstract transport
  • TcpTransport extends Transport - tsp socket implementation
  • WsTransport extends Transport - websocket implementation

Implement ping

  • Send ping packet {} on periodically interval (default: 60sec)
  • Ping if connection is active
  • Ping only after timeout of inactivity

Remove sessions from collection

Memory leak

We have sessions collections in lib/channel.js but we have no me place in metacom where we will remove sessions from collection (with timeout).

Also we need to rename LONG_RESPONSE to SESSION_TIMEOUT and maybe move it and it's usage to channel.js.

To Reproduce

  • metacom version 1.2.0
  • impress version 2.0.4

Create HTTP transport for browser implementation

Now we removed obsolete http support in Metacom class (browser implementation) and but we need new implementation with introspection, scaffolding and combine both transports with strategy pattern in Metacom class.

  • Refactor current abstractions to be strategy
  • Implement http strategy

Handshake procedure

Steps:

  1. At first connect client will send status = 0 and token generated from device identifier, hardware or browser fingerprint.
  2. Server may accept with status = 2 and pass token generated from server bit prefix, client connection bit prefix and cryptographic functions. Client will save token to local storage and pass to server at the next handshake.
  3. But server may reject with status = 3 and pass token of rejection, it contains server bit prefix, rejection reason id and short descriptive message.
  4. When client restores connection or connects again after offline or period of inactivity it will pass status = 1 and stored token to restore session.

See #3 for handshake structures

Need unsubscribe for methods

Need unsubscribe method from ws events

Example:
metacom.api[group].subscribe();
metacom.api[group].unsubscribe();

Fix double open websocket

Run metarhia/Example with latest (unpublished) metacom code on server and client. See Network/WS in DevTools: double websocket connection starts in 90% cases.

Use Proxy to intercept and pass all method calls

Proposed client syntax:

const client = metacom.connect('tcp://domain.name:2000');
const app = client.application('applicationName');
app.methodName(par1, par2, par3, (err, data) => {
  console.dir({ err, data });
});

Move some rpc machinery to impress

Method Channel.prototype.rpc contains both protocol logic and method execution logic, so we can decompose it and move method execution logic to impress.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.