First Rémi, let me say that I congratulate you for your extremely hard work towards this really useful library. I found telemetry on my personal intent to re-invent the wheel ONCE more. So thank, you and if you help me I will work towards improving and porting your library.

I ported telemetry (plain C) to the K22FN512 (a cortex M4 cpu) platform on the K22F freedom board. I took me a week since I was not very sure how is the actual output string were and how the output should look on the debug serial port I use.

So my first questions are:

(1) Which is the minimal code to port ? I asked this because I have to rewrote the following routines:

void newentry_u8(FILE * file, const char * topic, uint8_t data)
{
fputs("u8", file);
fputs("\t;\t", file);
fputs(topic, file);
fputs("\t;\t", file);
fprintf(file, "%hhu", data);
fputs("\t;\t", file);
publish_u8(topic,data);
fputs("\n", file);
}

into this:

void newentry_u8(FILE * file, const char * topic, uint8_t data)
{
PRINTF("u8");
PRINTF("\t;\t");
PRINTF(topic);
PRINTF("\t;\t");
PRINTF("%hhu", data);
PRINTF("\t;\t");
publish_u8(topic,data);
PRINTF("\n");
}

...since my platform does not support fputs.

Is there a more elegant way to do this in Telemetry?

Another issue is the following setup in main:
//
// Telemetry setup
//
transport.write = tlm_write;
transport.read = tlm_read;

then I also rewrote:

int32_t tlm_write(void * buf, uint32_t sizeToWrite)
{
uint8_t * caster = (uint8_t*)(buf);
uint32_t i;
for(i = 0 ; i < sizeToWrite ; i++)
{
// fprintf(output, "%02x", caster[i]);
PRINTF("%02x", caster[i]);
}
}

(2) In the examples you open a file and use fprintf and that is why you use output. Is this necessary ?
In the example:

output = fopen("valid_vectors.csv", "w");
if (output == NULL) {
    printf("Failure : Could not open file for write.");
    return -1;
}

(3) Can we mix messages ? What I mean with this is that I was planing to use serial as I always do with "general" debug information and plus Telemetry. is this possible, pyTelemetry wont get confused ?

For example this is a direct example of what I'm having now:

Actual dip-switch status is 15 

f70100746f706963008a56c77f

Initiating TMP175 temperature read.

Ch.0 (0x 38E), Ch.1 (0x 455) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f706963008b56477f

Initiating TMP175 temperature read.

Ch.0 (0x 3D5), Ch.1 (0x 4C9) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f706963008c357df77f

Initiating TMP175 temperature read.

Ch.0 (0x 228), Ch.1 (0x 3BD) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f706963008d35777f

Initiating TMP175 temperature read.

Ch.0 (0x 251), Ch.1 (0x 4D0) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f706963008e14e77f

Initiating TMP175 temperature read.

Ch.0 (0x 38F), Ch.1 (0x 455) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f706963008f14677f

Initiating TMP175 temperature read.

Ch.0 (0x 24F), Ch.1 (0x 4D3) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f7069630090fb167f

Initiating TMP175 temperature read.

Ch.0 (0x 3BE), Ch.1 (0x 55E) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f7069630091fb967f

Initiating TMP175 temperature read.

Ch.0 (0x 28B), Ch.1 (0x 3D4) 

Temp (0x4A59) 

 Actual dip-switch status is 15 

f70100746f7069630092da067f

Initiating TMP175 temperature read.

As you can see Telemetry output is inserted in the middle. Will this work as intended in pyTelemetry ?

When I want to publish a value, the function "end" is called. All time this function is used, I can see the program into Hard_Fault_handler.

Simplification of the build process.

Distributions will no longer assembled by gathering files from different folders, which is too complex and obscure for the unfamiliar user.

Instead, distributions will be pre-constructed (each distribution folder will contain a .h/.hpp for interface and .c/.cpp for distribution)

Then, only C core code will be copied into their respective c_api subfolder
A JSON file could be used to list all distributions and make adding a new one easy.

I'm using Telemetry for a project on an ESP32 running FreeRTOS (and it's working great so far, for variables), however when I call attach("testtopic",&mycallback); , mycallback seemingly never gets called. I have no debugger available, but i put a global variable change and a print to console right at the start of the mycallback function (whose signature is void (TM_msg*) ); those don't ever come out. A test of a counter I'm running in parallel on another topic comes through no problem.

Am I doing something wrong? attach(t,*funptr) should call the funptr function when something with topic t gets received, right?

Scaffold a base config file where will be defined hardware specific routines (driver callbacks) and data (override default platform little endian, etc)

struct TM_transport {
  int32_t (*read)(uint8_t * buf, uint32_t sizeToRead);
  int32_t (*readable)();
  int32_t (*write)(uint8_t * buf, uint32_t sizeToWrite);
  int32_t (*writeable)();
};

Return types for each should match the potential 'readable' or 'writeable'. For instance, if 'sizeToRead' is an uint32_t, then the return should also be a uint32_t. This issue is present in the examples, source code, and wiki.

Build and Test C and C++ api with Appveyor

Some sort of json equivalent for embedded

Strategy :
Move structure definitions inside telemetry.h.
Move structures declarations inside telemetry.c

Include telemetry.h inside Telemetry.cpp, not Telemetry.hpp. This way, importing Telemetry.hpp will not make C api visible. An explicit include to telemetry.h would be required.

Because the transport should be provided by default, since platform is supported.

To allow for custom data types

Thanks Rémi for this really this really useful library. I am trying to run the simple example code
on a arduino and get compiler error
testTelemetry:4: error: 'Telemetry' does not name a type
Telemetry TM;
Any suggestions as to what the problem is?
Thanks.

I'm using the Arduino Library on an Arduino Uno.

For the first test, I want to send a simple integer value.
void loop() {
TM.pub_u8("abcd",10);
delay(1000);
}

The received data in Ubuntu on the serial port are:
\xF7\x01\x00""abcd\x00\n\x02}\x7F\x7F

Do you have an idea, what's wrong? Which data should be received for sending "pub_u8("abcd",10)"?

Adding a platform should be made easier. Ideally, add a #define PLATFORM_NAME and define all 4 functions read readable etc.
Then, build system should be able to compile for this new platform and generate the appropriate distribution using the standard build flow.

I'm looking for a simple way to get some data across a Bluetoother RfComm channel. Telemetry looks like a nice option, but I need the data to be received by an Android phone. If I implemented this would it be accepted into this repository? Or any suggestions on how to pursue this?

In the idea to add Continous Integration to this repo, tests should be written for at least the Mbed distribution.
There is no true technical limit preventing it. The BufferedSerial library should be mocked.

The tests should be limited to a dictionnary (topic, data, datatype) -> frame containing both valid and non valid frames. This dictionnary could be built with a script using the C api.
The code under test will be challenged against this dictionnary and must pass all frames.

Such improvement could allow for very fast iteration of the protocol, and could allow to support a lot more platforms such as Arduino, etc.

They are not tied to internal class properties and are impossible to use inside callbacks. To move outside

• dictionnary implementation for Arduino and / or mbed ?
• add function filter(TM_msg * msg, const char * topic) that returns true if received msg matched a given topic
• add function filter(TM_msg * msg, const char * topic, TM_type type) that returns true if received msg matched a given topic and received type matched a given type
• add function update(TM_msg * msg, const char * topic, TM_type type, float & output) that returns true if received msg matched a given topic, type matches as well, and writes received data inside output variable. Do it for every supported type

I work in an environment Ubuntu 16.04 LTS and I use a Teensy microcontroller 3.2.
I downloaded Telemetry-2.0.1 and I identified the file telemetry_version.h
With the compiler of the Arduino IDE:

#include <telemetry_version.h>
generates an error, details are:
Arduino : 1.6.9 (Linux), TD: 1.30, Carte : "Teensy 3.2 / 3.1, Serial, 96 MHz optimize speed (overclock), US English"

/home/leruste/SCAO/Arduino/test/test.ino/test.ino.ino:1:31: fatal error: telemetry_version.h: No such file or directory
 #include <telemetry_version.h>
                               ^
compilation terminated.
Error compiling for board Teensy 3.2 / 3.1.

Ce rapport pourrait être plus détaillé avec
l'option "Afficher les résultats détaillés de la compilation"
activée dans Fichier -> Préférences.

In advance, thank you for helping me solve this problem.

Cordially.

Regis LERUSTE

For now the C++ api is just part of Mbed.
Move code in core repository.
Implement interface
Include in build system
Test with catch

This will require minor changes in C api as well.

Need to define a context structure holding all telemetry-related information. Then define internal methods such as

void _publish_f32(TM_context* context, char * topic, float data)

That will be called internally by

void publish_f32(char * context, float data)

For C api, a single context will be created and existing interface will be kept. For C++, a new context will be instanciated by each new class instance

For now, transports will allow to not send a frame if buffer is full.

void send(void * buf, uint32_t size)
{
  // If user forgot to define transport by calling init_telemetry, abort
  if(!transportPtr)
    return;

  if(transportPtr->writeable() && size > 0)
  {
    transportPtr->write(outgoingBuffer, size);
  }
}

User should be given the choice to wait or discard

Instead, it should be possible to set the state later.

init_telemetry(&transport);
// Late initialization
TM_state state;
set_state_telemetry(&state);

This will make the library easier to understand on first approach, and has no impact on functionnality

• Adding a # to a topic will describe specific data types

Examples:
foo#xy (xy coordinates)
foo#xyz (xyz coords)
foo#m22 (2_2 matrix)
foo#m33 (3_3 matrix)

• Associated with varargs interface for publishing
  Publish pair of coordinates x = 0.5 y = 0.9
  TM.publish("foo#xy",0.5,0.9);
  Publish matrix
  0.5 1.0
  1.5 2.0
  TM.publish("foo#m22",0.5,1.0,1.5,2.0);
• And the plots opened by the command line interface will understand all those datatypes.
  Coordinates plotted in a 2D or 3D space
  Matrices will be displayed as transformation matrix on an arbitrary object; in 2D or 3D space.

Topics with hashtags will be referenced as tagged ?

See Overdrivr/pytelemetry#8

First Rémi, thank you for your hard work. I ported telemetry (plain C) to the K22FN512 (a cortex M4 cpu) platform on the K22F freedom board. And only recently I'm in the need of sending data from the host to the embedded.

I'm facing the issue that my variable is not updated even thou that I use update_telemetry(). I'm sure the issue is with:

void update_telemetry()
{
  // If user forgot to define transport by calling init_telemetry, abort
  if(!transportPtr)
    return;

  uint32_t amount = transportPtr->readable();
  uint32_t i = 0 ;
  for(i = 0 ; i < amount ; i++)
  {
    uint8_t c;
    transportPtr->read(&c,1);
    feed(c);
  }
}

The issue is that I don't have any "serial.readable();" And place the following in the driver:

*/ int32_t readable() {
    uint32_t amount = 0;

    return amount;
}

Now my question is, amount has to be a single 8 bit char (amount = 1)or a whole message (amount = N)? That is the main difficult to porting since I do not know wow the real function "readable" works..

warmest regards

The goal is to simplify the process for the user to make some variables remotely writeable

#include <Telemetry.h>

Telemetry TM;

void setup() {
  // Init the writeable parameter
  int32_t someParam = 0;

  TM.begin(115200);
  TM.attach_i32("foo", &someParam); // someParam is now writeable by publishing int32_t on topic "foo" 
}

void loop() {
  TM.pub_i32("feedback",myParameters.someParam);
  TM.update();
  delay(10); 
}

Current limitations of mbed build system

• all .c files must be renamed to .cpp

Using arm gcc cross compiler https://launchpad.net/gcc-arm-embedded
Should output a firmware compatible with arm mbed on mass-storage-erase bootloader

telemetry_core.c

void try_update_hashtable(TM_msg * msg)

During compilation: "warning: enumeration value 'TM_string' not handled in switch"

May want to include a default case or explicitly handle this case to reduce compiler warnings. I realize that it is only one warning, but larger projects can end up with many such warnings which can cause documentation to have to be written to justify leaving the warning or cause someone to fork your repository without keeping it up.

Too error prone. To pass with subscribe

Currently, read and write are defined as

uint32_t (void * data, ..)

It should be better to use

uint32_t (uint8_t * data, ..

https://www.cs.auckland.ac.nz/software/AlgAnim/huffman.html
https://en.wikipedia.org/wiki/Huffman_coding

The program was running a loop in which I was plotting 3 variables.
The loop continue while à push a button.
and when I pushed the button this error append.
and no more variable were received ...

:> Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/threading.py", line 862, in run
self._target(_self._args, *_self._kwargs)
File "/usr/local/lib/python3.5/site-packages/pytelemetrycli/runner.py", line 43, in run
self.telemetryWrapper.update()
File "/usr/local/lib/python3.5/site-packages/pytelemetry/pytelemetry.py", line 65, in update
self.api.update()
File "/usr/local/lib/python3.5/site-packages/pytelemetry/telemetry/telemetry.py", line 145, in update
self.delimiter.decode(c)
File "/usr/local/lib/python3.5/site-packages/pytelemetry/telemetry/framing.py", line 52, in decode
self.on_frame_decoded_callback(self.payload)
File "/usr/local/lib/python3.5/site-packages/pytelemetry/telemetry/telemetry.py", line 148, in _on_frame_detected
topic_data = self._decode_frame(frame)
File "/usr/local/lib/python3.5/site-packages/pytelemetry/telemetry/telemetry.py", line 121, in _decode_frame
data, = unpack_from(fmt, frame, i+1)
struct.error: unpack_from requires a buffer of at least 8 bytes