Need more porting documentations about bare metal system about azure-c-shared-utility HOT 15 CLOSED

Hi @maxpeng,
The big question is what TLS you want to use. If OpenSSL will work for you, then I suggest you start here, which is a working example for ESP32, and adapt out the FreeRTOS-specific stuff, which there isn't much of. This project already uses LWIP directly via socket_async.c, and should be able to adapt it to your own LWIP just by rewriting (or perhaps re-using) the socket_async_os.h file, which just points socket_async.c to the correct LWIP includes.

The Espressif SDK that the sample points you to contains the implementation of OpenSSL that the sample uses. If you can't use the ESP32 sample's OpenSSL, then things get more complicated, and we'd need to know what TLS you are using.

from azure-c-shared-utility.

Hi Roy,
Thanks for your reply!

I use mbed TLS for porting. From your reply, I read tlsio_openssl_compact.c, which also uses socket_async.c. There is no problem to implement the same functionality provided by socket_async.c by using lwip RAW TCP API.

So the left question, will I lose some functionalities if I don't implement xio socket adapter?

from azure-c-shared-utility.

@maxpeng

If you are using MBED TLS it sounds like you should use the existing binding to MBED TLS (https://github.com/Azure/azure-c-shared-utility/blob/master/adapters/tlsio_mbedtls.c). The lower bound of this adapter is also an IO interface and you would need a socket IO implementation that works for your system.

For example if you have a Berkeley sockets API adapter over lwip for your target then you could reuse the socketio_berkeley.
Otherwise you can write an socket IO implementation that calls into lwip.

HTH,
/Dan

from azure-c-shared-utility.

Hi Dan,

Yes, I am trying to port tlsio_mbedtls.c for bare metal + lwip, since not RTOS is used, I can not use socket API of lwip.

It seems I have 2 options to implement the TLS functionality, one is use "asynchronous socket", another one is implement "socket IO". I want to know the consequences of these 2 different methods.

Thanks!

from azure-c-shared-utility.

Hi Max,

As Dan said, you'll want to start with tlsio_mbedtls.c and societio_berkeley.c instead of tlsio_openssl_compact.c and socket_async.c. The sample I pointed at earlier is still otherwise a good starting point. I had to do a quick and dirty port of mbedTLS just a few days ago and it worked okay.

from azure-c-shared-utility.

You are going to need a socket API of some sort. Lwip provides a socket API -- you can't use that? Having the streaming abstraction is essential. Doing DNS lookup without using lwip is also going to be extremely painful, and lwip's gettaddrinfo is based on lwip's sockets.

from azure-c-shared-utility.

From lwIP WIKI - Porting For Bare Metal

In NOSYS mode, the netconn and socket APIs are not available. All applications must be written using the raw (or native) API.

That's why I cannot use blocking mode socket API.

What is the advantage of using socket IO?

from azure-c-shared-utility.

Socketio and sockets are not the same thing. The socketio that Dan was referring to is an Azure IoT layer written on top of sockets. Sockets is an API found in full lwip that implements the TCP layer of IP communication. Azure IoT relies on TLS for communication, and TLS relies on TCP. Without a TCP layer (e.g. sockets) you cannot do TLS. So it while it would be possible for you to re-implement an entire TCP stack on bare metal it would mean re-implementing lwip except with the added challenge (which the lwip team declined to accept) of doing it with no threading support. I don't see how this could be made practical.

from azure-c-shared-utility.

Hi Roy,

I fully understand what is the socket for network communication, maybe I didn't describe my question clearly.

Let's assume what defines by socketio.h is an interface, I see wsio.c references uws_client.h, uws_client.c references socketio.h, that means wsio depends on the socketio interface. Because I don't understand the functionality prvodies by wsio now, I want to know what features will be missed if I don't implement the socketio interface.

from azure-c-shared-utility.

Hi Max,

Sorry for the overexplanation; let me see if I understand your question better now. You don't necessarily need a socketio implementation at all. The important piece that you need is a tlsio adapter, which is an xio interface that performs TSL communication. Most, but not all, of our tlsio adapters use one or another implementation of scoketio; The tlsio_openssl_compact, which uses socket_async.c is one exception, and tlsio_arduino.c, which uses arduino system internals is another.

So neither socketio nor sockets is essential. The essential piece is that the platform_get_default_tlsio() call in the platform adapter that you write returns the tlsio adapter that you implement and that the tlsio adapter meets this spec. This is the component that the upper layers do all the talking to Azure with, and it's the only one of the required adapters that's non-trivial.

In terms of code that you can look at to see how it's done, tlsio_openssl_compact avoids socketio and hews most closely to the spec, and tlsio_mbedtls.c uses mbed but requires a socketio, so they are closest to what you're doing.

from azure-c-shared-utility.

After thinking about this a bit more I should add a couple of things. One is that socket_async.c is a thin wrapper around a socket, which probably disqualifies it for your application. The other is that if you were to implement a bare-metal socketio and use it with tlsio_mbedtls.c then you'd have the puzzle solved. Implementing socketio is probably the only way to go for your situation, though. Despite its name, socketio doesn't actually expose a socket; it is just presumed to use one. The easiest thing would probably be for you to start with socketio_berkeley.c and replace the socket calls with the bare-metal stuff. If you do, be aware that you should destroy queued messages when close() is called rather than destroy(), which is its current behavior.

from azure-c-shared-utility.

@maxpeng
Sorry for the delay.
Basically what Roy said, best is to start from socket_berkeley and develop your IO implementation that talks to whatever socket API you have on your platform.

As background, the reason for the IO API is to have an abstraction that can be used by libraries like uAMQP, uMQTT etc., which have a lower bound that talks to a stream (bytes in/bytes out). For example the AMQP connection layer does not care to what concrete implementation talk to at its lower bound (be that directly to a TCP socket, a TLS implementation, a Websocket tunnel, etc.). That makes a very nice separation of concerns which allows the layers to be designed properly, not having unwanted dependencies.

To clarify one more question: what does wsio do?
wsio is an IO implementation that uses the uws_client (which is a slim WebSocket client). It manages the WebSocket handshake and once the IO is open will send receive the bytes over WebSocket frames.
This essentailly gives WebSocket support to the AMQP and MQTT libraries above it.

HTH,
/Dan

from azure-c-shared-utility.

Dan & Roy, thanks for your explanation.

Dan, you mentioned about websocket, it seems it could provide a solution to solve the proxy problem if the device locates behind a proxy and firewall. This drives me to try the iothub_client_sample_mqtt_websockets example on Linux, I see it uses httpapi and libcurl. Is the httapi and libcurl the necessity for constrained device to use websocket?

from azure-c-shared-utility.

@maxpeng
Indeed, the Websocket tunnel is used to address issues with proxies for AMQP and MQTT (it is a rather common case in corporate environments and not only for the AMQP and MQTT ports for TCP to be blocked).
The wsio and uws_client implementation does not require httpapi or curl. It is self contained. Without looking at the CMakeLists and full contents, iothub_client_sample_mqtt_websockets requiring httpapi and curl might be due to linking in the file upload feature.
All in all, when using AMQP or MQTT over WS for messaging or device twin you will not need httpapi/curl on your embedded device.

HTH,
/Dan

from azure-c-shared-utility.

Thanks for Dan and Roy, I close this because I have got the necessary guidance for the porting.

from azure-c-shared-utility.