mbedtls_ssl_close_notify will silently failed if preceeded by an incomplete mbedtls_ssl_write about mbedtls HOT 2 OPEN

@gilles-peskine-arm You have considered a fix that would return WANT_WRITE instead of returning 0. I think that would create a problem for developers that use edge-based I/O (epoll). My code would end up waiting for the EPOLLOUT event that would never come.

I think that proper logic would require that flush is handled similarly to handshake. Maybe have a successful flush be followed by a close_notify attempt. I think this would require an additional field keeping track of the currently pending operation, as out_left would be insufficient.

from mbedtls.

I have found an existing problems when using edge-based I/O. With TLS 1.3 when the server sends a ticket, I get WANT_READ, even though more data is already available for reading. Since EPOLLIN never arrives, my code never proceeds further. I suspect similar use of WANT_* in other parts of the library.

In my opinion, a proper way to handle this would be to have WANT_READ, WANT_WRITE, and WANT_AGAIN. The last error code would cover exactly this case, and I would call the function again immediately.

Since that change would break existing code, I would propose a backward-compatibility layer such as:

mbedtls_ssl_read(...) {
    int ret = WANT_AGAIN;
    while (ret == WANT_AGAIN) {
        ret = mbed_ssl_read_impl(...);
    }
    return ret;
}

from mbedtls.