There is a race condition when piping to Websocat : please see this two tests, usi

Experimented a bit: <div class="snippet-clipboard-content notranslate position-rel

Try --no-close option in <a href="https://github.com/

--no-close --one-message <a href="https://github.com/

Does not seem to be failing for me: <div class="snippet-clipboard-content notransl

Another command to try: <div class="snippet-clipboard-content notranslate position

race condition when piping and closing,about vi/websocat

Comments (26)

vi commented on August 24, 2024 2

Note that --no-close --line --one-message is going to have a short version -nl1.

from websocat.

vi commented on August 24, 2024 1

Re-uploaded the files. Please re-check.

do you think it can be solved, or it is by design in windows?

Not sure. I don't have an actual Windows, so it's not convenient to develop Windows-specific things.

For info: the bug was that main thread was exiting without waiting for completion of the additional thread that writes to stdout.

from websocat.

vi commented on August 24, 2024

Does sending the Close message prevent it from receiving remaining buffered incoming messages?

I expected Websocket Close message to behave like shutdown(SHUT_RD) and only signal that there will be no more incoming data to server.

from websocat.

vi commented on August 24, 2024

Experimented a bit:

$ for i in `seq 1 40`; do echo  foo | websocat - ws://echo.websocket.org/; done | nl
     1	foo
     2	foo
     3	foo
     4	foo
     5	foo
     6	foo
     7	foo
     8	foo
     9	foo
    10	foo
    11	foo
    12	foo
    13	foo
    14	foo
    15	foo
    16	foo
    17	foo
    18	foo
    19	foo
    20	foo
    21	foo
    22	foo
    23	foo
    24	foo
    25	foo
    26	foo
    27	foo
    28	foo
    29	foo
    30	foo
    31	foo
    32	foo
    33	foo
    34	foo
    35	foo
    36	foo
    37	foo
    38	foo
    39	foo
    40	foo

At least most messages get though.

Maybe your server does not react to WebSocket close message properly (closes the connection without flushing the outgoing data)? Or maybe application returns from WebSocket incoming message's handler before sending the reply (e.g. queues up the request somewhere, then replies after serving it from queue)? In this case you should indeed avoid closing websocat's stdin too early by using read or sleep.

Maybe later I'll implement a "once" mode to make it send and receive exactly one WebSocket message.

from websocat.

vi commented on August 24, 2024

Try --no-close option in https://github.com/vi/websocat/releases/download/v1.0.0-beta/websocat_nossl_1.0_i686-pc-windows-gnu.exe.

You'll probably see the reply, but it will hang and not exit unless server closes the connection on its own behalf.

from websocat.

vi commented on August 24, 2024

--no-close --one-message https://github.com/vi/websocat/releases/download/v1.0.0-beta/websocat_nossl_1.0_i686-pc-windows-gnu.exe

from websocat.

alkorsan commented on August 24, 2024

Does sending the Close message prevent it from receiving remaining buffered incoming messages?

yes , the server don't send anything after receiving the close message (I inspected with wireshark).

Maybe your server does not react to WebSocket close message properly (closes the connection without flushing the outgoing data)?

yes, this is exactly what happens and I think it is the expected behavior, the server is the websocket server that google chrome protocol offer to instrument chrome devtool using the debugging protocol, so it follow the rfc6455, if I m correct the RFC says :

If an endpoint receives a Close frame and did not previously send a
   Close frame, the endpoint MUST send a Close frame in response.

so this is what chrome is doing, send the close when he received the close message, then the RFC says:

However, there is no guarantee that the endpoint that has already sent a Close frame will continue to process data.
After both sending and receiving a Close message, an endpoint
considers the WebSocket connection closed and MUST close the
underlying TCP connection.  The server MUST close the underlying TCP
connection immediately;

and I don t think it is possible to change the chrome behavior easly.

At least most messages get though.

yes, ws://echo.websocket.org is so fast, I get the response in the first milliseconds before sending the close message by Websocat, chrome is different as it have to do work before sending the response and it take seconds sometimes.

here is a quick test:
this will open chrome with the debugging port needed to control the websocket server of chrome:
$ ./chrome --remote-debugging-port=9222 --user-data-dir=tempProfile
this will open a new tab and save the ws link of this page so we can control it using the devtool debugging protocol:
$ WSurl=$(curl -sg http://127.0.0.1:9222/json/new | grep webSocketDebuggerUrl | cut -d'"' -f4 | head -1)
and now we can use the Websocat to communicate with $WSurl and control mostly everything in chrome.
the following example will make chrome navigate to www.example.com:
$ printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.example.com"} }' | $websocat --origin http://127.0.0.1:9222 --protocol tcp -t - "$WSurl"
and here it happen the race condition, because chrome need to wait until the page is loaded, then send the response to Websocat with fail or success. So I think it needs a timeout and close only after that timeout, like curl behavior.

I will test now the solution you made and comeback with the results. thank you very much for all.

from websocat.

alkorsan commented on August 24, 2024

with the new options --no-close --one-message , this is the situation, a new race condition:
wireshark shows that chrome websocket server always send the response as expected, but the response doesn't always get printed by websocat, seems to be another type of race condition that happens internally, in websocat parsing maybe? or maybe it is caused by the --one-message flag (because using this flag terminate the shell session but it doesn't send the close message to server)

here is two tests in cygwin, the first I got the response but the second test I did not receive it, but wireshark shows that chrome sent the response as you can see here:

in fact comparing the wireshark captures of both tests I can t find any difference between them, they are similar, but as you see in cygwin in test 1 websocat printed the response , and test 2 he did not print anything. I uploaded both tests pcap captures here
test 1 good.pcapng: https://www.cloudshark.org/captures/303296cf4a77
test 2 bad.pcapng: https://www.cloudshark.org/captures/423aa6a72e8e

from websocat.

vi commented on August 24, 2024

Does not seem to be failing for me:

$ for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | websocat --one-message --no-close --origin http://127.0.0.1:9222 --protocol tcp -t  - ws://127.0.0.1:9222/devtools/page/534C46061E2BA5ADDCCC792F8E3F3EE0; echo; } | nl
     1	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"412748BF684E5B20F2B04AD8E05F05DF"}}
     2	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"23E4FDF7B50F6E6C773DD81C6290C600"}}
     3	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"B0F280E6CF10E97EB7F2F41F1A6D9290"}}
     4	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"E8C271385F8C0218E1D2FD1A21A19F12"}}
     5	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"5C68B863C126001C2D36B434FFD9A3E5"}}
     6	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"8481D27B342060D63A306962341626C1"}}
     7	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"E84A17A90B60DA3E15479E9ACD0BAB30"}}
     8	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"10D538C950AACD3BF82AAA9C21CFBB8D"}}
     9	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"4D0108AF5A39E88EA22876B5FE454E94"}}
    10	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"D0AE7081A16CDAB01ACEE39A55121474"}}
    11	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"6A52843A86DF8A78AC5C5D3F4C3769EC"}}
    12	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"89DF5BCEC7CF903D0E5B7BEFA0F483DA"}}
    13	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"961E22440C7CED9FD17AC31C4B4C06EA"}}
    14	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"9DD5CF05C023EBFFA02C6C4F82A9BAFA"}}
    15	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"313CF29372A0DAEE0E0BA1015F7879B5"}}
    16	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"2B8FFB4D2F438BBFC3EEAA92F1F8EB76"}}
    17	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"A5E9132F438D87A2DD7ABB96541720BA"}}
    18	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"9ECAC2ABE67FFE3055FAC2E08FD2C5F3"}}
    19	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"36925974C9F08EBA28F5D9277F76F97A"}}
    20	{"id":2,"result":{"frameId":"534C46061E2BA5ADDCCC792F8E3F3EE0","loaderId":"AF5EA039B470DD01EB4039B73888CEAB"}}

Maybe the resulting line (which lacks trailing \n) gets overwritten by Cygwin shell prompt? Try adding ; echo; echo at the end. Try also running with RUST_LOG=info environment variable.

~~Another thing remains to be fixed is interaction between --line and --one-message.~~

from websocat.

vi commented on August 24, 2024

Another command to try:

printf ... | RUST_LOG=info websocat  --no-close --one-message --origin http://127.0.0.1:9222 --protocol tcp -t  - msg2line:ws://127.0.0.1:9222/devtools/page/534C46061E2BA5ADDCCC792F8E3F3EE0

Or more cleaner approach (to avoid hacky ;echo or msg2line:):

printf '%s\n' ... | websocat --no-close --one-message --line ...

from websocat.

alkorsan commented on August 24, 2024

using your examples here are the results:

$  for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat --one-message --no-close --origin http://127.0.0.1:9222 --protocol tcp -t  - $WSurl; echo; } | nl
     1  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"69E84F29D0831F01775CC03B98A4E35D"}}












     2  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"13009EF40443A46EE6C1F68B21F83B30"}}

again

$  for ((i=0; i<20; ++i)) { sleep 1 ;printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat --one-message --no-close --origin http://127.0.0.1:9222 --protocol tcp -t  - $WSurl; echo; } | nl
     1  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"E4FC6FE44E1E103490FCA599FC281015"}}

     2  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"93F6A4ED3288C8ABD705EE5F71E553D6"}}

     3  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"A4B43935CAB68BE2D6E1A78A47A9EAD4"}}
     4  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"C3378360592FA6DB9F4E70B4DE855273"}}
     5  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"660943EF43C6365B97330FFB86CFE2AA"}}
     6  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"CC91E20EF3D72234A366B995D19FF6AB"}}
     7  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"C0370B0CC4D1561C1C99A643425119EB"}}


     8  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"08165859BD84C904B43318E4B8E68EFC"}}

     9  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"C0C4DD8F09997C68E6737D4BD09D5F81"}}



    10  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"C00621065B3765A9291125AB259E1828"}}

I added echo $i :

$  for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat --one-message --no-close --origin http://127.0.0.1:9222 --protocol tcp -t  - $WSurl; echo $i; } | nl
     1  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"6E3B04365A98B3B81576439920520D0E"}}0
     2  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"9D8A73E6E5E3F97165C3A726874D4664"}}1
     3  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"48A848E703D98F89C324C3407797D229"}}2
     4  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"51B230B6C29BEBAF5E19695CB900CC53"}}3
     5  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"11B1F8151F71E4491400705D657BA6E5"}}4
     6  5
     7  6
     8  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"DC4E80B8ECA3FEB38797C16FAF9F6012"}}7
     9  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"154E0458D6846B0CB494728B39808986"}}8
    10  9
    11  10
    12  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"FCB45408D37DA62B3F1F3C9A53B2F48F"}}11
    13  12
    14  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"3DAF301C1D631214ED29A78F7F9A1703"}}13
    15  14
    16  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"AA03DA6D5433F3F25562B4C8A616E90F"}}15
    17  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"6F676CFEBFF85D29CDF5B26A019DF5BD"}}16
    18  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"6FC6D99072020B05554016A2093522E1"}}17
    19  18
    20  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"5713A60F44229EE0ED4BF249A8B6B5BE"}}19

$  for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat --one-message --no-close --origin http://127.0.0.1:9222 --protocol tcp -t  - $WSurl; echo $i; } | nl
     1  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"C6D2415343DECA149144EE84652F9D95"}}0
     2  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"40C065B94A306A30DBF7A8CA6AAA288D"}}1
     3  2
     4  3
     5  4
     6  5
     7  6
     8  7
     9  8
    10  9
    11  10
    12  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"70642FEAB45A2B15C0571502E47D0953"}}11
    13  12
    14  13
    15  14
    16  15
    17  16
    18  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"89C35BEAB535D59E4052F531619E72B4"}}17
    19  18
    20  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"7C913EA37CCB78B070A865D41DDDE66B"}}19

$  for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat --one-message --no-close --origin http://127.0.0.1:9222 --protocol tcp -t  - $WSurl; echo $i; } | nl
     1  0
     2  1
     3  2
     4  3
     5  4
     6  5
     7  6
     8  7
     9  8
    10  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"6A3566E35C9D4FCF61A9607DF1AA3484"}}9
    11  10
    12  11
    13  12
    14  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"82767B1B27F074D2612A03D4E4DB5410"}}13
    15  14
    16  15
    17  16
    18  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"10C900E3374888EC9D8175C4DF16F788"}}17
    19  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"95CAF0F76CCED3FC64C2E03E3BF3AD9E"}}18
    20  {"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"3B1172BE7A852DA1C186434205DEFF11"}}19

from websocat.

alkorsan commented on August 24, 2024

Another command to try:
printf ... | RUST_LOG=info websocat --no-close --one-message --origin http://127.0.0.1:9222 --protocol tcp -t - msg2line:ws://127.0.0.1:9222/devtools/page/534C46061E2BA5ADDCCC792F8E3F3EE0

the results:
here websocat printed the response (line 7)

$ printf  '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | RUST_LOG=info $websocat  --no-close --one-message --origin http://127.0.0.1:9222 --protocol tcp -t  - msg2line:$WSurl
 INFO 2018-06-21T19:37:16Z: websocat: Serving ThreadedStdio to Message2Line(WsClient("ws://127.0.0.1:9222/devtools/page/D666F02DF2D945BC3A383F34150FBB1C")) with Options { websocket_text_mode: true, websocket_protocol: Some("tcp"), udp_oneshot_mode: false, unidirectional: false, unidirectional_reverse: false, exit_on_eof: false, oneshot: false, unlink_unix_socket: false, exec_args: [], ws_c_uri: "ws://0.0.0.0/", linemode_retain_newlines: false, origin: Some("http://127.0.0.1:9222"), custom_headers: [], websocket_version: None, websocket_dont_close: true, one_message: true }
 INFO 2018-06-21T19:37:16Z: websocat::stdio_threaded_peer: get_stdio_peer (threaded)
 INFO 2018-06-21T19:37:16Z: websocat::ws_client_peer: get_ws_client_peer
 INFO 2018-06-21T19:37:16Z: websocat::ws_client_peer: Connected to ws
 INFO 2018-06-21T19:37:16Z: websocat: Forward finished
{"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"278B09EC51518C5B511D60188E227DDC"}}
 INFO 2018-06-21T19:37:18Z: websocat: Reverse finished
 INFO 2018-06-21T19:37:18Z: websocat: Finished

here the response was not printed even wireshark show that the response was sent to websocat

$ printf  '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | RUST_LOG=info $websocat  --no-close --one-message --origin http://127.0.0.1:9222 --protocol tcp -t  - msg2line:$WSurl
 INFO 2018-06-21T19:37:38Z: websocat: Serving ThreadedStdio to Message2Line(WsClient("ws://127.0.0.1:9222/devtools/page/D666F02DF2D945BC3A383F34150FBB1C")) with Options { websocket_text_mode: true, websocket_protocol: Some("tcp"), udp_oneshot_mode: false, unidirectional: false, unidirectional_reverse: false, exit_on_eof: false, oneshot: false, unlink_unix_socket: false, exec_args: [], ws_c_uri: "ws://0.0.0.0/", linemode_retain_newlines: false, origin: Some("http://127.0.0.1:9222"), custom_headers: [], websocket_version: None, websocket_dont_close: true, one_message: true }
 INFO 2018-06-21T19:37:38Z: websocat::stdio_threaded_peer: get_stdio_peer (threaded)
 INFO 2018-06-21T19:37:38Z: websocat::ws_client_peer: get_ws_client_peer
 INFO 2018-06-21T19:37:38Z: websocat::ws_client_peer: Connected to ws
 INFO 2018-06-21T19:37:38Z: websocat: Forward finished
 INFO 2018-06-21T19:37:40Z: websocat: Reverse finished
 INFO 2018-06-21T19:37:40Z: websocat: Finished

Or more cleaner approach (to avoid hacky ;echo or msg2line:):
printf '%s\n' ... | websocat --no-close --one-message --line ...

results:
here I got the response

$ printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | RUST_LOG=info $websocat  --no-close --one-message --line --origin http://127.0.0.1:9222 --protocol tcp -t  - $WSurl
 INFO 2018-06-21T19:51:30Z: websocat: Serving Line2Message(ThreadedStdio) to Message2Line(WsClient("ws://127.0.0.1:9222/devtools/page/D666F02DF2D945BC3A383F34150FBB1C")) with Options { websocket_text_mode: true, websocket_protocol: Some("tcp"), udp_oneshot_mode: false, unidirectional: false, unidirectional_reverse: false, exit_on_eof: false, oneshot: false, unlink_unix_socket: false, exec_args: [], ws_c_uri: "ws://0.0.0.0/", linemode_retain_newlines: false, origin: Some("http://127.0.0.1:9222"), custom_headers: [], websocket_version: None, websocket_dont_close: true, one_message: true }
 INFO 2018-06-21T19:51:30Z: websocat::stdio_threaded_peer: get_stdio_peer (threaded)
 INFO 2018-06-21T19:51:30Z: websocat::ws_client_peer: get_ws_client_peer
 INFO 2018-06-21T19:51:30Z: websocat::ws_client_peer: Connected to ws
 INFO 2018-06-21T19:51:30Z: websocat: Forward finished
{"id":2,"result":{"frameId":"D666F02DF2D945BC3A383F34150FBB1C","loaderId":"CFED751D525EC7DDC75A80F3916F7B8B"}}
 INFO 2018-06-21T19:51:30Z: websocat: Reverse finished
 INFO 2018-06-21T19:51:30Z: websocat: Finished

and here there were no response

$ printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | RUST_LOG=info $websocat  --no-close --one-message --line --origin http://127.0.0.1:9222 --protocol tcp -t  - $WSurl
 INFO 2018-06-21T19:51:28Z: websocat: Serving Line2Message(ThreadedStdio) to Message2Line(WsClient("ws://127.0.0.1:9222/devtools/page/D666F02DF2D945BC3A383F34150FBB1C")) with Options { websocket_text_mode: true, websocket_protocol: Some("tcp"), udp_oneshot_mode: false, unidirectional: false, unidirectional_reverse: false, exit_on_eof: false, oneshot: false, unlink_unix_socket: false, exec_args: [], ws_c_uri: "ws://0.0.0.0/", linemode_retain_newlines: false, origin: Some("http://127.0.0.1:9222"), custom_headers: [], websocket_version: None, websocket_dont_close: true, one_message: true }
 INFO 2018-06-21T19:51:28Z: websocat::stdio_threaded_peer: get_stdio_peer (threaded)
 INFO 2018-06-21T19:51:28Z: websocat::ws_client_peer: get_ws_client_peer
 INFO 2018-06-21T19:51:28Z: websocat::ws_client_peer: Connected to ws
 INFO 2018-06-21T19:51:28Z: websocat: Forward finished
 INFO 2018-06-21T19:51:29Z: websocat: Reverse finished
 INFO 2018-06-21T19:51:29Z: websocat: Finished

so the race condition is always happening!
I used RUST_LOG=trace and uploaded two tests to pastebin, because the logs have +150 lines and it may help.
bad example, no response printed: https://pastebin.com/F2EfYtrA
good example , the response was printed (look the last four lines): https://pastebin.com/w66Pq7kF

from websocat.

alkorsan commented on August 24, 2024

I will make a clean windows and do tests again, then if I get the same results, I will test linux to understand and compare maybe I can find the cause. thank you

from websocat.

vi commented on August 24, 2024

Do you always, 100% get the reply when not using --one-message? Expected "yes".

from websocat.

vi commented on August 24, 2024

Reproduced the problem on Linux with threadedstdio: instead of -.

There is no proper asynchronous stdin/stdou on Windows (at least currently), so - on Windows is forced to be threadedstdio:.

from websocat.

alkorsan commented on August 24, 2024

here are all the examples:
with the must/minimal flags

$ for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t - $WSurl; echo $i; } | nl

send but never print any response

using -l

$ for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t -l - $WSurl; echo $i; } | nl

doesn t even send the request, printf need \n

using -l and printf with \n

$ for ((i=0; i<20; ++i)) { printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t -l - $WSurl; echo $i; } | nl

send but never print any response

using --one-message

$ for ((i=0; i<20; ++i)) { printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t --one-message - $WSurl; echo $i; } | nl

send but never print any response

using --one-message --no-close

$ for ((i=0; i<20; ++i)) { printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t --one-message --no-close - $WSurl; echo $i; } | nl

send but race condition:sometimes print and other times not

using --no-close

$ for ((i=0; i<20; ++i)) { printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t --no-close - $WSurl; echo $i; } | nl

send only one time but never print any response and stay waiting, wireshark shows that a response was sent by chrome.

using --no-close --line

$ for ((i=0; i<20; ++i)) { printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t --no-close --line - $WSurl; echo $i; }

works fine!!! it always print the response , but it run only one time and stay waiting (seems that it need --one-message to break the waiting)

using --no-close --line --one-message

$ for ((i=0; i<20; ++i)) { printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t --no-close --line --one-message - $WSurl; echo $i; }

send and receive but print sometimes and sometimes not:the race condition

from websocat.

alkorsan commented on August 24, 2024

the beautiful and strange thing is this example that always works:

runscript() {
    for ((i=0; i<20; ++i)) {   
        printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }'
        read -r line
        echo "$i ___ Received response: $line" >&2
    } 
}
export -f runscript
$websocat -t -l $WSurl sh-c:'bash -c runscript'

it always print the response:

$ $websocat -t -l $WSurl exec:bash --exec-args -c runscript
0 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"51D0E5DC2615C19592367C52224F9D89"}}
1 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"DE96BAA08712802ADDF9122238E8F466"}}
2 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"F92DD93914794AEA5E44E3068ECA3AA8"}}
3 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"40217451E5387F1CC0D77678F3891EB9"}}
4 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"31EA873BCC115A9D32F5CF972C10B3E6"}}
5 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"749347DBA75E83259CA6EB266E6132ED"}}
6 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"D08854D3E86AA725FC0C2CD9FA1003CB"}}
7 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"7AADBBEFEEDB6B982E759975C4DC5B52"}}
8 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"3A92A5C850C30DA21664113F8A927ABD"}}
9 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"C8268959806900EB963791B28A5A0737"}}
10 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"1394760A553B4939AF64B8A62957A258"}}
11 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"7DC6E46BF7CFBFC16F11754A0DEC4C76"}}
12 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"526423FEA7FE4AB5684899EC41279E46"}}
13 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"E07F522A3320CB8FF3D76A9333931D40"}}
14 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"113EB34A5325785AC118D867D695C316"}}
15 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"55541F53B743EC6693A837DE7BCFF93F"}}
16 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"81B487E1DA0C1D0719337EC0663895B0"}}
17 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"FC8115EB0D2D10A0FBB2FBCD1A3A7FF6"}}
18 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"D0BF4461A3D756E6AE0E97578C4F0187"}}
19 ___ Received response: {"id":2,"result":{"frameId":"4FB7B6B91715D3B2C585EFC3AC98F40E","loaderId":"5C18359B36149014EC51C9D1C72A63B4"}}

from websocat.

alkorsan commented on August 24, 2024

There is no proper asynchronous stdin/stdou on Windows (at least currently), so - on Windows is forced to be threadedstdio:.

do you think it can be solved, or it is by design in windows?

from websocat.

alkorsan commented on August 24, 2024

I don't know how you did it so quickly but you solve it sir 🥇
normally this is done with expect or fifo/coproc methods like with telnet/netcat/socat/ssh... or any other interactive program, but with Websocat now we do not need to script that part :) 👍
thank you for this gem and all your effort, time and dedication.

from websocat.

alkorsan commented on August 24, 2024

To make it clear for new comers this is what one have to put:

method 1: using --no-close --line --one-message printf '%s\n' (the new line is obligatory or it will stop waiting...)

for ((i=0; i<20; ++i)) { printf '%s\n' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t --no-close --line --one-message - $WSurl; } | nl

good 0 loss

method 2: using --no-close --one-message and echo at the end to print a newline, method 1 is better

for ((i=0; i<20; ++i)) { printf '%s' '{ "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} }' | $websocat -t --no-close --one-message - $WSurl; echo $i; } | nl

good 0 loss

from websocat.

alkorsan commented on August 24, 2024

This is just to confirm that cmd.exe is working like cygwin/mintty without any problem with the new websocat beta version

for %i in (1 2 3 4 5 6 7 8 9) do echo { "id":2, "method":"Page.navigate", "params":{"url": "http://www.google.com"} } | %websocat% -t --no-close --one-message - ws://127.0.0.1:9222/devtools/page/3D4852B252E923E8A6F5341AE72701E3 & echo %i
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"EBD3A6C5CDB4198BE9824B6CF1779916"}}1
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"29F398ED632A57DB5D005B5F7FCF8EF1"}}2
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"030D6A44D7EF36CA492477AF6579A060"}}3
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"02EC554FE6BA2986CF385ECEA5120C44"}}4
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"087B5B0155B355B9EB1CC5577C31F7ED"}}5
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"E20755F6834E3471B74D04200323E92A"}}6
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"1FA591D6573D8DCABC4BF10EBD2CEED8"}}7
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"55AC20637B1F0CD5C8BEF08C4854CBB0"}}8
{"id":2,"result":{"frameId":"3D4852B252E923E8A6F5341AE72701E3","loaderId":"D5A86D920930BA9F7D4592DF4B3021FC"}}9

NB: in cmd.exe script it is for %%i, and in an interactive window it is for %i

from websocat.

vi commented on August 24, 2024

Do you feel Websocat is ready enough to be announced to potential users (e.g. in Reddit groups, etc.)?

from websocat.

alkorsan commented on August 24, 2024

To be correct in my advice I have to tell you a truth: in fact I was not looking for a socat alternative for websocket, I was looking for a command line client for websocket. so I can tell you about this part (the websocket client) but I can't speak about Socat part (because I don t know half of socat usage).

so, websocat as a websocket client is magical now it can do what no other CLI can do, I did a small study on more than 23 web socket client CLI, and I can confirm to you that from this list only 2 can do a little of what websocat can do , and they need scripting with expect or coproc/fifo. the other clients all work only interactivly only. in the other hand Websocat have all what is needed to use it in scripts now, which the others can t do. it merit to add another name "the websocket curl" which others are using but not making it to work. so I think websocat is ready to publish from my point of view of websocket CLI client.

in reality I was so frustrated because I did not found websocat early, I spent a week searching all the big internet using google github yandex baidu hackernews slashdot metager unbubble stack-exchange/overflow ... and websocat never appered in the results. when I beleived that it doesn t exist a websocket client from cli , I made a small stupid search on how to integrate one of those interactive websocket clients that exist already, so I searched this noob keywords in google: wscat script and I got this https://stackoverflow.com/questions/48912184/wscat-commands-from-script-how-to-pass

so I was surprised to have the answer of my long search in my last noob search, lol,this is how I found websocat!!; and I was telling to my self why the owner of websocat is not publishing it as a websocket client from command line . you know sir a lot of people need this , a lot of people are using the command line and don t want to use node only for this, and there exist no solution that can do it , in fact, I found that people don't want to program it, programmers don't see any benefit in a websocket client for the command line, because they say "websocket is for javacript and should be used from there" , and I m still thinking in why ...

so you solved a big problem for us (command line users) at less.
I searched too in npm crates and a lot of other sites I don t remember, but your Websocat never appered in the results, and I think it s because of this :
let s see in crates for exemple what it says:
https://crates.io/crates/websocat

Connection forwarder from/to web sockets to/from usual sockets, in style of socat

when I read this I will never use it, I don t know what this phrase mean in fact, it s like a chemical function , and search engines will never show it because even the word websocket (without space) is not present, so I think you have to work on SEO with the descriptions

the actual documentation is hard to understand, maybe because I needed simple instructions for a simple usage of a simple websocket client form a simple command line. if it was not your comments this days while solving those issues, and the article of stackoverflow, I would never know all the power of Websocat.
so this is the point , the documentation don t tell you how to use it as a websocket client, but how to use it as a socat.

another point which is very important i think is the ssl, today everything is ssl, so large part of users will not see any benefit yet without ssl. but this can be published in another day like websocat v2 now support ssl.

other than this, I think it is ready to publish as a websocket command line client for scripts and interactively.

a last word, I think socat users are few, they are the pros and advanced users, generally today people are specialists of some domain but ignore the majority of other solutions or even there existence, so i think publishing websocat as a websocket client for the command line scripting will attract more people than publishing as a socat chemical functions which few people will understand and attempt to touch it. but as I told you i m ignorant of half of socat usage so maybe I m wrong in all this.

tell me if you want me to publish the list of the other cli I did study (it s ready , a list of links and why they don t work)

sorry to be so long

if you want me to translate anything to arabic frensh or spanish I m ready and very good at , sorry that english is my last language and i m very bad with it, so can t help to make the documentation easier :(

from websocat.

vi commented on August 24, 2024

Maybe open a separate issue about popularisation of websocat with a

checklist

of items to be done (i.e. better documentation, more keywords, description changes, posting a comment to specific public resources).

(syntax for the checklist is * [ ] an item)

from websocat.

vi commented on August 24, 2024

Maybe you are also interested in discussing websocat in chat format?

from websocat.

alkorsan commented on August 24, 2024

cool

from websocat.

race condition when piping and closing about websocat HOT 26 CLOSED

Comments (26)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent