ggerganov / wave-share Goto Github PK

View Code? Open in Web Editor NEW

2.2K 40.0 138.0 117 KB

Serverless, peer-to-peer, local file sharing through sound

Home Page: https://ggerganov.github.io/wave-share

License: MIT License

C++ 56.99% JavaScript 35.53% Shell 0.56% CMake 6.93%

data-over-sound webrtc fsk p2p webrtc-signaling file-sharing data-transfer modem ultrasonic

wave-share's Introduction

wave-share

A proof-of-concept for WebRTC signaling using sound. Works with all devices that have microphone + speakers. Runs in the browser.

Nearby devices negotiate the WebRTC connection by exchanging the necessary Session Description Protocol (SDP) data via a sequence of audio tones. Upon successful negotiation, a local WebRTC connection is established between the browsers allowing data to be exchanged via LAN.

See it in action (2min video):

Try it yourself: ggerganov.github.io/wave-share

Latest news:
I've extracted the data-over-sound algorithm into a standalone library called ggwave.
It can be embedded easily into other projects.

How it works

The WebRTC technology allows two browsers running on different devices to connect with each other and exchange data. There is no need to install plugins or download applications. To initiate the connection, the peers exchange contact information (ip address, network ports, session id, etc.). This process is called "signaling". The WebRTC specification does not define any standard for signaling - the contact exchange can be achieved by any protocol or technology.

In this project the signaling is performed via sound. The signaling sequence looks like this:

Peer A broadcasts an offer for a WebRTC connection by encoding the session data into audio tones
Nearby peer(s) capture the sound emitted by peer A and decode the WebRTC session data
Peer B, who wants to establish connection with peer A, responds with an audio answer. The answer has peer B's contact information encoded in it. Additionally, peer B starts trying to connect to peer A
Peer A receives the answer from peer B, decodes the transmitted contact data and allows peer B to connect
Connection is established

The described signaling sequence does not involve a signaling server. Therefore, an application using signaling through sound can be, for example, served by a static web page. The only requirement is to have control over the audio output/capture devices.

An obvious limitation (feature) of the current approach is that only nearby devices (e.g. within the same room) can establish connection with each other. Moreover, the devices have to be connected in the same local network, because NAT is not available.

Sound Tx/Rx

The data communicated through sound contains the contact information required to initialize the WebRTC connection. This data is stored in the Session Description Protocol (SDP) format. Since data-over-sound has significant limitations in terms of bandwidth and robustness it is desirable to transmit as few data as possible. Therefore, the SDP is stripped from all irrelevant information and only the essential data needed to establish the connection is transmitted. Currently, the sound packet containing the minimum required SDP data has the following format:

Size, [B]	Description
1	Type of the SDP - Offer or Answer
1	Packet size in bytes (not including ECC bytes)
4	IP address of the transmitting peer
2	Network port that will be used for the communication
32	SHA-256 fingerprint of the session data
40	ICE Credentials - 16 bytes username + 24 bytes password
32	ECC correction bytes used to correct errors during Tx

The total size of the audio packet is 112 bytes. With the current audio encoding algorithm, the SDP packet can be transmitted in 5-10 seconds (depending on the Tx protocol used). Using slower protocols provides more reliable transmission in noisy environments or if the communicating devices are far from each other.

Data-to-sound encoding

The current approach uses a multi-frequency Frequency-Shift Keying (FSK) modulation scheme. The data to be transmitted is first split into 4-bit chunks. At each moment of time, 3 bytes are transmitted using 6 tones - one tone for each 4-bit chunk. The 6 tones are emitted in a 4.5kHz range divided in 96 equally-spaced frequencies:

Freq, [Hz]	Value, [bits]	Freq, [Hz]	Value, [bits]	...	Freq, [Hz]	Value, [bits]
`F0 + 00*dF`	Chunk 0: `0000`	`F0 + 16*dF`	Chunk 1: `0000`	...	`F0 + 80*dF`	Chunk 5: `0000`
`F0 + 01*dF`	Chunk 0: `0001`	`F0 + 17*dF`	Chunk 1: `0001`	...	`F0 + 81*dF`	Chunk 5: `0001`
`F0 + 02*dF`	Chunk 0: `0010`	`F0 + 18*dF`	Chunk 1: `0010`	...	`F0 + 82*dF`	Chunk 5: `0010`
...	...	...	...	...	...	...
`F0 + 14*dF`	Chunk 0: `1110`	`F0 + 30*dF`	Chunk 1: `1110`	...	`F0 + 94*dF`	Chunk 5: `1110`
`F0 + 15*dF`	Chunk 0: `1111`	`F0 + 31*dF`	Chunk 1: `1111`	...	`F0 + 95*dF`	Chunk 5: `1111`

For all protocols: dF = 46.875 Hz. For non-ultrasonic protocols: F0 = 1875.000 Hz. For ultrasonic protocols: F0 = 15000.000 Hz.

Getting the local IP address

For convenience, a simple WebRTC hack is used to automatically detect the local IP address of your machine, so you don't have to provide it manually. However, the latest WebRTC spec prevents this from being possible for security reasons, so at some point this "feature" will stop working in all browsers. For example, it no longer works on Safari.

Build

Web Assembly module `wave.wasm`

You will need an Emscripten compiler. Run the compile.sh script.

CLI tool `wave-share`

Important: This CLI tool was the prototype for the now standalone library ggwave. Make sure to check it out, as it has more up-to-date examples for the application of this data-over-sound type of communication.

This is a simple tool that receives and sends data using the explained wave-share sound tx/rx protocol. Type some text on the standard input and press Enter to transmit.

# build
git clone https://github.com/ggerganov/wave-share
cd wave-share && mkdir build && cd build
cmake ..
make

# running
./wave-share

Here is a short video demonstrating how to use the CLI tool:

Known problems / stuff to improve

Does not work with: IE, IE Edge, Chrome/Firefox on iOS, Safari on macOS
Ultrasonic sound transmission does not work on most devices. Probably hardware limitations?
In presence of multiple local networks, cannot currently select which one to use. Always the first one is used
There is occasionally sound cracking during transmission. Need to optimize the Tx code
The size of the emscripten generated .js is too big (~1MB). Rewrite in pure JS?
On mobile, using Firefox, the page can remain running in the background even after closing the tab

wave-share's People

Contributors

Stargazers

Watchers

Forkers

thearchiver nanangarsyad sensecollective trailblazr duyouhua justloveben fun-alex-alex2006hw send2cloud threatinteltest ghosthamlet eos21 cristian357r4 lauren2018 sahwar jeanpenty botsonar amitshah aqua-regia zyjiang08 rotron mdheller asdlei99 jimpick yjsyyyjszf raul215 tommyteavee zhangchn zaphodddd srikwit rkram3r vivekjishtu hirajanwin osterbit vasiltabakov edwinyzh starters ackza mahmoudimus tavein ohoud21 cybernetics explife0011 vn-os softwareinspiration hurd-x0 blueff h4xl0r sec-js devilld coderaldrich phil153 well-it-wasnt-me 5l1v3r1 arman1388 karanpatel863 m4rm0k ch4p34un0ir 422053362 world-wide-hackers dileepindla zjnxyz cualquiercosa327 marciopocebon nsauzede animeshsahuind fsoeltoni fadinglr zhjygit chaichaizi vincentlz cursorhu donfanning akmalviya03 0xbadcoffe c0axial berkshireasia smoonthsky imuzi army11711 wjt2015 sts0mrg0 elmoiv zhaopufeng jjwygjj jinghao666 flywz afriestad nwuking ajunlonglive xjtunk mikerobotics cloudnepal dwebfan fairhopeweb shalevy1 icodein 00mjk chetankomal suryatmodulus birdol

wave-share's Issues

C compilation issues

Hello, Georgi.

Awesome idea, awesome execution.
Really impressed with the speed of transmission in your demo.

# git clone "https://github.com/ggerganov/wave-share"; cd wave-share && mkdir build && cd build; cmake ..; make

Cloning into 'wave-share'...
remote: Enumerating objects: 122, done.
remote: Total 122 (delta 0), reused 0 (delta 0), pack-reused 122
Receiving objects: 100% (122/122), 107.20 KiB | 560.00 KiB/s, done.
Resolving deltas: 100% (62/62), done.
-- The C compiler identification is GNU 10.2.0
-- The CXX compiler identification is GNU 10.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found FFTW: /usr/lib/libfftw3.so;/usr/lib/libfftw3f.so  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vganzin/work/whatnot/wave-share/build
Scanning dependencies of target wave-share
[ 50%] Building CXX object CMakeFiles/wave-share.dir/main.cpp.o
In file included from /home/vganzin/work/whatnot/wave-share/main.cpp:7:
/home/vganzin/work/whatnot/wave-share/reed-solomon/rs.hpp: In member function ‘void RS::ReedSolomon::EncodeBlock(const void*, void*)’:
/home/vganzin/work/whatnot/wave-share/reed-solomon/rs.hpp:76:17: warning: ISO C++ forbids variable length array ‘stack_memory’ [-Wvla]
   76 |         uint8_t stack_memory[MSG_CNT * msg_length + POLY_CNT * ecc_length * 2];
      |                 ^~~~~~~~~~~~
/home/vganzin/work/whatnot/wave-share/reed-solomon/rs.hpp: In member function ‘int RS::ReedSolomon::DecodeBlock(const void*, const void*, void*, uint8_t*, size_t)’:
/home/vganzin/work/whatnot/wave-share/reed-solomon/rs.hpp:152:17: warning: ISO C++ forbids variable length array ‘stack_memory’ [-Wvla]
  152 |         uint8_t stack_memory[MSG_CNT * msg_length + POLY_CNT * ecc_length * 2];
      |                 ^~~~~~~~~~~~
[100%] Linking CXX executable wave-share
/usr/bin/ld: CMakeFiles/wave-share.dir/main.cpp.o: in function `update() [clone .part.0]':
main.cpp:(.text+0x747): undefined reference to `SDL_PollEvent'
/usr/bin/ld: main.cpp:(.text+0x76c): undefined reference to `SDL_PauseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x78e): undefined reference to `SDL_GetQueuedAudioSize'
/usr/bin/ld: main.cpp:(.text+0x7c9): undefined reference to `SDL_PauseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x7d4): undefined reference to `SDL_CloseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x7e4): undefined reference to `SDL_PauseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x7ef): undefined reference to `SDL_CloseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x7f4): undefined reference to `SDL_CloseAudio'
/usr/bin/ld: main.cpp:(.text+0x7f9): undefined reference to `SDL_Quit'
/usr/bin/ld: main.cpp:(.text+0x806): undefined reference to `SDL_PauseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x816): undefined reference to `SDL_PauseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x859): undefined reference to `SDL_PauseAudioDevice'
/usr/bin/ld: main.cpp:(.text+0x8e8): undefined reference to `SDL_ClearQueuedAudio'
/usr/bin/ld: CMakeFiles/wave-share.dir/main.cpp.o: in function `init()':
main.cpp:(.text+0x95c): undefined reference to `SDL_LogSetPriority'
/usr/bin/ld: main.cpp:(.text+0x966): undefined reference to `SDL_Init'
/usr/bin/ld: main.cpp:(.text+0x988): undefined reference to `SDL_SetHintWithPriority'
/usr/bin/ld: main.cpp:(.text+0x996): undefined reference to `SDL_GetNumAudioDevices'
/usr/bin/ld: main.cpp:(.text+0x9b5): undefined reference to `SDL_GetAudioDeviceName'
/usr/bin/ld: main.cpp:(.text+0x9d5): undefined reference to `SDL_GetNumAudioDevices'
/usr/bin/ld: main.cpp:(.text+0xa08): undefined reference to `SDL_GetAudioDeviceName'
/usr/bin/ld: main.cpp:(.text+0xa35): undefined reference to `SDL_memset'
/usr/bin/ld: main.cpp:(.text+0xa6d): undefined reference to `SDL_memset'
/usr/bin/ld: main.cpp:(.text+0xa82): undefined reference to `SDL_GetAudioDeviceName'
/usr/bin/ld: main.cpp:(.text+0xaa6): undefined reference to `SDL_GetAudioDeviceName'
/usr/bin/ld: main.cpp:(.text+0xab9): undefined reference to `SDL_OpenAudioDevice'
/usr/bin/ld: main.cpp:(.text+0xbac): undefined reference to `SDL_GetAudioDeviceName'
/usr/bin/ld: main.cpp:(.text+0xbd3): undefined reference to `SDL_GetAudioDeviceName'
/usr/bin/ld: main.cpp:(.text+0xbeb): undefined reference to `SDL_OpenAudioDevice'
/usr/bin/ld: main.cpp:(.text+0xc00): undefined reference to `SDL_GetError'
/usr/bin/ld: main.cpp:(.text+0xd09): undefined reference to `SDL_GetError'
/usr/bin/ld: main.cpp:(.text+0xdba): undefined reference to `SDL_OpenAudioDevice'
/usr/bin/ld: main.cpp:(.text+0xdf4): undefined reference to `SDL_OpenAudioDevice'
/usr/bin/ld: main.cpp:(.text+0xe09): undefined reference to `SDL_GetError'
/usr/bin/ld: main.cpp:(.text+0xe1c): undefined reference to `SDL_LogError'
/usr/bin/ld: main.cpp:(.text+0xe30): undefined reference to `SDL_CloseAudio'
/usr/bin/ld: CMakeFiles/wave-share.dir/main.cpp.o: in function `DataRxTx::send()':
main.cpp:(.text._ZN8DataRxTx4sendEv[_ZN8DataRxTx4sendEv]+0x30d): undefined reference to `SDL_QueueAudio'
/usr/bin/ld: CMakeFiles/wave-share.dir/main.cpp.o: in function `DataRxTx::receive()':
main.cpp:(.text._ZN8DataRxTx7receiveEv[_ZN8DataRxTx7receiveEv]+0x137): undefined reference to `SDL_DequeueAudio'
/usr/bin/ld: main.cpp:(.text._ZN8DataRxTx7receiveEv[_ZN8DataRxTx7receiveEv]+0xdf6): undefined reference to `SDL_GetQueuedAudioSize'
/usr/bin/ld: main.cpp:(.text._ZN8DataRxTx7receiveEv[_ZN8DataRxTx7receiveEv]+0x1812): undefined reference to `SDL_GetQueuedAudioSize'
/usr/bin/ld: main.cpp:(.text._ZN8DataRxTx7receiveEv[_ZN8DataRxTx7receiveEv]+0x1858): undefined reference to `SDL_ClearQueuedAudio'
/usr/bin/ld: CMakeFiles/wave-share.dir/main.cpp.o: in function `main':
main.cpp:(.text.startup+0x41e): undefined reference to `SDL_Delay'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/wave-share.dir/build.make:105: wave-share] Error 1
make[1]: *** [CMakeFiles/Makefile2:95: CMakeFiles/wave-share.dir/all] Error 2
make: *** [Makefile:103: all] Error 2

p.s. I don't think you should be worried about 1mb emscripten module. Modern sites gets over 5mb easily and nobody complains.

sometimes the received string is wrong

Hello,
Sometimes the receiver print a string, but not the same to sender, there is some mistake, especially at low volume. Is there any solution, since the wrong return may cause other problems.

Compile ain't works

Hello!

I installed Emscripten compiler from synaptic.
Then i do compile.sh (with sudo. without have last line - permission denied)

Hope to find help! Thanks!

=========================

Traceback (most recent call last):
File "/usr/share/emscripten/em++.py", line 14, in
sys.exit(emcc.run(sys.argv))
^^^^^^^^^^^^^^^^^^
File "/usr/share/emscripten/emcc.py", line 1151, in run
linker_inputs = phase_compile_inputs(options, state, newargs, input_files)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/usr/share/emscripten/emcc.py", line 2677, in phase_compile_inputs
compile_source_file(i, input_file)
File "/usr/share/emscripten/emcc.py", line 2657, in compile_source_file
cmd = get_clang_command(input_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/emscripten/emcc.py", line 2598, in get_clang_command
return get_compiler(src_file) + get_cflags(state.orig_args) + compile_args + [src_file]
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/emscripten/emcc.py", line 903, in get_cflags
ports.add_cflags(cflags, settings)
File "/usr/share/emscripten/tools/ports/init.py", line 364, in add_cflags
port.get(Ports, settings, shared)
File "/usr/share/emscripten/tools/ports/sdl2.py", line 23, in get
ports.fetch_project('sdl2', 'https://github.com/libsdl-org/SDL/archive/' + TAG + '.zip', SUBDIR, sha512hash=HASH)
File "/usr/share/emscripten/tools/ports/init.py", line 247, in fetch_project
with shared.Cache.lock():
File "/usr/lib/python3.11/contextlib.py", line 137, in enter
return next(self.gen)
^^^^^^^^^^^^^^
File "/usr/share/emscripten/tools/cache.py", line 74, in lock
self.acquire_cache_lock()
File "/usr/share/emscripten/tools/cache.py", line 43, in acquire_cache_lock
raise Exception('Attempt to lock the cache but FROZEN_CACHE is set')
Exception: Attempt to lock the cache but FROZEN_CACHE is set

I feel like I don't understand what this is for

It looks like you're making a WebRTC modem so that you can get data across WebRTC

WebRTC has a data channel

Am I misunderstanding what this is for?