discordjs / voice Goto Github PK

Currently, @discordjs/voice hooks directly into Discord.js. It would be nicer if the users could set their own adapters so they could use it with other Discord API libraries or some other arbitrary gateway connection.

The repo should be updated to have documentation that is compatible with the documentation generator, and then the site repository https://github.com/discordjs/site should also be updated to use this documentation.

It would be nice to have a comprehensive example showing how an audio player (using a queue) can be implemented.

Please describe the problem you are having in as much detail as possible:

After joining a voice channel, you are unable to switch the voice channel programmatically without severing the connection and starting over.

Include a reproducible code sample here, if possible:

const connection = joinVoiceChannel(channel1);

// wait some time

joinVoiceChannel(channel2);

// nothing happens

Please add documentation or an example for Javascript.

The entersState helper should be exported from the main library since many people will find it useful. It makes a good replacement for Events.once with AbortControllers for those who aren't on Node.js v15.

When trying to build with tsc, I get several missing module errors. I'm not sure if this is an edge case or not, as I'm not using discord.js, rather a library called discord-rose, so my suspicion is that these missing modules are installed with discord.js, so they are overlooked. Here is the error stack I received:

node_modules/@discordjs/voice/dist/audio/AudioPlayer.d.ts:5:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.

5 import TypedEmitter from 'typed-emitter';
                           ~~~~~~~~~~~~~~~

node_modules/@discordjs/voice/dist/DataStore.d.ts:1:32 - error TS2307: Cannot find module 'discord-api-types/v8/gateway' or its corresponding type declarations. 

1 import { GatewayOPCodes } from 'discord-api-types/v8/gateway';
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/@discordjs/voice/dist/networking/Networking.d.ts:4:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.    

4 import TypedEmitter from 'typed-emitter';
                           ~~~~~~~~~~~~~~~

node_modules/@discordjs/voice/dist/networking/VoiceUDPSocket.d.ts:2:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.

2 import TypedEmitter from 'typed-emitter';
                           ~~~~~~~~~~~~~~~

node_modules/@discordjs/voice/dist/networking/VoiceWebSocket.d.ts:2:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.

2 import TypedEmitter from 'typed-emitter';
                           ~~~~~~~~~~~~~~~

node_modules/@discordjs/voice/dist/receive/SSRCMap.d.ts:1:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.

1 import TypedEmitter from 'typed-emitter';
                           ~~~~~~~~~~~~~~~

node_modules/@discordjs/voice/dist/util/adapter.d.ts:1:91 - error TS2307: Cannot find module 'discord-api-types/v8/gateway' or its corresponding type declarations.

1 import { GatewayVoiceServerUpdateDispatchData, GatewayVoiceStateUpdateDispatchData } from 'discord-api-types/v8/gateway';
                                                                                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/@discordjs/voice/dist/VoiceConnection.d.ts:8:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.

8 import TypedEmitter from 'typed-emitter';

When configuring tsc to ignore the errors, the library does work like normal.

Further details:

• @discordjs/voice version: 0.2.0
• Node.js version: 14.17.0
• Operating system: Windows 10 Pro Build 19042
• Priority this issue should have: Not entirely sure, as this could just be an edge case as I'm not using Discord.js

Relevant client options:

Using discord-rose at master.

When running npm pack --dry to see what files will be included in the final tar, it includes various useless assets such as the .github folder, examples folder, tsconfig.json, etc.

This can be solved by declaring a files array in the package.json to only include necessary files.

Someone in the Discord server posted this trace when trying to use the library with tweetnacl:

2021-06-09T17:27:40.526365+00:00 app[worker.1]: /app/node_modules/@discordjs/voice/dist/networking/Networking.js:413
2021-06-09T17:27:40.526401+00:00 app[worker.1]:                 secretbox.methods.close(opusPacket, connectionData.nonceBuffer, secretKey),
2021-06-09T17:27:40.526402+00:00 app[worker.1]:                                   ^
2021-06-09T17:27:40.526402+00:00 app[worker.1]: 
2021-06-09T17:27:40.526403+00:00 app[worker.1]: TypeError: secretbox.methods.close is not a function
2021-06-09T17:27:40.526404+00:00 app[worker.1]:     at Networking.encryptOpusPacket (/app/node_modules/@discordjs/voice/dist/networking/Networking.js:413:35)
2021-06-09T17:27:40.526404+00:00 app[worker.1]:     at Networking.createAudioPacket (/app/node_modules/@discordjs/voice/dist/networking/Networking.js:397:53)
2021-06-09T17:27:40.526404+00:00 app[worker.1]:     at Networking.prepareAudioPacket (/app/node_modules/@discordjs/voice/dist/networking/Networking.js:321:37)
2021-06-09T17:27:40.526405+00:00 app[worker.1]:     at VoiceConnection.prepareAudioPacket (/app/node_modules/@discordjs/voice/dist/VoiceConnection.js:282:33)
2021-06-09T17:27:40.526405+00:00 app[worker.1]:     at /app/node_modules/@discordjs/voice/dist/audio/AudioPlayer.js:419:54
2021-06-09T17:27:40.526405+00:00 app[worker.1]:     at Array.forEach (<anonymous>)
2021-06-09T17:27:40.526406+00:00 app[worker.1]:     at AudioPlayer._preparePacket (/app/node_modules/@discordjs/voice/dist/audio/AudioPlayer.js:419:19)
2021-06-09T17:27:40.526406+00:00 app[worker.1]:     at AudioPlayer._stepPrepare (/app/node_modules/@discordjs/voice/dist/audio/AudioPlayer.js:391:22)
2021-06-09T17:27:40.526406+00:00 app[worker.1]:     at prepareNextAudioFrame (/app/node_modules/@discordjs/voice/dist/DataStore.js:71:31)
2021-06-09T17:27:40.526407+00:00 app[worker.1]:     at audioCycleStep (/app/node_modules/@discordjs/voice/dist/DataStore.js:56:5)

This was fixed after they installed libsodium-wrappers.

When an audio player is unable to take 5 consecutive audio packets from a stream, the stream is killed. This limit should be a configurable behaviour.

If the producer of a voice adapter realises that the voice adapter can no longer be used (e.g. shard has been disconnected), it is currently unable to relay this information to @discordjs/voice.

In this situation, the producer should be able to signal to the library that the adapter can no longer be used, and the library should destroy the voice connection. When it is possible to recreate the voice adapter, the user can then recreate the voice connection.

Because peer dependencies are automatically installed in npm v7, it doesn't make sense to list all the audio dependencies in there as they will all be installed when only one from each category is needed at most.

Instead, a notice should be put in the README/documentation that states the supported versions of the dependencies, and informs the user that they need to manually install the dependencies.

@kyranet raised a good point in https://github.com/discordjs/guide/pull/595/files#r573217744. The library should prevent users from playing the same resource across multiple players. This could be achieved by having the resource "locked" while being played by an audio player, and then attempting to play a locked resource should throw.

For usability, it would be nice if we could have default values for createAudioResource options.

These default values could be:

• inputType: StreamType.Arbitrary
• name: unset
• inlineVolume: false

This would allow createAudioResource('file.mp3') instead of createAudioResource('file.mp3', { inputType: StreamType.Arbitrary })

It would be nice if discord can automatically end the voice call if there is only participant in the group call(which had 2 or more participants obviously) for more than 2 minutes.

Thank you so much

Currently, many of the classes are imported from their files, which isn't ideal.

Add an information in the player about time streamed.
such as <Player>.streamTime

If you want to play an audio resource on an audio player, you need to wait until the stream becomes readable:

const player = createAudioPlayer();
const audioResource = createAudioResource(...);

audioResource.once('readable', () => player.play(audioResource));

Without this check, the audio player could attempt to read the stream before it is ready and eventually decide the stream is unplayable before destroying it.

Additionally, the above example doesn't even handle the case where a stream might never be readable.

This isn't really a user-friendly experience. Instead, it would be nicer to add an additional state to AudioPlayer, e.g. Preparing or Loading, that transitions into Playing once the stream becomes readable. The user can then decide how long they want to allow the player to stay in the loading state before giving up and cancelling.

#33 implemented a link to the Contributing Guide in the README (https://github.com/discordjs/voice/blame/main/README.md#L35), though CONTRIBUTING.md does not exist.

cc: @iCrawl

I assumed that Readable.readable was synonymous with the readable event, which is emitted when there is data to be read from the stream. Instead, Readable.readable actually means that the stream has not been destroyed or emitted 'error' or 'end'.

AudioPlayer should be updated to check for this properly.

It would be useful to include the following debug information:

• Inbound/outbound WebSocket messages
• State transitions for VoiceConnection, Networking, and AudioPlayer
• "Doctor" feature that gives debug information for available dependencies

Currently, you can only pass a string, name, as metadata that can be attached to a track. This should be replaced with metadata, that allows a user to pass whatever they want in this place.

To stay consistent with the rest of the discord.js organisation, American English should be used instead of British English.

For example, audio resources follow behaviour as opposed to behavior.

During listening, playback is abruptly interrupted with an error.

Error:

AudioPlayerError: aborted
    at connResetException (node:internal/errors:683:14)
    at TLSSocket.socketCloseListener (node:_http_client:407:19)
    at TLSSocket.emit (node:events:377:35)
    at node:net:661:12
    at TCP.done (node:_tls_wrap:577:7) {
  resource: AudioResource {
    playbackDuration: 93900,
    started: true,
    edges: [ [Object], [Object], [Object] ],
    playStream: Encoder {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 5,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      encoder: null,
      _options: [Object],
      _required: 3840,
      _buffer: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: [Function: bound onwrite]
    },
    metadata: Track {
      url: 'youtube url',
      title: 'video title',
      onStart: [Function: onStart],
      onFinish: [Function: onFinish],
      onError: [Function: onError]
    },
    volume: VolumeTransformer {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 5,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      _readInt: [Function (anonymous)],
      _writeInt: [Function (anonymous)],
      _bits: 16,
      _bytes: 2,
      _extremum: 32768,
      volume: 0.5,
      _chunk: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: [Function: bound onwrite]
    },
    encoder: Encoder {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 5,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      encoder: null,
      _options: [Object],
      _required: 3840,
      _buffer: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: [Function: bound onwrite]
    },
    audioPlayer: AudioPlayer {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      subscribers: [Array],
      _state: [Object],
      behaviors: [Object],
      debug: [Function (anonymous)],
      [Symbol(kCapture)]: false
    }
  }
}

--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.3.1
- prism-media: 1.2.9

Opus Libraries
- @discordjs/opus: 0.5.3
- opusscript: not found

Encryption Libraries
- sodium: 3.0.2
- libsodium-wrappers: not found
- tweetnacl: not found

FFmpeg
- version: 4.2.4-1ubuntu0.1
- libopus: yes
--------------------------------------------------

• @discordjs/voice version: 0.3.1
• Node.js version: 13.0.0-dev.41673b738232f64da2ded3b15be0f798135ae351
• Operating system: Ubuntu 20.04.2 LTS

Relevant client options:

• partials: none
• gateway intents: GUILDS, GUILD_MESSAGES, GUILD_VOICE_STATES

These are the scenarios for a stream finishing (defined as there being no data left to read from the stream):

1. The stream has come to a graceful end (either close, end or finish are emitted). This is already handled by the library. Since no errors are emitted, eventually, the AudioPlayer#_step() method will find that the stream is no longer readable and transition to an idle state.
2. The stream has emitted an error. This is currently not handled by the library.

Handling an error should look something like this:

• The error should be caught and propagated via an error event
• If after the error has been emitted, the resource is still being played, then the player should transition to the Idle state. This allows the user to play a new stream in its place without transitioning to the Idle state. Transitioning to the Idle state and then emitting an error event would be problematic.

Additionally, this condition is used to check whether to terminate the _step method: state.resource.playStream.readableEnded || state.resource.playStream.destroyed. This could really just be replaced by !state.resource.playStream.readable which is also more readable.

Instead of listening for the raw event from a Discord.js Client, the hook should listen for the VOICE_SERVER_UPDATE and VOICE_STATE_UPDATE event forwarded by client.ws as this reduces overhead.

As spotted by @almostSouji, discord/discord-api-docs#1741 shows that the endpoint in a VOICE_SERVER_UPDATE packet can sometimes be null.

If a VoiceConnection is destroyed (i.e. cannot be reused), then it should be unsubscribed from any audio player it is connected to. This will prevent a memory leak where a destroyed stream will still be kept by an audio player unless manually removed.

Continuing on from discordjs/discord.js#3305, the transcoding performance can be improved. Currently, for arbitrary inputs with no inline volume transformer, the following pipeline is used:

Input -> FFmpeg transcoder -> S16LE PCM -> Opus Encoder -> Opus Output

This proposed change would give the following pipeline which is less computationally expensive (as discussed in the PR, transcode time was reduced by around 10%):

Input -> FFmpeg transcoder -> Ogg Opus -> Ogg Opus Demuxer -> Opus Output

This pathway should be avoided for streams that use inline volume, as it would be more computationally expensive.

After a very rudimentary test, I found that 40 seconds of playback of an Ogg Opus audio stream had spent 105ms time blocking the thread with packet encryption using sodium on my system.

This could contribute to stream stutters at scale.

Offloading this task to worker threads can help to reduce this.

The last important piece of code to be tested is the Networking.ts file. Once this is done, we can start to move towards a v1.0.0 release 🚀

Should allow you to see the ping data of a connection - similar to Discord.js's WebSocketShard.

This should be implemented in the Networking class. If the user wants to access the ping stat, they should have to access it directly:

if (voiceConnection.state.status === VoiceConnectionStatus.Ready) {
  const ping = voiceConnection.state.networking.ping;
}

This is to stay in line with the rest of the library - aliases and getter utilities are reduced wherever possible.

So that the documentation can be parsed correctly by https://github.com/discordjs/docgen/, the documentation should be converted from TSDoc to JSDoc.

This documentation will serve to:

• Inform users of how they can start to migrate to this module
• The benefits of moving to this module
• Some features that will be missing from this module and explanations as to why
  • Audio seeking
  • Voice receive (:cry:)

Currently, @discordjs/voice has made improvements over Discord.js v11 and v12 voice by updating the audio dispatch cycle.

In v11 and v12, each audio dispatch cycle is 20ms long. At the start of the cycle, an Opus packet is read, encrypted, and dispatched. The reading (the underlying Opus encoder is thread-blocking) and encrypting of the packets can block the thread for up to a few milliseconds, which is actually quite significant for audio work. The time required for each of these two steps is also variable. For example, in one cycle, it may take 0ms to do these two things. In another, it may take 2ms.

This causes a problem where the audio packet is dispatched at times that aren't exactly 20ms. In the example above, there would be a 2ms gap between the first and second audio packets being dispatched, and this can be observed in the Discord client as a very short gap in transmission.

A fix was explored in @discordjs/voice which involves performing packet dispatch at the start of each audio cycle. This results in a much more consistent 20ms interval. However, this only works with one audio player - adding another audio player could recreate the situation above. One audio player may be scheduled to perform its next audio cycle at the same time the thread is being blocked by packet encoding/encryption of another player.

The ideal fix would of course be to make this blocking work non-blocking. However, an additional and more realistic fix for the time being is to instead have a single clock that drives each audio player.

At the start of each interval of the clock, it should dispatch audio packets from all the audio players, and then allow each audio player to do the thread-blocking work of preparing each audio packet. After this is done, the timeout should be refreshed so that the next cycle is ready to begin 20ms after the current cycle started.

This would allow for smoother playback when more than one audio player is used.

In the future, we should definitely explore moving the blocking work to worker threads. While #52 is open, I've also found that every 30 seconds of audio playback involves blocking the thread for around 1.8 seconds with just the Opus encoder - this is far from ideal!

If a user wants to create a resource and catch errors, they will have to do something like this:

const player = createAudioPlayer();
const resource = createAudioResource('input.mp3', options);

resource.playStream.on('error', console.error);

player.play(resource);

They would have to apply the error handler for every resource they create. Maybe it would be better for these errors to be propagated through the AudioPlayer when the resource is connected to it?

Current Scenario

The library takes the following approach to disconnects (i.e. the voice websocket closing):

If the close code is 4015 or a non-Discord close code (anything <4000), then a new websocket is opened and it will try to resume the session (see voice close code documentation)

If this fails, then the websocket will disconnect once again, and lead to the scenario below.

If the close code is 4014, the voice connection will enter a disconnected state. At the moment I am finding this problematic because of discord/discord-api-docs#2450.

A bot being moved into a different voice channel on the same guild can trigger a close with the code 4014 and it will reconnect automatically as new voice server and state packets are sent. However, the same close code is emitted for a bot being kicked from the guild or disconnected from the voice channel.

The issue here is that if the bot is moved to a voice channel, the disconnected event will be emitted only for the client to automatically reconnect itself a few moments later, whereas the latter two cases will stay disconnected unless the user intervenes.

This isn't really friendly for the user when they're trying to figure out whether their bot has actually disconnected from a guild's voice channels, or if they just moved channels.

The final case is for Discord codes that are not 4014 or 4015. For these codes, the bot will automatically try to rejoin the channel and will enter the Signalling state.

Documentation

There should be some effort in documenting this to make it as easy as possible for the user to handle disconnects properly. As well as documenting this, example code that addresses the above issues would be helpful too.

It isn't really clear how a VoiceReceiver will behave once a connection is disconnected/destroyed. This should be thought about and cleared up. As of now, nothing happens and streams are not destroyed in this instance.

A fairly simple starter example should be created to show users the basics of using this library:

• Joining a voice channel
• Playing a resource
• Avoiding indefinitely waiting for events to occur

Currently, the biggest thread-blocker in playing audio is Opus encryption using @discordjs/opus or opusscript. Because the transformer pipeline is fairly independent from the rest of the library, it would be possible to offload this work to a worker thread so that the main process doesn't get blocked.

This would help with reducing jitter/stutter in streams.

When playing the audio from dropbox, the playback ends unexpectedly in the middle of the audio without throwing an error.
However, when playing the downloaded audio file, it will play to the end without any issue.

ends unexpectedly

  const connection = joinVoiceChannel({
    guildId: guild.id,
    channelId: vc.id,
    adapterCreator: guild.voiceAdapterCreator,
    selfMute: false,
    debug: true,
  });
  connection.on("debug",console.log);
  connection.on("error",console.error);
  const resource = createAudioResource("https://www.dropbox.com/s/5f1l6seztxvq373/Cartoon%20-%20On%20_%20On%20%28feat.%20Daniel%20Levi%29%20_NCS%20Release_.mp3?dl=1", {
    inputType: StreamType.Arbitrary,
  });
  const player = createAudioPlayer({
    behaviors: {
      noSubscriber: NoSubscriberBehavior.Pause,
    },
  });
  player.on("debug",console.log);
  player.on("error",console.error);
  player.play(resource);
  const promises = [];
  promises.push(entersState(connection, VoiceConnectionStatus.Ready, 1000 * 10));
  promises.push(entersState(player, AudioPlayerStatus.AutoPaused, 1000 * 10));
  await Promise.all(promises);
  connection.subscribe(player);
  await entersState(player, AudioPlayerStatus.Playing, 100);
  await entersState(player, AudioPlayerStatus.Idle, 2**31-1);
  connection.destroy();

play to the end

  const resource = createAudioResource(fs.createReadStream("./audio.mp3"), {
    inputType: StreamType.Arbitrary,
  });

Windows 10

--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.3.1
- prism-media: 1.2.9

Opus Libraries
- @discordjs/opus: 0.5.3
- opusscript: not found

Encryption Libraries
- sodium: not found
- libsodium-wrappers: not found
- tweetnacl: 1.0.3

FFmpeg
- version: 4.3.1-2020-11-19-essentials_build-www.gyan.dev
- libopus: yes
--------------------------------------------------

Ubuntu 20.04(WSL2)

--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.3.1
- prism-media: 1.2.9

Opus Libraries
- @discordjs/opus: 0.5.3
- opusscript: not found

Encryption Libraries
- sodium: not found
- libsodium-wrappers: not found
- tweetnacl: 1.0.3

FFmpeg
- version: 4.2.4-1ubuntu0.1
- libopus: yes
--------------------------------------------------

• @discordjs/voice version: 0.3.1
• Node.js version: 14.15.5(Windows 10),14.16.0(WSL2-Ubuntu20.04)

Relevant client options:

• partials: none
• gateway intents: NON_PRIVILEGED

play to the end

state change:
from {"status":"idle","resource":false,"stepTimeout":false}
to {"status":"buffering","resource":true,"stepTimeout":false}
state change:
from {"status":"buffering","resource":true,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
to {"status":"autopaused","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
[NW] [WS] >> {"op":0,"d":{"server_id":"750031320205230311","user_id":"735835853372522546","session_id":"6a2f514b36ee6e18b5693d2695231a30","token":"64e47b95c802a107"}}
[NW] state change:
from {"code":0,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":false}
to {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":false}
[NW] [WS] << {"op":8,"d":{"v":4,"heartbeat_interval":13750.0}}
[NW] [WS] << {"op":2,"d":{"streams":[{"type":"video","ssrc":56152,"rtx_ssrc":56153,"rid":"","quality":0,"active":false}],"ssrc":56151,"port":50008,"modes":["aead_aes256_gcm_rtpsize","aead_aes256_gcm","xsalsa20_poly1305_lite_rtpsize","xsalsa20_poly1305_lite","xsalsa20_poly1305_suffix","xsalsa20_poly1305"],"ip":"103.194.167.85","experiments":["bwe_conservative_link_estimate","bwe_remote_locus_client"]}}
[NW] state change:
from {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":false}
to {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
[NW] [WS] >> {"op":1,"d":{"protocol":"udp","data":{"address":"124.85.238.198","port":52993,"mode":"xsalsa20_poly1305_lite"}}}
[NW] state change:
from {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
to {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
[NW] [WS] << {"op":4,"d":{"video_codec":"H264","secret_key":[33,61,144,252,209,45,62,11,64,2,42,73,159,84,147,237,54,111,233,159,57,197,206,113,90,61,226,185,112,190,236,253],"mode":"xsalsa20_poly1305_lite","media_session_id":"38ee242ae2f8fe471d9f063b187675bd","audio_codec":"opus"}}
[NW] state change:
from {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
to {"code":4,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151,"encryptionMode":"xsalsa20_poly1305_lite","secretKey":{"0":33,"1":61,"2":144,"3":252,"4":209,"5":45,"6":62,"7":11,"8":64,"9":2,"10":42,"11":73,"12":159,"13":84,"14":147,"15":237,"16":54,"17":111,"18":233,"19":159,"20":57,"21":197,"22":206,"23":113,"24":90,"25":61,"26":226,"27":185,"28":112,"29":190,"30":236,"31":253},"sequence":48630,"timestamp":2216838939,"nonce":0,"nonceBuffer":{"type":"Buffer","data":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]},"speaking":false,"packetsPlayed":0}}
state change:
from {"status":"autopaused","missedFrames":0,"playbackDuration":100,"resource":true,"silencePacketsRemaining":0,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":100,"resource":true,"silencePacketsRemaining":0,"stepTimeout":false}
[NW] [WS] >> {"op":5,"d":{"speaking":1,"delay":0,"ssrc":56151}}
[NW] [WS] >> {"op":3,"d":1623501476079}
[NW] [WS] << {"op":6,"d":1623501476079}
[NW] [WS] >> {"op":3,"d":1623501489832}
[NW] [WS] << {"op":6,"d":1623501489832}
[NW] [WS] >> {"op":3,"d":1623501503589}
[NW] [WS] << {"op":6,"d":1623501503589}
[NW] [WS] >> {"op":3,"d":1623501517342}
[NW] [WS] << {"op":6,"d":1623501517342}
[NW] [WS] >> {"op":3,"d":1623501531100}
[NW] [WS] << {"op":6,"d":1623501531100}
[NW] [WS] >> {"op":3,"d":1623501544858}
[NW] [WS] << {"op":6,"d":1623501544858}
[NW] [WS] >> {"op":3,"d":1623501558620}
[NW] [WS] << {"op":6,"d":1623501558620}
[NW] [WS] >> {"op":3,"d":1623501572377}
[NW] [WS] << {"op":6,"d":1623501572377}
[NW] [WS] >> {"op":3,"d":1623501586135}
[NW] [WS] << {"op":6,"d":1623501586135}
[NW] [WS] >> {"op":3,"d":1623501599891}
[NW] [WS] << {"op":6,"d":1623501599891}
[NW] [WS] >> {"op":3,"d":1623501613646}
[NW] [WS] << {"op":6,"d":1623501613646}
[NW] [WS] >> {"op":3,"d":1623501627399}
[NW] [WS] << {"op":6,"d":1623501627399}
[NW] [WS] >> {"op":3,"d":1623501641152}
[NW] [WS] << {"op":6,"d":1623501641152}
[NW] [WS] >> {"op":3,"d":1623501654912}
[NW] [WS] << {"op":6,"d":1623501654912}
[NW] [WS] >> {"op":3,"d":1623501668669}
[NW] [WS] << {"op":6,"d":1623501668669}
[NW] [WS] >> {"op":5,"d":{"speaking":0,"delay":0,"ssrc":56151}}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":208120,"resource":true,"silencePacketsRemaining":0,"stepTimeout":false}
to {"status":"idle","resource":false,"stepTimeout":false}

ends unexpectedly

state change:
from {"status":"idle","resource":false,"stepTimeout":false}
to {"status":"buffering","resource":true,"stepTimeout":false}
[NW] [WS] >> {"op":0,"d":{"server_id":"750031320205230311","user_id":"735835853372522546","session_id":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","token":"8033b12c62e8ac02"}}
[NW] state change:
from {"code":0,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":false}
to {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":false}
[NW] [WS] << {"op":8,"d":{"v":4,"heartbeat_interval":13750.0}}
[NW] [WS] << {"op":2,"d":{"streams":[{"type":"video","ssrc":56325,"rtx_ssrc":56326,"rid":"","quality":0,"active":false}],"ssrc":56324,"port":50008,"modes":["aead_aes256_gcm_rtpsize","aead_aes256_gcm","xsalsa20_poly1305_lite_rtpsize","xsalsa20_poly1305_lite","xsalsa20_poly1305_suffix","xsalsa20_poly1305"],"ip":"103.194.167.85","experiments":["bwe_conservative_link_estimate","bwe_remote_locus_client"]}}
[NW] state change:
from {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":false}
to {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
[NW] [WS] >> {"op":1,"d":{"protocol":"udp","data":{"address":"124.85.238.198","port":50595,"mode":"xsalsa20_poly1305_lite"}}}
[NW] state change:
from {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
to {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
[NW] [WS] << {"op":4,"d":{"video_codec":"H264","secret_key":[79,99,160,224,227,8,92,61,7,169,69,57,136,222,196,243,197,72,173,125,243,208,3,252,35,68,147,254,167,60,28,236],"mode":"xsalsa20_poly1305_lite","media_session_id":"38ee242ae2f8fe471d9f063b187675bd","audio_codec":"opus"}}
[NW] state change:
from {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
to {"code":4,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324,"encryptionMode":"xsalsa20_poly1305_lite","secretKey":{"0":79,"1":99,"2":160,"3":224,"4":227,"5":8,"6":92,"7":61,"8":7,"9":169,"10":69,"11":57,"12":136,"13":222,"14":196,"15":243,"16":197,"17":72,"18":173,"19":125,"20":243,"21":208,"22":3,"23":252,"24":35,"25":68,"26":147,"27":254,"28":167,"29":60,"30":28,"31":236},"sequence":21329,"timestamp":2610706401,"nonce":0,"nonceBuffer":{"type":"Buffer","data":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]},"speaking":false,"packetsPlayed":0}}
state change:
from {"status":"buffering","resource":true,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
to {"status":"autopaused","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
state change:
from {"status":"autopaused","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
[NW] [WS] >> {"op":5,"d":{"speaking":1,"delay":0,"ssrc":56324}}
[NW] [WS] >> {"op":3,"d":1623502100140}
[NW] [WS] << {"op":6,"d":1623502100140}
[NW] [WS] >> {"op":3,"d":1623502113901}
[NW] [WS] << {"op":6,"d":1623502113901}
[NW] [WS] >> {"op":3,"d":1623502127659}
[NW] [WS] << {"op":6,"d":1623502127659}
[NW] [WS] >> {"op":5,"d":{"speaking":0,"delay":0,"ssrc":56324}}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":48520,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
to {"status":"idle","resource":false,"stepTimeout":false}

It would be nice to have a comprehensive example showing how to use voice receive.

It would be nice to have the following utilities:

• Detection for when a user stops speaking
• Attaching RTP header data to the buffers in AudioReceiveStream to help make sense of the data (e.g. to aid with jitter buffering or packet reordering)

Currently, createAudioPlayer([options]) takes optional options, but if you specify options, then you need to specify all of the possible options.

The function signature should be updated to allow passing in partial options.

The library should implement existing voice receive logic from discord.js even though the this is officially undocumented.

• Allow users to set selfDeaf to false when joining a voice channel (to receive packets)
• Expose messages from UDP and WS
• Have a "keep alive" option that periodically sends silence packets to a given voice channel to continue receiving audio
• Readable Opus streams for a given user id or ssrc

The library could easily infer some stream types, e.g. a stream would be input type OggOpus if stream instanceof prism.opus.OggDemuxer held. This would save from having to specify streamType so explicitly.

Couldn't think of a good title

When using the example code as written, from my experience it would not play without waiting for the resource to be ready.

Original Code

player.play(resource)

Modified line

resource.playStream.once(`readable`, () => player.play(resource));

I would submit a PR to modify the example, however I'm unsure if this is an edge case due to some unknown cause.

Further details:

• @discordjs/voice version: 0.2.0
• Node.js version: 14.17.0
• Operating system: Windows 10 Pro Build 19042
• Priority this issue should have: Unsure, as this could be an edge case.

Relevant client options:

Using discord-rose at master.

Currently, the NoSubscriberBehaviour exists but it cannot be set by the user and is currently fixed to one mode.

This should be updated to allow the user to pick the behaviour that they'd like.

If you attempt to play a resource that has ended or been destroyed, the audio player will transition to the Playing state, but then immediately to the Idle state since the resource can't actually be played.

It would be better if it just stayed in the state it's currently in if the resource isn't playable, and instead an error should be thrown.

Consider adding eslint-plugin-tsdoc to a linting workflow to automate discovery of such mistakes.