Different TTS models about call-gpt HOT 5 CLOSED

@teledoc2 do you know what audio format you are outputting to the stream? Twilio only accepts ulaw_8000 since this is what is supported by the publicly switched telephone network. You can see where we set this with ElevenLabs: https://github.com/twilio-labs/call-gpt/blob/main/services/tts-service.js#L20

from call-gpt.

thank you so much for the prompt response, yes both neets.ai and deepgram tts can produce output format ulaw_8000 (mulaw_8000 in neets.ai) and i added it as a param in the request, per docs, but still not able to pass the audio to twilio. When using base64 there is just a loud static and without base64 just silent and in both cases the terminal is showing no activity. Any advice would be appreciated. I just looked at openAI TTS and it seems to be a feasible solution so will try it next. Any other ideas would be greatly appreciated. Thanks so much! Abe

from call-gpt.

@teledoc2 Have you managed to successfully implement it with openAI TTS? I have tried but only hear static on my end. If you did, could you please share your solution?

from call-gpt.

Neets was fairly easy to implement as they support output in mulaw, worked without any issues. To get openAI TTS to work you would probably need to use something like wavefile to convert it from wav to mulaw.

      const response = await fetch(
        'https://api.neets.ai/v1/tts',
        {
          method: 'POST',
          headers: {
            'Content-Type': 'application/json',
            'X-API-Key': 'xxxxxx',
            accept: 'audio/wav',
          },
          body: JSON.stringify({
            text: partialResponse,
            voice_id: 'vits-eng-1',
            fmt: 'mulaw',
            params: {
              model: 'vits'
            }
          })
        }
      )

from call-gpt.

@c0dr Thanks! Your code works well for Neets ai API.

Do you know how we can add fillers like 'Umm', 'Sure' or 'Let me think'? I'm planning to add fillers to reduce the perception of latency between the 'human voice input' and the 'tts of gpt response'.

from call-gpt.