mmattdonk / ai-tts-donations Goto Github PK

And whitelisted voices

_{From SyncLinear.com | SOL-3}

I wouldn't think of it as a priority but it would definitely make it futureproof

Only allow x voice and x voice. opposite of blacklist

Linear

This happens because we are not using multithreading at the moment, I tried to get multithreading to work, however it just didn't work. So if you know how to get multithreading working, I'd appreciate a PR :)

kinda like setup.py from TwitchTunes, ooh maybe a website to generate a config.json and .env?

Instead of having the uberduck model name, it could be handy to have a way to have custom prefixes for voices, for example:
narrator=fate-ur-ts-tck-narrator this way it would recognize narrator: message as fate-ur-ts-tck-narrator: message

Sentry Issue: AI-TTS-DONATIONS-3

ReadTimeout: The read operation timed out
(5 additional frame(s) were not displayed)
...
  File "httpcore\_sync\http11.py", line 189, in _receive_response
    event = self._receive_event(timeout)
  File "httpcore\_sync\http11.py", line 225, in _receive_event
    data = self.socket.read(self.READ_NUM_BYTES, timeout)
  File "httpcore\_backends\sync.py", line 61, in read
    return self.sock.recv(n)
  File "contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "httpcore\_exceptions.py", line 12, in map_exceptions
    raise to_exc(exc) from None

ReadTimeout: The read operation timed out
(9 additional frame(s) were not displayed)
...
  File "httpx\_client.py", line 942, in _send_handling_redirects
    response = self._send_single_request(request)
  File "httpx\_client.py", line 978, in _send_single_request
    response = transport.handle_request(request)
  File "httpx\_transports\default.py", line 177, in handle_request
    status_code, headers, byte_stream, extensions = self._pool.handle_request(
  File "contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "httpx\_transports\default.py", line 79, in map_httpcore_exceptions
    raise mapped_exc(message) from exc

From 3 functions into just 1, don't want that much copy paste :)

Do this after #34

like blacklisting, except it just replaces the word instead of blocking the tts

serve the audio via an overlay, maybe a new repo for this? it would be really niche.

its really meant for the IT nerds for streamers to be able to set it up by themselves and have the streamer just input 1 overlay into their OBS.

This is a known issue, and is being fixed in #34.

probably drake or kanye west lol

listen_channel_subscriptions

if it voice starts with FY: then use this, or if the uberduck fails.

this will require a new function in the Fakeyou.py API wrapper.

so:
Uberduck (or FakeYou if it's a direct model ID or if the voice starts with FY:) > fakeyou > fallback voice

This is, safe to say, awful.

A stream overlay for the TTS being processed, something small that the streamer can put (browser source running on a local server). Would help the cheerer confirm that their voice is actually working

Example (see the top left):

Describe the bug
please note maybe I'm doing something wrong but taking the example of {'sonic': 'TM:z3mvtbcg4k4z'} crashes the bot. and trying to use my own alias results in the same issues {'sarah':'starlight-glimmer'}

Maybe I'm missing some syntax but I brought it straight from the example. however, the example seems confusing too and may need higher contrast on the text as well, as certain colored monitors will not display the text clearly. Looking at it, it is unclear what 'TM:z3mvtbcg4k4z' refers to, as there doesn't seem to be a voice under that?

i have a feeling this is why sometimes it will disconnect and not reconnect. also sometimes 1 thing goes wrong on the bot and the whole bot crashes requiring a restart, i feel the script being asynchronous will fix this.

regex:

\((.*?)\)

https://regex101.com/r/TtMoEy/1

Show the logs from bits or rewards with a button that allows the streamer to replay the donation or to remove/skip it

I think it would be a good feature to make it work without having to add || between messages. I know nothing about regex or coding but I would make it stop when it sees a new voice text: or a sound (1).

I don't know if I should make another feature request for this other suggestions since they go in hand with this syntax feature but I would make it so if you make a mistake it would go back to the previous voice, for example:
drake: hello i am drake (1) spngbob: i am still drake
it would read it all with Drake's voice as "hello i am drake" (sound effect) "spngbob: i am still drake".
(even the spngbob part)
This way you stop the bot from stopping completely when a user sends a wrong voice id.

I would also add a default non ai voice (or a default ai voice / a way to pick a default ai voice) to read the first segment if it's wrong since the beginning, that way you can always get tts output even if the person sending the message is a pepega

For example: if the person forgets to add a starting voice or makes a spelling mistake on the first voice id, it would read it as Brian until it sees a valid voice id.

Could be fun to have custom in sync images to voices like this one https://fakeyou.com/w2l/WT:7k5d07dcz6yk3qp2pkj3zsg4jpq71 that played depending on the voice that is playing

Sentry Issue: AI-TTS-DONATIONS-2

ZeroDivisionError: forsen
(4 additional frame(s) were not displayed)
...
  File "debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "bot.py", line 77, in <module>
    raise ZeroDivisionError("forsen")

Every 30 seconds Engine.IO connection drops and then reconnects.

This is with StreamElements selected in the config.json

This does not happen with Channel_Points in the config.json

21:49:50 ERROR engineio.client - packet queue is empty, aborting client.py:663
INFO root - Engine.IO connection dropped client.py:718
INFO root - Connection failed, new attempt in 0.95 seconds client.py:652
21:49:51 INFO root - Engine.IO connection established client.py:681
INFO root - Namespace / is connected client.py:562
INFO root - Emitting event "authenticate" [/] client.py:395
INFO root - Reconnection successful client.py:669
INFO root - Received event "authenticated" [/] client.py:580
INFO root - StreamElements connected! bot.py:563

Codacy detected an issue:

Message: No blank lines allowed after function docstring (found 1) (D202)

Currently on:

• Commit: c5f5567
• File: bot.py
• LineNum: 149

Buttons for:

• Skipping TTS

Sliders/Text fields for:

• Minimum Bits Amount
• Maximum Message Length

Text fields for:

• Configuring the bot for the first time (all the secret keys in .env)

makes it so that it just polls an external api (one made by me) so that the viewer doesn't need any credentials. barebones ai-tts-donations with only the testing interface

see #179 description

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• chore(deps): update dependency @tsconfig/docusaurus to v2.0.2
• chore(deps): update dependency @types/react-typist to v2.0.6
• chore(deps): update dependency autoprefixer to v10.4.17
• chore(deps): update dependency nodemon to v2.0.22
• chore(deps): update dependency postcss to v8.4.33
• chore(deps): update dependency pytest to v7.4.4
• chore(deps): update nextjs monorepo to v13.5.6 (eslint-config-next, next)
• chore(deps): update pnpm to v7.33.6
• fix(deps): update dependency @next-auth/prisma-adapter to v1.0.7
• fix(deps): update dependency node-fetch to v3.3.2
• fix(deps): update dependency redis to v4.6.12
• fix(deps): update docusaurus monorepo to v2.4.3 (@docusaurus/core, @docusaurus/module-type-aliases, @docusaurus/preset-classic, @docusaurus/types)
• fix(deps): update mantine monorepo to v6.0.21 (@mantine/carousel, @mantine/core, @mantine/form, @mantine/hooks, @mantine/next, @mantine/notifications, @mantine/nprogress)
• chore(deps): update dependency @prisma/nextjs-monorepo-workaround-plugin to v4.16.2
• chore(deps): update dependency @types/node to v18.19.8
• chore(deps): update dependency @types/node to v18.19.8
• chore(deps): update dependency @types/node to v20.11.5
• chore(deps): update dependency eslint to v8.56.0
• chore(deps): update dependency google-cloud-storage to v2.14.0
• chore(deps): update dependency prettier to v2.8.8
• chore(deps): update dependency rich to v13.7.0
• chore(deps): update dependency tailwindcss to v3.4.1
• chore(deps): update dependency tslib to v2.6.2
• chore(deps): update dependency typescript to v5.3.3
• chore(deps): update dependency typescript to v5.3.3
• chore(deps): update dependency vercel to v31.4.0
• chore(deps): update dependency wheel to v0.42.0
• fix(deps): update dependency @sentry/nextjs to v7.93.0
• fix(deps): update dependency @tabler/icons-react to v2.46.0
• fix(deps): update dependency @tanstack/react-query to v4.36.1
• fix(deps): update dependency @vercel/analytics to v1.1.2
• fix(deps): update dependency dotenv to v16.3.1
• fix(deps): update dependency embla-carousel-autoplay to v7.1.0
• fix(deps): update dependency embla-carousel-react to v7.1.0
• fix(deps): update dependency fastify to v4.25.2
• fix(deps): update dependency fastify-raw-body to v4.3.0
• fix(deps): update dependency framer-motion to v10.18.0
• fix(deps): update dependency next-intl to v2.22.1
• fix(deps): update dependency pusher to v5.2.0
• fix(deps): update dependency react-tsparticles to v2.12.2
• fix(deps): update dependency superjson to v1.13.3
• fix(deps): update dependency tabler-icons-react to v1.56.0
• fix(deps): update dependency tsparticles to v2.12.0
• fix(deps): update dependency turbo to v1.11.3
• fix(deps): update dependency zod to v3.22.4
• fix(deps): update emotion monorepo (@emotion/react, @emotion/server)
• fix(deps): update trpc monorepo to v10.45.0 (@trpc/client, @trpc/next, @trpc/react-query, @trpc/server)
• chore(deps): update dependency @prisma/nextjs-monorepo-workaround-plugin to v5
• chore(deps): update dependency @types/node to v20
• chore(deps): update dependency vercel to v33
• chore(deps): update nextjs monorepo to v14 (major) (eslint-config-next, next)
• fix(deps): update dependency @mdx-js/react to v3
• fix(deps): update dependency @tanstack/react-query to v5
• fix(deps): update dependency next-intl to v3
• fix(deps): update dependency superjson to v2
• fix(deps): update dependency tsparticles to v3
• fix(deps): update docusaurus monorepo to v3 (major) (@docusaurus/core, @docusaurus/module-type-aliases, @docusaurus/preset-classic, @docusaurus/types)
• fix(deps): update mantine monorepo to v7 (major) (@mantine/carousel, @mantine/core, @mantine/form, @mantine/hooks, @mantine/notifications, @mantine/nprogress)
• 🔐 Create all rate-limited PRs at once 🔐

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

• fix(deps): update dependency next-auth to v4.24.5 [security]

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update react monorepo (@types/react, @types/react-dom)
• chore(deps): update dependency eslint to v8.56.0
• chore(deps): update dependency pedalboard to v0.8.7
• chore(deps): update dependency sentry-sdk to v1.39.2
• chore(deps): update dependency nodemon to v3
• chore(deps): update dependency prettier to v3
• chore(deps): update pnpm to v8
• fix(deps): update dependency clsx to v2
• fix(deps): update dependency prism-react-renderer to v2
• fix(deps): update dependency pusher-js to v8
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

To avoid donos like (1) || (2) || (3) || (4) || (5) || (6) || (7) || (8) || (9) || (10) || (11) || (12) || (13) || (14) || (15) || (16) ...

"solrock" = random pokemon name = not relevant/searchable = bad

_{From SyncLinear.com | SOL-11}

At the moment the project spins around being able to use the huge amount of uberduck.ai voices, but it would be really useful to be able to self host our own tacotron2 voices for a faster synthesis

Sentry Issue: AI-TTS-DONATIONS-1Q

KeyError: 'state'
  File "threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "bot.py", line 679, in test_tts
    request_tts(message=message, failed=False)
  File "bot.py", line 372, in request_tts
    check_tts_response: dict = tts_provider.check_tts(job_response["uuid"])
  File "~\ai-tts-donations\API\fakeyou.py", line 74, in check_tts
    if response.json()["state"]["status"] == "attempt_failed":

I have a feeling this is an uberduck problem, but its still a big problem.

The only case I've seen this happen to is elpws (so far):
Windows 10 | Python 3.9.7

The streamer can type Cheer1 gaben: open cases and it would simulate a donation. Would be useful for testing and debugging and also incase the streamer restarts right as someone donates.

Sentry Issue: AI-TTS-DONATIONS-4

TypeError: An asyncio.Future, a coroutine or an awaitable is required
  File "bot.py", line 731, in <module>
    asyncio.run(main_loop())
  File "nest_asyncio.py", line 33, in run
    task = asyncio.ensure_future(main)
  File "asyncio\tasks.py", line 677, in ensure_future
    raise TypeError('An asyncio.Future, a coroutine or an awaitable is '

I should add a common errors part in the readme for common errors streamers might encounter

Remove stuff like pre-commit and pyupgrade from the regular requirements.txt, and add them in a seperate dev-dependencies file for people who want to develop on the app.

maybe do multiple API requests at the same time, find the length of each voice and wait that much time before playing the next sound ApuThink