I’ve been made aware that the code I’ve written to access text-to-speech from ElevenLabs API is no longer working.

I’ve tested it and it seems that the CORS-Proxy that is being used in ai-character-chat currently doens’t allow POST methods (which is being used to ‘post’ the text to be ‘spoken’ in ElevenLabs).

Not a major/priority issue but might be nice to be fixed. I also wonder how many are using text to speech (even just using the Speech Synthesis code) in the ai-character-chat…

  • @perchanceMEnglish
    2•7 months ago

    Okay, thanks for the example, I think it’s all fixed now. Hopefully I didn’t break anything. Been up for two days straight tho so I wouldn’t bet on it, but I did some basic tests and it seems good. Will check lemmy messages first thing tomorrow 🫡

    • alloEnglish
      2•7 months ago

      haha. one and a half days for me and maybe 30+ hours straight just now on something with your wonderful comments update :) can’t wait to share it! No sleeping yet!

      • BluePowerEnglish
        2•7 months ago

        Same 😄

        I’ll complete and share the project possibly (and hopefully) as an actual plugin after the post-announcement update of my generator hub page, but I’ll be releasing the “early implementations” somewhere in my experiment generator so everyone can try it right now and give some feedback on it.

    • VioneTOPMEnglish
      1•7 months ago

      It seems to work now, though there are some inconsistencies with the chunk text arrangement, which causes the text in the stream to be quite jumbled. I’m looking into it now, I’ll update if it is still inconsistent with the order.

      EDIT: It is inconsistent with the order of the chunks. Maybe there is a way to parse it in order? Currently I’m having it pushed into an array, then sort that array by the index, then join the sorted array to be a string before pushing it to the t2s, though it is still inconsistent and sometimes the streaming finishes and the text to be spoken is not yet queued up.

      Here are the code hacks to re-sort the chunks to order lmao.

      • @perchanceMEnglish
        1•7 months ago

        I wasn’t able to reproduce this when trying it with this code:

        oc.thread.on("StreamingMessage", async function (data) {
  let lastChunkI = -1;
  let chunksText = "";
  let chunks = [];
  for await (let chunk of data.chunks) {
    chunks.push(chunk);
    chunksText += chunk.text;
    if(chunk.index !== lastChunkI+1) console.warn("OUT OF ORDER CHUNKS!", chunks);
    lastChunkI++;
  }
  console.log("chunks:", chunks);
  console.log("chunksText:", chunksText);
});

        Or have I misunderstood the problem?

        • VioneTOPMEnglish
          1•7 months ago

          You have. On trying your code. it gave me the OUT OF ORDER CHUNKS!:



          Here’s the character I used with the custom code to check the out of order chunks.: Link to Character

          • @perchanceMEnglish
            2•7 months ago

            Ahh, thank you! Was very confused at first because I iteratively made your character closer to the default Assistant to work out why it was happening in yours but not the assistant, and found that the profile pic was the cause lmao, but eventually realised that it was because data url vs normal URL, and the data url (being larger) was making an async IndexedDB request take a few milliseconds longer, which caused the out-of-order-ness. But I shouldn’t have even been doing those db requests in the first place, so I’ve removed them, and this race condition type bug shouldn’t be possible at all now. Thanks again!!

            • VioneTOPMEnglish
              2•7 months ago

              Niiceeeeeeeeeee it is good now. Thanks again!