Stop Audio Stream on Twilio WS - Python

Question

I want to stop the audio that has been sent to Twilio. I have a code that streams audio response from Eleven labs to Twilio WS call. here is how I start the call:

@application.post('/call')

async def handle_incoming_calls(request: Request, From: Annotated[str, Form()]): response = VoiceResponse()

connect = Connect()
URL = f"wss://{PUBLIC_URL}/stream"
STATUS_URL = f"https://{PUBLIC_URL}/status-call"
connect.stream(url=URL,status_callback=STATUS_URL,status_callback_method='POST',name=str("<name>"))
response.append(connect)

return Response(content=str(response), media_type='text/xml')

MORE CODE PROCESSING THE MESSAGE, GENERATES AI RESPONSE TO USER INPUT AND THEN I STREAM THE ELEVEN LABS VOICE TO TWILIO CALL LIKE THIS:

async def stream(audio_stream, twilio_ws, stream_sid,call_sid):
global send_stream_task

print(f"\n\nNew streamSID: {stream_sid}\n\n")
async for chunk in audio_stream:
    if chunk:         
        audio = chunk
        
        b64_audio = base64.b64encode(audio).decode('utf-8')
        
        message = json.dumps({'event': 'media', 'streamSid': stream_sid,
                              'media': {'payload': b64_audio, }})
        send_stream_task[call_sid] = asyncio.create_task(twilio_ws.send_text(message))
        await send_stream_task[call_sid]

Even if I stop the stream function, the data it already sent is being played in the Twilio call I'm doing the following to stop the function:

listen_task = {} # init listen task with dict

send_to_tts = {} #init sent tts task with dict send_stream_task = {} # init send stream task with dict

async def cancel_tasks(call_sid,stream_sid): global listen_task,send_to_tts,send_stream_task

if call_sid in send_stream_task:
    if send_stream_task[call_sid].cancel():
        print("\n\nsend stream cancelled correctly\n\n")

if call_sid in listen_task:
    if listen_task[call_sid].cancel():
        print("Stream Sid: ",stream_sid)
        print("\n\nlisten canelled correctly\n\n")
        #stop_media_stream(call_sid, stream_sid)

if call_sid in send_to_tts:
    if send_to_tts[call_sid].cancel():
        #del listen_task[call_sid]
        print("\n\nsend to TTS cancelled correctly\n\n") 

I run this when I get new user input, and it stops the current running functions. But the data that I already sent to the call is still being played. I tried using the following function to stop the stream, but I only got a 404 error:

def stop_media_stream(call_sid, stream_sid):
account_sid = os.environ['TWILIO_ACCOUNT_SID']
auth_token = os.environ['TWILIO_AUTH_TOKEN']
client = Client(account_sid, auth_token)

stream = client.calls(str(call_sid)) \
            .streams(str(stream_sid)) \
            .update(status='stopped')

Error MSG:

 HTTP Error Your request was:
POST /Accounts/<ACCOUND_SID>/Calls/<CALL_SID>/Streams/<STREAM_SID>.json
Twilio returned the following information:
Unable to update record: The requested resource /2010-04-01/Accounts/<ACCOUND_SID>/Calls/<CALL_SID>/Streams/<STREAM_SID>.json was not found
More information may be available here:
https://www.twilio.com/docs/errors/20404 

The stop_media_stream function stops the media stream, but it does not stop the already streamed audio from being played.

The problem with this is that I can send a 1-min audio in a few seconds, so even If I stopped the stream, I will still hear the audio for a while. Does anybody know another way?

Answer 1

Could you share the function used for Elevenlabs, i got this but the stream is not working for a for loop (it will just return 1 chunk)

response = eleven_labs_client.generate(
                            text=text_stream(),
                            voice=Voice(
                                voice_id='EuQDS5FR7dD1UoZkGdbZ',
                                settings=VoiceSettings(stability=0.5, similarity_boost=0.8, style=0.21, use_speaker_boost=True)
                            ),
                            model="eleven_multilingual_v2",
                            stream=True
                        )

                        


                        # Write each chunk of audio data to the stream
                        for chunk in response: #response:
                            if chunk:
                               # Just 1 chunk here

And also, is your websocket async? How did you get this to work, FLASK doesn't let me.

Answer 2

To stop the playing media stream you can use Clear message - https://www.twilio.com/docs/voice/twiml/stream#message-clear-to-twilio

Just send a JSON message:

  { 
    "event": "clear",
    "streamSid": "<STREAM_ID>",
  }