How can I properly map speakers in Twilio when utilizing live transcription and connecting to a "client" after a call with "from" and "to" numbers?

Question

I am using python and working with Twilio where my Twilio number first calls the "to" number (from the "from" number). Once the "to" number picks up, the call is then forwarded to a "client." I am using live audio transcription for this setup, but I'm facing challenges in mapping the speakers correctly in the transcription.

Here is the flow:

1. My Twilio number initiates the call to the "to" number.
2. Once the "to" number answers, the call is connected to a "client."
3. I am using Twilio's live transcription feature, but I need to map the transcription correctly to distinguish between the "to" number and the "client."

response = VoiceResponse()
start = Start()
start.stream(url=f"{wss_url}/stream/{stream_sid}", track="both")
response.append(start)
dial = Dial()
client = Client(agent_id)
dial.append(client)
response.append(dial)

I am using track="both" , but I am still unable to map the speakers correctly. How can I achieve proper speaker mapping in this scenario?