CODEWITHSUNDEEP
audiovideosynchronizationtwilio
Brendan Hill
Brendan Hill

Reputation: 3752

How accurate is the timing/syncing of Twilio compositions?

I have a use case for Twilio compositions (two video/audio streams) which relies heavily on the timing/syncing accuracy of the participant streams.

We produce a composition of both videos, and also compositions of each individual audio streams. The word timings within each stream are extracted and compared which relies heavily on the syncing accuracy between the two streams.

In particular we require accurate syncing (to the best of the available data) even when there were network issues, disconnects/reconnects and so on.

Can any comment on how accurate and robust the timing/syncing of Twilio's compositions are for this use case?

Have you had similar use cases and can you comment on your experience with accuracy down to say ~100ms?

Sorry - I know this is a general and vague question - just not sure where else to ask.

Upvotes: 1

Views: 64

Answers (1)

Brendan Hill
Brendan Hill

Reputation: 3752

Posting an answer here for everyone's future reference.

Having done hundreds of compositions in a wide variety of test scenarios (joining at different times, reconnections, dodgy internet, closing browser, etc) I can happily report that Twilio compositions system generates extremely reilable and accurate compositions in terms of timing and syncing.

However it seems necessary to include both participants in the composition to ensure the start times are consistent.

My solution to creating an individual participant's composition (fully synced with the other participants was:

  1. Create a composition with the video from both participants
  2. Make other participant's video z-ordered behind the main participant, and placed 16x16 in the corner
  3. Only include the audio streams from the participant of interest

Doing this in turn for each participant produces individual participant compositions which are completely aligned with each other.

Upvotes: 0

Related Questions