Twilio Video sizing in Firefox

Question

I am trying to get a video panel of a specific size working but am struggling to get it working in browsers other than Chrome. My stack is Angular 5, Twilio Video (1.10.0 using npm i twilio-video) and Twilio library 1.13 (from here: //media.twiliocdn.com/taskrouter/js/v1.13/taskrouter.min.js)

I have the following code (which is based on the QuickStart project: https://github.com/twilio/video-quickstart-js) which works perfectly fine in Chrome and gives me a window that is 195 wide, but when I run the same code in Firefox I get a video window that is 640x480. Any help here would be much appreciated!

import { connect, Room, Track } from 'twilio-video';

makeConnection(token: string, roomName: string): void {
  connect(token,
    {
      audio: true,
      name: roomName,
      video: { width: 195 }
    }
  ).then(room => {
    this.room = room;

    this.roomJoined(room);
  }, error => {
    this.error = error;
  });
}

roomJoined(room) {
  // Attach LocalParticipant's Tracks, if not already attached.
  let previewContainer = document.getElementById('local-media');
  if (!previewContainer.querySelector('video')) {
    console.log("Adding preview");
    this.attachParticipantTracks(room.localParticipant, previewContainer);
  }

  // Attach the Tracks of the Room's Participants.
  room.participants.forEach(participant => {
    console.log("Already in Room: '" + participant.identity + "'");
    let viewContainer = document.getElementById('remote-media');
    this.attachParticipantTracks(participant, viewContainer);
  });

  // When a Participant joins the Room, log the event.
  room.on('participantConnected', participant => {
    console.log("Joining: '" + participant.identity + "'");
  });

  // When a Participant adds a Track, attach it to the DOM.
  room.on('trackAdded', (track, participant) => {
    console.log(participant.identity + " added track: " + track.kind);
    let viewContainer = document.getElementById('remote-media');
    this.attachTracks([track], viewContainer);
  });

  // When a Participant removes a Track, detach it from the DOM.
  room.on('trackRemoved', (track, participant) => {
    console.log(participant.identity + " removed track: " + track.kind);
    this.detachTracks([track]);
  });

  // When a Participant leaves the Room, detach its Tracks.
  room.on('participantDisconnected', (participant) => {
    console.log("Participant '" + participant.identity + "' left the room");
    this.detachParticipantTracks(participant);
  });

  // Once the LocalParticipant leaves the room, detach the Tracks
  // of all Participants, including that of the LocalParticipant.
  room.on('disconnected', () => {
    console.log('Left');
    if (this.previewTracks) {
      this.previewTracks.forEach( (track) => {
        track.stop();
      });
    }
    room.participants.forEach(participant => this.detachParticipantTracks(participant));
  });
}

// Attach the Tracks to the DOM.
attachTracks(tracks, container) {
  tracks.forEach(track => {
    container.appendChild(track.attach());
  });
}

// Attach the Participant's Tracks to the DOM.
attachParticipantTracks(participant, container) {
  let tracks = Array.from(participant.tracks.values());
  this.attachTracks(tracks, container);
}

// Detach the Tracks from the DOM.
detachTracks(tracks) {
  tracks.forEach(track => {
    track.detach().forEach( (detachedElement) => {
      detachedElement.remove();
    });
  });
}

// Detach the Participant's Tracks from the DOM.
detachParticipantTracks(participant) {
  let tracks = Array.from(participant.tracks.values());
  this.detachTracks(tracks);
}

disconnectFromRoom(): void {
  console.log("Disconnecting");
  this.room.disconnect();
}

Answer 1

In case anyone should come across the same behaviour: I've been having the problem, that Firefox was ignoring the aspectRatio setting for a specific camera (Microsoft LifeCam HD-3000, to be precise) and always created a 4:3 image, although the aspect ratio was set to 16:9. My code looked something like this:

createLocalVideoTrack({
  height: 720,
  aspectRatio: 16/9,
  deviceId,
})

This worked fine in every browser but Firefox. The only way to fix it, was to specifically also provide the width property:

createLocalVideoTrack({
  height: 720,
  width: 720 * (16/9)
  aspectRatio: 16/9,
  deviceId,
})

Answer 2

You can add video size like this

 function attachTracks(tracks, container) {

   tracks.forEach(function(track) {
    container.appendChild(track.attach());
   });

   $('#remote-media > video').css({'width': '100%'});
  $('#local-media > video').css({'width': '100%', 'margin-left': '0px'});
}

Answer 3

Twilio developer evangelist here.

After a bit of testing myself, I've discovered that Firefox doesn't like many values of width as part of the media constraints.

If you switch your constraint from

video: {
  width: 195
}

to

video: {
  width: {
    exact: 195
  }
}

Then try to get the camera stream, it will fail with the message "Constraints could be not satisfied." In this case, Firefox was ignoring 195 before, because it was a suggestion, when using exact it either has to comply or fail, and it failed.

My recommendation is to provide a range of constraints for the width that browsers can pick their best option from. In your case, the ideal width is 195, but if Firefox isn't going to support that, we should give it a range of acceptable widths. Something like:

video: {
  width: {
    ideal: 195,
    min: 160,
    max: 320
  }
}

Then, I recommend you size the resulting <video> element with CSS to make it fit the size you wanted too.

You can read more about setting media constraints and ranges on MDN.

Let me know if that helps at all.