CODEWITHSUNDEEP
javascriptreactjsgoogle-chrome-extension
Nabil Alhusail
Nabil Alhusail

Reputation: 3

Chrome extension Can't play meeting audio

I'm developing a google chrome extension using Manifest V2

The goal of the extension is to enable the user to start online meetings within the browser

I developed pages using react, and then inject these pages into an iFrame inside the currently open website

I can capture the audio/video from the user, and I can send it successfully I receive audio & video streams from the other users that join the meeting, and I can play their video.

The problem is with audio, I receive the stream, but it doesn't play

I tried playing it

in all of these cases, I couldn't play the audio that is streamed

I'm not even getting an error message in the console

one thing I did notice, is that I can play my own audio (echo my microphone input), and if I add it to an <audio> tag with controls visible, it starts counting up. But when I put the attendees audio track on <audio> tags, the controls stays at 0:00

Here's the code for the VideoPlayer.jsx

import React, { useEffect, useRef, useState } from 'react';

import msg from '../../lib/msg';
import { Flex } from '@chakra-ui/react';

const VideoPlayer = (props) => {
  const { sessionId } = props;
  const [participants, setParticipants] = useState([]);
  const [currentParticipant, setCurrentParticipant] = useState(null);
  const [videoTrack, setVideoTrack] = useState(null);
  const [audioTrack, setAudioTrack] = useState(null);
  const [isLocal, setIsLocal] = useState(false);

  useEffect(() => {
    if (currentParticipant) {
      if (currentParticipant.videoTrack) {
        setVideoTrack(currentParticipant.videoTrack);
      } else {
        setVideoTrack(null);
      }
      //set the audio track regardless if it's allowed or not, that's a decision for another function
      if (currentParticipant.audioTrack) {
        setAudioTrack(currentParticipant.audioTrack);
      } else {
        setAudioTrack(null);
      }
    }
  }, [currentParticipant]);

  const videoEl = useRef(null);
  const audioEl = useRef(null);

  /**
   * Set the video element's source
   */
  useEffect(() => {
    if (!videoEl.current || !videoTrack) return;

    videoEl.current.srcObject = new MediaStream([videoTrack]);
  }, [videoEl, videoTrack]);

  /**
   * Set the audio element's source
   */
  useEffect(() => {
    if (!audioEl.current || !audioTrack || isLocal) return;
    audioEl.current.srcObject = new MediaStream([audioTrack]);
  }, [audioEl, isLocal, audioTrack]);

  useEffect(() => {
    const reloadParticipants = () => {
      chrome.runtime.getBackgroundPage(function (win) {
        var p = win.getParticipants();
        setParticipants(p);
      });
    };
    const updateTracks = async (message, sender, sendResponse) => {
      
      chrome.runtime.getBackgroundPage(function (win) {
        var participantList = win.getParticipants();
        var index = participantList.findIndex(
          (item) => item.session_id === sessionId
        );
        var participant = participantList[index];
        setCurrentParticipant(participant);
        setIsLocal(participant.local);
      });
    };

    const msgListeners = [
      { action: msg.broadcast.participantListUpdated, fn: reloadParticipants },
      { action: msg.broadcast.tracksUpdated, fn: updateTracks },
    ];
    const initMessageListeners = async () => {
      chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
        msgListeners.forEach(async (item) => {
          if (message.action === item.action) {
            await item.fn(message, sender, sendResponse);
          }
        });
      });
    };

    initMessageListeners();
    reloadParticipants();
  }, [sessionId]);

  return (
    <div className="tile-container">
      <audio autoPlay playsInline ref={audioEl} />
      <video
        autoPlay
        muted
        playsInline
        ref={videoEl}
        style={{
          width: '100%',
          height: '100%',
          position: 'absolute',
          objectFit: 'cover',
        }}
      />
    </div>
  );
};

export default VideoPlayer;

Then I'm using webpack to package it into a page

import React from 'react';
import { render } from 'react-dom';

import VideoPlayer from './VideoPlayer';
import './index.css';

render(<VideoPlayer />, window.document.querySelector('#app-container'));

I inject this from the content script using this code

// at top I have the variable
const videoPlayerUrl = chrome.runtime.getURL('/videoPlayer.html');

// I monitor for an event "participant joined" then call the following function
const createParticipantOverlay = (item) => {
    var participantContainer = $(`
    <div id="participant-${item.session_id}" class="bm_videoContainer">
      <iframe
        id="bm_ownerVideoFrame"
        class="bm_videoFrame"
        allow="autoplay *"
        src="${videoPlayerUrl}?session_id=${item.session_id}"
      ></iframe>
      <div id="bm_dragHandler" class="bm_dragHandler"></div>
    </div>`);
    var container = $('#meeting-container');
    container.append(participantContainer);

    var dragHandler = $(`#participant-${item.session_id}`).find(
      '#bm_dragHandler'
    );
    dragHandler.on('mousedown', mouseDown);
    dragHandler.on('mouseup', mouseUp);
    dragHandler.on('mousemove', mouseMove);
  };

note: I'm using daily.co for managing the call

Upvotes: 0

Views: 183

Answers (1)

Jessica Mitchell
Jessica Mitchell

Reputation: 156

I built something similar last year with Daily and manifest v2 (I work at Daily by the way, so hopefully I can help sort this out!) Here is the participant Tile React component I used to get this working for reference: https://github.com/daily-demos/daily-notion-chrome-extension/blob/main/src/pages/Content/components/Tile.jsx

Looking at your code, nothing super obvious is standing out to me.. Is there a public repo I could look at? You may need to listen for more than just the participant-joined Daily event. Do you also have an event listener for participant-updated?

Upvotes: 0

Related Questions