Opentok Screen Sharing with Audio

Question

I try to create a Screen Sharing application with the opentok JS client that shares the publishers audio as well.

Screen Sharing works fine. But the audio is never shared.

Now, I noticed a warning in the console (Firefox) saying Invalid audioSource passed to Publisher - when using screen sharing no audioSource may be used. Does that mean it is not possible at all, or that the audio source is invalid?

score 13 · Answer 1 · answered Jan 29 '18 at 05:21

With v2.13.0 it is now possible to pass a MediaStreamTrack as a custom audioSource and videoSource to initPublisher. This means you are able to add your microphone audio to the screen sharing stream. This will only work in Chrome or Firefox. IE does not support MediaStreamTrack's and Safari does not currently support screensharing.

const publish = Promise.all([
  OT.getUserMedia({
    videoSource: 'screen'
  }),
  OT.getUserMedia({
    videoSource: null
  })
])
.then(([screenStream, micStream]) => {
  return OT.initPublisher(null, {
    videoSource: screenStream.getVideoTracks()[0],
    audioSource: micStream.getAudioTracks()[0]
  });
});

Here is a sample of it all working https://output.jsbin.com/wozuhuc This sample will only work in Firefox because Chrome needs an extension.

This works, but the streams type will be set to 'camera' instead of 'screen'. You also have to manually set `fitMode` to 'contain' in Ot.initPublisher's property input. This will fix screen sizing issues when screensharing. — Anthony Harley, Mar 27 '18 at 20:51

score 0 · Answer 2 · answered Jan 17 '18 at 12:47

0

If you create a subscriber, and connect it to the session, it will receive audio and video from all publishers. As far as I know, there is no audio in screen sharing, that's why you cannot publish it. That should solve it. I hope this helps.

answered Jan 17 '18 at 12:47

Francisco Javier Cano Sandoval

294
1
5

So you are saying that audio is not available for screen sharing - what a pity! – Rico Leuthold Jan 17 '18 at 14:42
Well, screen itself does not have audio, so that makes sense for me. – Francisco Javier Cano Sandoval Jan 17 '18 at 15:59
It doesn't really make much sense that we need to send audio separately, and becomes more complex. But at least I know why it's not working. – Bojan Kogoj Jan 19 '18 at 13:42

score 0 · Accepted Answer · answered Jan 17 '18 at 14:55

0

I contacted the tokbox support and they confirmed, that the audio has to be published in an additional stream.

answered Jan 17 '18 at 14:55

Rico Leuthold

1,865
4
32
47

score 0 · Answer 4 · answered Dec 11 '20 at 10:21

I had a go at making this work in Chrome, which is possible by using getDisplayMedia({video: true, audio: true}), which now shows a tickbox to allow the user to share device audio:

You can then use this to create a normal publisher which uses the video and audio streams from this call like so:

 let prom = navigator.mediaDevices.getDisplayMedia({ video:true, audio: true });

    prom.then(function(result) {
        console.log("Collected permission. Going to start publishing.");
        desktopStream = result;
        var audioStream = desktopStream.getAudioTracks()[0];
        var videoStream = desktopStream.getVideoTracks()[0];

        console.log(audioStream);

        // Screen sharing is available. Publish the screen.
        screenPublisher = OT.initPublisher('ownScreen',
            {
                videoSource: videoStream,
                audioSource: audioStream,
                name: 'Sharing Screen',
                maxResolution: { width: 1920, height: 1920 },
                mirror: false,
                fitMode: "contain",
            },
            function(error) {
                if (error) {
                    console.log(error);
                    // Look at error.message to see what went wrong.
                } else {
                    session.publish(screenPublisher, function(error) {
                        if (error) {
                            handleError();
                        }

                        $('#shareScreen').fadeOut(150, function(){
                            $('#stopShare').fadeIn(150);
                        });

                        $('#stopShare').addClass('blob blue');

                    });
                }
            }
        );

You can also add a name to the screen share publisher to allow subscribers to distinguish between a regular video publisher and this new custom screen share publisher.

Opentok Screen Sharing with Audio

4 Answers4

Linked