21

So from my web server, I would like to use FFMPEG to transcode a media file for use with an HTML <audio> or <video> tag. Easy enough right?

The conversion would need to take place in real-time, when an HTTP client requested the converted file. Ideally the file would be streamed back to the HTTP client as it is being transcoded (and not afterwards at the end, since that would potentially take a while before any data starts being sent back).

This would be fine, except that in today's browsers, an HTML5 audio or video tag requests the media file in multiple HTTP requests with the Range header. See this question for details.

In that question linked above, you can see that Safari requests weird chunks of the file, including the ending few bytes. This poses a problem in that the web server WOULD have to wait for the conversion to finish, in order to deliver the final bytes of the file to conform to the Range request.

So my question is, is my train of thought right? Is there a better way to deliver transcoding content to an <audio> or <video> tag that wouldn't involve waiting for the entire conversion to finish? Thanks in advance!

Community
  • 1
  • 1
TooTallNate
  • 1,440
  • 2
  • 19
  • 39
  • 2
    So what happens if 50 people try to view your video at the same time (or if I press refresh 100 times)? Realtime transcoding might work for one video on a workstation, but I'd suspect it's too resource-intensive to do for every request on a server. Maybe a "please wait while we convert-then-cache" strategy would work? – Seth Sep 03 '10 at 21:51
  • 1
    Caching the converted file is what I was already considering at this point. But [my application](http://github.com/TooTallNate/nTunes) is intended to be locked down to a few users anyways, as openly allowing HTTP access to your entire iTunes library is not so much a good idea in my opinion. – TooTallNate Sep 03 '10 at 22:33
  • I would go with YouTube's approach (Seth's suggestion above)-- I'm guessing they asked this question at some point :) – Dolph Dec 14 '10 at 03:17

5 Answers5

11

I recently run into the same issue since I want to serve my library to browsers. Surprisingly, the idea to send the stream through ffmpeg and deliver on the fly works quite well. The primary problem was to support seeking...

Following, you find code sniplets in Python using Flask to solve the problem:

We need a function to stream the content:

@app.route('/media/<path:path>.ogv')
def media_content_ogv(path):
    d= os.path.abspath( os.path.join( config.media_folder, path ) )
    if not os.path.isfile( d ): abort(404)
    start= request.args.get("start") or 0
    def generate():
        cmdline= list()
        cmdline.append( config.ffmpeg )
        cmdline.append( "-i" )
        cmdline.append( d );
        cmdline.append( "-ss" )
        cmdline.append( str(start) );
        cmdline.extend( config.ffmpeg_args )
        print cmdline
        FNULL = open(os.devnull, 'w')
        proc= subprocess.Popen( cmdline, stdout=subprocess.PIPE, stderr=FNULL )
        try:
            f= proc.stdout
            byte = f.read(512)
            while byte:
                yield byte
                byte = f.read(512)
        finally:
            proc.kill()

    return Response(response=generate(),status=200,mimetype='video/ogg',headers={'Access-Control-Allow-Origin': '*', "Content-Type":"video/ogg","Content-Disposition":"inline","Content-Transfer-Enconding":"binary"})

Then we need a function to return the duration:

@app.route('/media/<path:path>.js')
def media_content_js(path):
    d= os.path.abspath( os.path.join( config.media_folder, path ) )
    if not os.path.isfile( d ): abort(404)
    cmdline= list()
    cmdline.append( config.ffmpeg )
    cmdline.append( "-i" )
    cmdline.append( d );
    duration= -1
    FNULL = open(os.devnull, 'w')
    proc= subprocess.Popen( cmdline, stderr=subprocess.PIPE, stdout=FNULL )
    try:
        for line in iter(proc.stderr.readline,''):
            line= line.rstrip()
            #Duration: 00:00:45.13, start: 0.000000, bitrate: 302 kb/s
            m = re.search('Duration: (..):(..):(..)\...', line)
            if m is not None: duration= int(m.group(1)) * 3600 + int(m.group(2)) * 60 + int(m.group(3)) + 1
    finally:
        proc.kill()

    return jsonify(duration=duration)

And finally, we hack that into HTML5 using videojs:

<!DOCTYPE html>
<html>
<head>
    <link href="//vjs.zencdn.net/4.5/video-js.css" rel="stylesheet">
    <script src="//vjs.zencdn.net/4.5/video.js"></script>
    <script src="http://code.jquery.com/jquery-1.9.1.min.js"></script>
</head>
<body>
    <video id="video" class="video-js vjs-default-skin" controls preload="auto" width="640" height="264">
    </video>
    <script>
        var video= videojs('video');
        video.src("media/testavi.avi.ogv");

        // hack duration
        video.duration= function() { return video.theDuration; };
        video.start= 0;
        video.oldCurrentTime= video.currentTime;
        video.currentTime= function(time) 
        { 
            if( time == undefined )
            {
                return video.oldCurrentTime() + video.start;
            }
            console.log(time)
            video.start= time;
            video.oldCurrentTime(0);
            video.src("media/testavi.avi.ogv?start=" + time);
            video.play();
            return this;
        };

        $.getJSON( "media/testavi.avi.js", function( data ) 
        {
            video.theDuration= data.duration;
        });
    </script>
</body>

A working example can be found at https://github.com/derolf/transcoder .

dero

user3612643
  • 2,858
  • 3
  • 19
  • 38
  • This helped me so much! I am doing the same thing but with node, I just translated your python code to node and also changed some errors in the javascript (oldCurrentTime is not a function). It finally works now, after trying to implement seek for 2 whole days. Thank you! – Gustav P Svensson May 13 '20 at 13:37
3

Thanks for the reply Camilo. I took a closer look at the HTTP spec regarding the Range request and found:

The header SHOULD indicate the total length of the full entity-body, unless
this length is unknown or difficult to determine. The asterisk "*" character
means that the instance-length is unknown at the time when the response was
generated.

So it's really just a matter of testing how the browsers react when replying with a Content-Range: bytes 0-1/*, for example. I'll let you know what happens.

TooTallNate
  • 1,440
  • 2
  • 19
  • 39
  • How did you find the compatability with this approach? Did it work well across browsers? Last time I did this I found it was better to take a "guess" at the length and zero-fill the remainder by guessing large. This seems to work with constant bitrates decently. – jocull May 31 '13 at 21:29
  • I never ended up trying to be honest. Anybody else know? Since the spec specifies it it _may_ work out, but it's such a niche part of the spec that we might need to open issues on the various browsers' bug trackers to get it working properly. – TooTallNate Jan 11 '14 at 05:17
0

AFAIK you can encode to stdout in ffmpeg. So you could configure your HTTP server to:

  • start encoding to cache when GET recieved.
  • stream requested range of bytes to client.
  • filling the buffer and using it for subsequent ranges.

I'm clueless but I think you can get away without knowing the final stream's lenght.

On a side note, I think this is prone to DoS.

Camilo Martin
  • 34,128
  • 20
  • 104
  • 150
0

I know this is an old thread but I post it anyway if someone finds this and need help.

'user3612643' answer is correct, that fixes the seek problem. However that introduces a new problem. The current time is no longer correct. To fix this we have to copy the original currentTime function.

Now everytime video.js calls currentTime (with no parameters) it will call oldCurrentTime which is the original currentTime function. The rest is the same as 'user3612643's answer (Thanks!). This works with the newest video.js (7.7.6)

    video = videojs("video");
    video.src({
      src: 'http://localhost:4000/api/video/sdf',
      type: 'video/webm'
    });


     // hack duration
     video.duration= function() {return video.theDuration; };
     video.start= 0;

     // The original code for "currentTime"
     video.oldCurrentTime = function currentTime(seconds) {
      if (typeof seconds !== 'undefined') {
        if (seconds < 0) {
          seconds = 0;
        }

        this.techCall_('setCurrentTime', seconds);
        return;
      }
      this.cache_.currentTime = this.techGet_('currentTime') || 0;
      return this.cache_.currentTime;
    }

      // Our modified currentTime
     video.currentTime= function(time) 
     { 
         if( time == undefined )
         {
             return video.oldCurrentTime() + video.start;
         }
         video.start= time;
         video.oldCurrentTime(0);
         video.src({
           src: "http://localhost:4000/api/video/sdf?start=" + time,
           type: 'video/webm'
          });
         video.play();
         return this;
     };

     // Get the dureation of the movie
     $.getJSON( "http://localhost:4000/api/video/sdf/getDuration", function( data ) 
     {
         video.theDuration= data.duration;
     });
Gustav P Svensson
  • 181
  • 1
  • 10
-1

This should be doable via VLC, I was able to get it to work by setting VLC to host a large avi file and transcode it to OGG, then my html5 referenced the stream:

<source src="http://localhost:8081/stream.ogg">

It was able to transcode in vlc, and render just fine in my chrome browser and on my android phone, but I ended up taking a different solution rather than going through the work of creating my own webapp to host my media collection and create streams for requested files - I looked and couldn't find a free one already out there that did it in a way I needed/liked.

TheDruidsKeeper
  • 325
  • 2
  • 11