My overall goal:
Server-side:
- I have batches of sequential, JPEG-encoded frames (8-16) arriving from time to time, generated at roughly 2 FPS.
- I would like to host an HLS live stream, where, when a new batch of frames arrives, I encode those new frames as h264
.ts
segments withffmpeg
, and have the new.ts
segments automatically added to an HLS stream (e.g..m3u8
file).
Client/browser-side:
- When the
.m3u8
is updated, I would like the video stream being watched to simply "continue", advancing from the point where new.ts
segments have been added. - I do not need the user to scrub backwards in time, the client just needs to support live observation of the stream.
My current approach:
Server-side:
To generate the "first" few segments of the stream, I'm attempting the below (just command-line for now to get ffmpeg working right, but ultimately will be automated via a Python script):
For reference, I'm using ffmpeg version 3.4.6-0ubuntu0.18.04.1.
ffmpeg -y -framerate 2 -i /frames/batch1/frame_%d.jpg \
-c:v libx264 -crf 21 -preset veryfast -g 2 \
-f hls -hls_time 4 -hls_list_size 4 -segment_wrap 4 -segment_list_flags +live video/stream.m3u8
where the /frames/batch1/
folder contains a sequence of frames (e.g. frame_01.jpg, frame_02.jpg, etc...). This already doesn't appear to work correctly, because it keeps adding #EXT-X-ENDLIST
to the end of the .m3u8
file, which as I understand is not correct for a live HLS stream - here's what that generates:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.000000,
stream0.ts
#EXTINF:4.000000,
stream1.ts
#EXTINF:2.000000,
stream2.ts
#EXT-X-ENDLIST
I can't figure out how to suppress #EXT-X-ENDLIST
here - this is problem #1.
Then, to generate subsequent segments (e.g. when new frames become available), I'm trying this:
ffmpeg -y -framerate 2 -start_number 20 -i /frames/batch2/frame_%d.jpg \
-c:v libx264 -crf 21 -preset veryfast -g 2 \
-f hls -hls_time 4 -hls_list_size 4 -segment_wrap 4 -segment_list_flags +live video/stream.m3u8
Unfortunately, this does not work the way I want it to. It simply overwrites stream.m3u8
, does and does not advance #EXT-X-MEDIA-SEQUENCE
, it does not index the new .ts
files correctly, and it also includes the undesirable #EXT-X-ENDLIST
- this is the output of that command:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.000000,
stream0.ts
#EXTINF:4.000000,
stream1.ts
#EXTINF:3.000000,
stream2.ts
#EXT-X-ENDLIST
Fundamentally, I can't figure out how to "append" to an existing .m3u8
in a way that makes sense for HLS live streaming. That's essentially problem #2.
For hosting the stream, I'm using a simple Flask app - which appears to be working the way I intend - here's what I'm doing for reference:
@app.route('/video/<string:file_name>')
def stream(file_name):
video_dir = './video'
return send_from_directory(directory=video_dir, filename=file_name)
Client-side:
I'm trying HLS.js in Chrome - basically boils down to this:
<video id="video1"></video>
...
<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
<script>
var video = document.getElementById('video1');
if (Hls.isSupported()) {
var hls = new Hls();
hls.loadSource('/video/stream.m3u8');
hls.attachMedia(video);
hls.on(Hls.Events.MANIFEST_PARSED, function() {
video.play();
});
}
else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = '/video/stream.m3u8';
video.addEventListener('loadedmetadata', function() {
video.play();
});
}
</script>
I'd like to think that what I'm trying to do doesn't require a more complex approach than what I'm trying above, but since what I'm trying to far definitely isn't working, I'm starting to think I need to come at this from a different angle. Any ideas on what I'm missing?
Edit:
I've also attempted the same (again in Chrome) with video.js
, and am seeing similar behavior - in particular, when I manually update the backing stream.m3u8
(with no #EXT-X-ENDLIST
tag), videojs
never picks up the new changes to the live stream, and just buffers/hangs indefinitely.
<video id="video1" class="video-js vjs-default-skin" muted="muted" controls>
<source type="application/x-mpegURL" src="/video/stream.m3u8">
</video>
...
<script>
var player = videojs('video1');
player.play();
</script>
For example, if I start with this initial version of stream.m3u8
:
#EXTM3U
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:8
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.000000,
stream0.ts
#EXTINF:4.000000,
stream1.ts
#EXTINF:2.000000,
stream2.ts
and then manually update it server-side to this:
#EXTM3U
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:8
#EXT-X-MEDIA-SEQUENCE:3
#EXTINF:4.000000,
stream3.ts
#EXTINF:4.000000,
stream4.ts
#EXTINF:3.000000,
stream5.ts
the video.js control just buffers indefinitely after only playing the first 3 segments (stream*.ts 0-2), which isn't what I'd expect to happen (I'd expect it to continue playing stream*.ts 3-5 once stream.m3u8
is updated and video.js
makes a request for the latest version of the playlist).