I am trying to transbox the webm output from MediaRecorder with MIME type video/webm;codecs="avc1.42C01E" and other variants of avc1. I want to generate fragmented mp4, with MIME type video/mp4;codecs="avc1.42C01E", containing the same video payload. (I hope to avoid transcoding the video payload, for the sake of latency and power.) This is doable, save for one issue.
Each key frame's data stream -- the first SimpleBlock in each webm Cluster's payload -- starts with SPS and PPS NALUs. That's fine, except for one thing.
The new SPS and PPS NALU for each key frame has new values for pic_parameter_set_id and seq_parameter_set_id. The first key frame has a pic_parameter_set_id value of 0, and subsequent ones have 1, 2, 3, and so forth. When it gets to 56, it cycles back to 0. (57 distinct values?)
The seq_parameter_set_id value goes, similarly, from 0 to 31 and cycles back.
What's more, the slice NALUs coming after each new pair of SPS and PPS refer to the new pic_parameter_set_id values.
This is problematic for a couple of reasons:
- Those id values are coded using exp-Golomb coding. That kind of coding uses more bits to encode larger numbers. So the longer one of these MediaRecorder sessions runs, the bigger the data structures get: PPS, SPS, and slice data.
- MP4 (Quicktime) carries codec-private data in the 'avcC' atom. It supposedly includes the SPS and PPS NALUs for the file. But if those NALUs change for every key frame, it's hard to generate a useful 'avcC'.
As a result, the SourceBuffer.appendBuffer() approach to playing these fmp4 data streams with MSE freezes its picture upon receiving the second key frame.
Is there a way to suppress or work around these changing parameter set ids in the H264 media generated by MediaRecorder, so the whole session uses the same SPS and PPS?
