6

I wrote a demo to record a video using MediaCodec and MediaMuxer.

I record a video with my demo and use ffprobe to check the video, the result is as follows:

  Duration: 00:00:06.86, start: 0.000000, bitrate: 723 kb/s
Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 320x240, 619 kb/s, SAR 1:1 DAR 4:3, 30.02 fps, 30 tbr, 90k tbn, 180k tbc (default)
Metadata:
  creation_time   : 2015-06-05 13:19:24
  handler_name    : VideoHandle
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 96 kb/s (default)
Metadata:
  creation_time   : 2015-06-05 13:19:24
  handler_name    : SoundHandle

It contains video and audio information, I found the audio properties are the same as I set in the source code, but the video properties are not all right. My video settings source code is as follows:

        MediaFormat format = MediaFormat.createVideoFormat(MIME_TYPE, mWidth, mHeight);
    format.setInteger(MediaFormat.KEY_COLOR_FORMAT,
            MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);
    format.setInteger(MediaFormat.KEY_BIT_RATE, 384000);
    format.setInteger(MediaFormat.KEY_FRAME_RATE, 19);
    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
    if (VERBOSE) Log.d(TAG, "format: " + format);
    mVideoEncoder = MediaCodec.createEncoderByType(MIME_TYPE);
    mVideoEncoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    mInputSurface = mVideoEncoder.createInputSurface();
    mVideoEncoder.start();

The width and height of the video are right but bitrate and framerate are higher than I set in source code. It results in that the video file size is much larger than I expected. Then, I modified my source code to remove audio recording thread and just record the video only. But it didn't make any differences, the bitrate and framerate are also higher. Who can tell me the reason and give me some advices?

And there is another problem. I record a broken video occasionally which can be played by system player, but the begining of the video is just black and normal image displayed after 1 or 2 seconds. I don't know how to upload file in stackoverflow, I can send the broken video file to anyone who need it. Is there someone came with this problem?

ADD： I found another strange thing: My video encoding config:

private int mWidth = 480;
private int mHeight = 848;
private int mVideoBitrate = 1200 * 1000;

    MediaFormat format = MediaFormat.createVideoFormat(MIME_TYPE, 480, 848);

    format.setInteger(MediaFormat.KEY_COLOR_FORMAT,
            MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);
    format.setInteger(MediaFormat.KEY_BIT_RATE, 1200000);
    format.setInteger(MediaFormat.KEY_FRAME_RATE, 30);
    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);

but the actual video info is:

  Duration: 00:00:06.01, start: 0.000000, bitrate: 6491 kb/s
Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 15 kb/s (default)
Metadata:
  creation_time   : 2015-09-30 15:44:18
  handler_name    : SoundHandle
Stream #0:1(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 480x848, 6484 kb/s, SAR 1:1 DAR 30:53, 16 fps, 30 tbr, 90k tbn, 180k tbc (default)
Metadata:
  creation_time   : 2015-09-30 15:44:18
  handler_name    : VideoHandle
dragonfly
  • 1,055
  • 12
  • 34
  • Hi @dragonfly Do you done both audio and video recording successfully?? – Ali Dec 04 '18 at 09:26
  • yes. I did it at last. – dragonfly Dec 12 '18 at 13:43
  • i heartly request you can you please share demo with me. @dragonfly – Ali Dec 12 '18 at 13:47
  • @Ali You can get what you want from this project:https://github.com/google/grafika – dragonfly Dec 17 '18 at 10:13
  • brother i use this grafika(`ContinuesCaptureActivity`) and create demo but problem is this demo work without Audio so how to i add ?? please see this question https://stackoverflow.com/questions/53494059/how-to-add-audio-to-video-while-recording-continuouscaptureactivity-grafika – Ali Dec 17 '18 at 10:17

1 Answers1

7

It looks like your expected frame rate (19fps) doesn't match the actual frame rate (30fps). The encoder is attempting to meet the bit rate requirements for frames submitted at 19fps, but they're coming in faster, so it misses and the actual bit rate is higher. I'm assuming the 30fps value is determined from the actual presentation time stamps in the video file, which are set by the presentation time stamps passed into MediaMuxer. If you want 19fps video, you need to generate time stamps that are (1000/19) milliseconds apart.

If your input video is 30fps, you will need to drop frames during the encoding process to get to 19fps. MediaCodec does not drop frames for you -- it encodes everything you pass in.

I don't know why you'd be getting a broken section at the start of the video.

fadden
  • 48,613
  • 5
  • 104
  • 152
  • 1
    If MediaCodec encodes every frame fed to it, the frame rate of target video will be determined by the rate of input source such as camera. In other words, MediaCodec is unable to determine the frame rate, but why MediaCodec allows application to set frame rate through MediaFormat.KEY_FRAME_RATE? Is this a bug or I made the wrong assumption? – dragonfly Jun 06 '15 at 12:30
  • 2
    You need to provide an estimate of the frame rate so the codec can configure an appropriate level of compression to achieve the target bit rate. KEY_FRAME_RATE is a way for your app to tell the codec what's coming, not tell the codec what to produce. It's also factored into the key frame frequency -- if you request 30fps with key frames every second, you'll get one key frame every 30 frames. Bear in mind that H.264 video streams don't have timestamps -- those are in the .mp4 file, added by MediaMuxer -- so some codecs may just pass the PTS values through unexamined. – fadden Jun 06 '15 at 15:34
  • 1
    Thanks, I record a video of 19 fps successfully with the help of you. At first, I skip frame in the process of drainEncoder, as a result, the image of the video is very obscure, just like mosaic. Then I change the way, I skip some frame at SurfaceTexture.OnFrameAvailableListener, the video recorded become normal. – dragonfly Jun 08 '15 at 08:23
  • Hi, fadden. Because of your kindness, I achieved my work at last. But there is still one thing I am worrying about. I created a question at http://stackoverflow.com/questions/30709672/worrying-about-the-compatibility-of-android-mediacodec-and-mediamuxer-since-api Could you give me some advices? – dragonfly Jun 08 '15 at 12:56
  • @fadden What should I do if I want a 30 fps video, and the encoder receives 60 frames every second? Can I set the presentation timeUs = 33ms and drop a half of frames? How to drop the frames? Just do not pass them to Muxer? (encoder receives frames from an Input Surface and pass them to MediaMuxer). – TOP Jun 20 '15 at 04:49
  • 1
    @Sunshinetpu: the encoder will encode every frame you give it. If you want to convert 60fps video to 30fps video, you need to not pass half the frames to the encoder. Calling the decoder's `releaseOutputBuffer()` with the render flag set to `false` should do the trick -- that frame won't be rendered on the encoder's input surface. If you don't control the decoder, things are harder. You can receive the frames with a SurfaceTexture, then render every-other frame to the encoder's Surface with GLES. (bigflake and Grafika have some useful code, but you'll need to put the pieces together.) – fadden Jun 20 '15 at 05:27
  • @fadden I have no decoder. I want to create a screen recorder. I use new API of Lollipop: MediaProjection and pass the InputSurface of encoder to it. So I have MediaProjection->InputSurface->Encoder->MediaMuxer. Can I drop the output buffers of Encoder (do not pass it to MediaMuxer)? – TOP Jun 20 '15 at 06:30
  • @Sunshinetpu: no. In an AVC stream, frames may be dependent upon other frames, so you can't drop them arbitrarily. You will need to send the raw frames through an intermediate step, e.g. a SurfaceTexture, and only send the ones you want to keep to the encoder. Dropping frames from a non-continuous source can be tricky though. If you're unclear on how to proceed, you should file a new question, as this is getting a big cramped. :-) – fadden Jun 20 '15 at 15:12
  • @Sunshinetpu I hear that screen recording on Android requires root permission, Does your screen recorder using MediaProjection need root permission? – dragonfly Jul 01 '15 at 01:32
  • @Sunshinetpu And I wonder whether the definition of video is good and the text in the video is clear. – dragonfly Jul 01 '15 at 03:19
  • @fadden Hi, I found that the profile of videos recorded are different when I use different phones. Some videos have the profile high, but some have the profile baseline. How can I set the profile? I need the profile high always. – dragonfly Sep 21 '15 at 03:39