0

In ffmpeg there is a structure AVFrame describing decoded video or audio data.

It has a void pointer opaque. The documentation claims it is "for some private data of the user".

What does this mean? Can it be used to transport any additional data as per-frame metadata?

Mogsdad
  • 40,814
  • 19
  • 140
  • 246
R2-D2
  • 1,264
  • 11
  • 23

2 Answers2

3

It is a field dedicated for user (as opposed to ffmpeg libraries) usage; ffmpeg will not touch this field in any way so you are free to use it as you see fit. There is a caveat though: some ffmpeg functions will make a copy of AVFrame (or maybe move reference from AVFrame to another), which includes copying this field's value. It might be somehow tricky to manage lifetime of a data pointed to by this field.

If you just need to handle some per-frame metadata, you might want to consider existing metadata storage available in AVFrame (see av_frame_get_metadata/av_frame_set_metadata)

Andrey Turkin
  • 1,637
  • 7
  • 17
  • Can you please comment on the supposed setup to use `av_frame_get_metadata` / `av_frame_set_metadata`? It does not seem to exist within `libav` version 6:9.18-0ubuntu0.14.04.1 – R2-D2 Oct 04 '16 at 16:22
  • @R2-D2 You tagged this with [tag:ffmpeg] and mentioned it in your title, but [you're not actually using FFmpeg](http://stackoverflow.com/questions/9477115/what-are-the-differences-and-similarities-between-ffmpeg-libav-and-avconv/9477756#9477756). I may be wrong, but I don't think the Libav fork has `av_frame_set_metadata`; or at least the ancient source you're using probably doesn't. – llogan Oct 04 '16 at 16:46
  • Libav never adopted metadata patchset; recent libav versions have side data which can be (ab)used to do similar things. More importantly, libav v9 is so old it has no metadata or side data and it even lacks buffer refs. It is pre-refcounted frames API which means using `opaque` is easier since you can link metadata lifetime with AVFrame lifetime. It is also your only choice with this libav version. – Andrey Turkin Oct 04 '16 at 17:31
  • I have a simple av_frame_get_metadata/av_frame_set_metadata example [here](https://stackoverflow.com/questions/51415447/ffmpeg-sidedata-or-metadata-per-frame). – Lihang Li Jan 04 '19 at 01:07
3

To expand a little on what @Andrey Turkin said, the purpose is to add application-specific object data to the AVFrame structure. The specific use case typically is when the application allocates the memory (using the get_buffer2 callback). This memory might just be a pointer, but it can sometimes be memory in the GPU or something like that. Regardless, if the application owns the data and creates an object associated with the allocated memory / picture buffer, it will typically want access to the associated object when the decoder returns a given AVFrame in the avcodec_decode_video2() function, and that's what you typically want to use the opaque field for.

Ronald S. Bultje
  • 9,576
  • 22
  • 43
  • 1
    Nowadays one would typically use opaque field (and maybe other fields too) in AVBuffer for these use cases, and not one in AVFrame. – Andrey Turkin Oct 04 '16 at 15:30
  • So, if I understand correctly: while encoding video data, any user-data pointed to by `opaque` will not show up in the resulting file/stream? – R2-D2 Oct 04 '16 at 15:31
  • 1
    yes. ffmpeg will not read or write to that field, expect for setting to set it to NULL on frame creation, and to copy its value when copying frames. – Andrey Turkin Oct 04 '16 at 15:36