Can you "stream" images to ffmpeg to construct a video, instead of saving them to disk?

Question

My work recently involves programmatically making videos. In python, the typical workflow looks something like this:

import subprocess, Image, ImageDraw

for i in range(frames_per_second * video_duration_seconds):
    img = createFrame(i)
    img.save("%07d.png" % i)

subprocess.call(["ffmpeg","-y","-r",str(frames_per_second),"-i", "%07d.png","-vcodec","mpeg4", "-qscale","5", "-r", str(frames_per_second), "video.avi"])

This workflow creates an image for each frame in the video and saves it to disk. After all images have been saved, ffmpeg is called to construct a video from all of the images.

Saving the images to disk (not the creation of the images in memory) consumes the majority of the cycles here, and does not appear to be necessary. Is there some way to perform the same function, but without saving the images to disk? So, ffmpeg would be called and the images would be constructed and fed to ffmpeg immediately after being constructed.

I don't know how you're creating the images, but ffmpeg accepts pipe inputs too: `ffmpeg -f image2pipe -c:v png -r 30000/1001 -i -`. — llogan, Nov 08 '12 at 18:19
For simplicity, just assume that `createFrame(i)` returns a Python Image Library image object, which we store in `img`. I think your response is a step in the right direction, but half the challenge would be piping the constructed images to ffmpeg while in the python program. — Brandon, Nov 08 '12 at 19:20
maybe queue and then pipe the images through a second thread? — unddoch, Nov 08 '12 at 20:01
May be able to send your input into a named pipe and pass that to ffmpeg, as well, basically the same process... — rogerdpack, Mar 04 '14 at 17:13

score 67 · Accepted Answer · answered Nov 08 '12 at 21:58

Ok I got it working. thanks to LordNeckbeard suggestion to use image2pipe. I had to use jpg encoding instead of png because image2pipe with png doesn't work on my verision of ffmpeg. The first script is essentially the same as your question's code except I implemented a simple image creation that just creates images going from black to red. I also added some code to time the execution.

serial execution

import subprocess, Image

fps, duration = 24, 100
for i in range(fps * duration):
    im = Image.new("RGB", (300, 300), (i, 1, 1))
    im.save("%07d.jpg" % i)
subprocess.call(["ffmpeg","-y","-r",str(fps),"-i", "%07d.jpg","-vcodec","mpeg4", "-qscale","5", "-r", str(fps), "video.avi"])

parallel execution (with no images saved to disk)

import Image
from subprocess import Popen, PIPE

fps, duration = 24, 100
p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'mjpeg', '-r', '24', '-i', '-', '-vcodec', 'mpeg4', '-qscale', '5', '-r', '24', 'video.avi'], stdin=PIPE)
for i in range(fps * duration):
    im = Image.new("RGB", (300, 300), (i, 1, 1))
    im.save(p.stdin, 'JPEG')
p.stdin.close()
p.wait()

the results are interesting, I ran each script 3 times to compare performance: serial:

12.9062321186
12.8965060711
12.9360799789

parallel:

8.67797684669
8.57139396667
8.38926696777

So it seems the parallel version is faster about 1.5 times faster.

For anyone who stumbles upon this in the future, replacing 'mjpeg' with 'png' and 'JPEG' with 'PNG' worked for me to use png. — Brandon, Nov 10 '12 at 23:26
I managed to get the best quality using `-vcodec png` and `im.save(p.stdin, 'PNG')` though the filesize is x4 — bluesummers, May 09 '17 at 12:02
Darn, the parrallel script worked perfectly until I updated to Python 3.6. Now I get `OSError: [WinError 6] The handle is invalid` on the `p = Popen(['ffmpeg',...` line. Any known work arounds? — zelusp, Jun 21 '17 at 15:59
Found a solution [here](https://stackoverflow.com/questions/40108816/python-running-as-windows-service-oserror-winerror-6-the-handle-is-invalid). Basically, just add `stdout=PIPE` as an extra argument to `Popen` — zelusp, Jun 21 '17 at 16:16

score 5 · Answer 2 · answered Mar 29 '19 at 14:06

imageio supports this directly. It uses FFMPEG and the Video Acceleration API, making it very fast:

import imageio

writer = imageio.get_writer('video.avi', fps=fps)
for i in range(frames_per_second * video_duration_seconds):
    img = createFrame(i)
    writer.append_data(img)
writer.close()

Can you "stream" images to ffmpeg to construct a video, instead of saving them to disk?

2 Answers2

Linked