convert gstreamer pipeline to opencv in python

Question

I have created a network stream with following gstreamer commands:

sender:

gst-launch-1.0 -v videotestsrc ! video/x-raw,framerate=20/1 ! videoscale ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! rtph264pay ! udpsink host=X.X.X.X port=5000

receiver:

gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink

This just works fine. I now want to include the stream on the receiver side in a python script. In the script I want to do some video processing with opencv.

Does anyone know how to convert the described pipeline, so that it can be used with opencv?

Thanks!

edit1:

Found out that this should work:

cap = cv2.VideoCapture("udpsrc port=5000 ! application/x- rtp,media=video,payload=26,clock-rate=90000,encoding-name=H264, payload=96 ! rtph264depay ! decodebin ! videoconvert ! appsink", cv2.CAP_GSTREAMER)

I get following error:

OpenCV Error: Assertion failed (size.width>0 && size.height>0) in imshow, file /home/nvidia/build-opencv/opencv/modules/highgui/src/window.cpp, line 331 Traceback (most recent call last): File "launchstream_ip.py", line 13, in <module> cv2.imshow('frame', frame) cv2.error: /home/nvidia/build-opencv/opencv/modules/highgui/src/window.cpp:331: error: (-215) size.width>0 && size.height>0 in function imshow

At the sender you posted X.X.X.X where must be the IP or host for the local machine, to use opencv try: `stream = cv2.VideoCapture("rtp://X.X.X.X:5000")` — Newton Pasqualini Filho, Jul 26 '18 at 14:01
Thank you for your replay. I get this error message: `[rtp @ 0x7a6460] Unable to receive RTP payload type 96 without an SDP file describing it` — Julius Morandell, Jul 26 '18 at 14:10
You could achieve this task with libValkka as well. See: http://www.valkka.fi — El Sampsa, Jul 29 '18 at 08:45

score 6 · Accepted Answer · answered Jul 26 '18 at 15:54

import numpy as np
import cv2
from multiprocessing import Process

def send():
    cap_send = cv2.VideoCapture('videotestsrc ! video/x-raw,framerate=20/1 ! videoscale ! videoconvert ! appsink', cv2.CAP_GSTREAMER)
    out_send = cv2.VideoWriter('appsrc ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000',cv2.CAP_GSTREAMER,0, 20, (320,240), True)

    if not cap_send.isOpened() or not out_send.isOpened():
        print('VideoCapture or VideoWriter not opened')
        exit(0)

    while True:
        ret,frame = cap_send.read()

        if not ret:
            print('empty frame')
            break

        out_send.write(frame)

        cv2.imshow('send', frame)
        if cv2.waitKey(1)&0xFF == ord('q'):
            break

    cap_send.release()
    out_send.release()

def receive():
    cap_receive = cv2.VideoCapture('udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! appsink', cv2.CAP_GSTREAMER)

    if not cap_receive.isOpened():
        print('VideoCapture not opened')
        exit(0)

    while True:
        ret,frame = cap_receive.read()

        if not ret:
            print('empty frame')
            break

        cv2.imshow('receive', frame)
        if cv2.waitKey(1)&0xFF == ord('q'):
            break

    cap_receive.release()

if __name__ == '__main__':
    s = Process(target=send)
    r = Process(target=receive)
    s.start()
    r.start()
    s.join()
    r.join()

    cv2.destroyAllWindows()

convert gstreamer pipeline to opencv in python

1 Answers1

Linked