Access MPEG transport packets via .NET

Question

I have to deal with MPEG 2 transport packets using .NET. What is the best way to do that? Currently I'm considering using OpenCV to accomplish that, but not sure yet if it's possible.

Answer 1

NOTE: I'm assuming you are using windows since you want to target .NET.
I have done so in the past. As far a I know there is no complete .NET available source code for you to use.

EDIT: OpenCV will not help you with your task. You can use OpenCV to display video (It uses FFMPEG internally) But you will not have access to the packets. Also hacking the FFMPEG library supplied with OpenCV is not easily done since on Windows it will be pre-compiled.

The path to go actually depends on your needs. However if you need to work at the packet level you will have to learn the MPEG2 TS specification. Wikipedia is a good place to start but eventually you will have to read the specification itself iso13818-1 and optionally iso13818-2. You can find copies of it around the net - just google it. You can find some reference implementations in C/C++ VLC, FFMPEG, libmpeg gstreamer (in the bad plugins) however I can assure you they are hard to read and not very well documented. Also writing a complete and robust MPEG TS muxer or demuxer is a hard task which requires tedious examination of the documentation. There is .NET tool called "MPEG-2 Transport Stream packet analyzer" written in .NET it looks like a complete implementation however the code is not freely available - maybe the author my be willing to sell it to you. You can access it from http://www.pjdaniel.org.uk/mpeg/

Depending on your C/C++ and programming skills I will recommend one of the following options:

1. NO C/C++ Skills but very high programming skills Or need to do just some basic work with the packets:
   Read the docs implement exactly what you need
2. Good C/C++ skills and the patience to compile FFMPEG with MinGW and read other people's code:
   Take FFMPEG (libavcodec) and look into the MpegTS implementation write your hooks into it and export plain C functions you can interop with .NET

I would recommend the second option unless you need to do remuxing or other kinds of serious manipulations on the bitstream itself

You should notice that given the complexity of the TS protocol it is easier to manipulate using C/C++ (Which was what I did in the end after starting with C#) and inerop it with .NET.

I had to write my own demuxer and muxer for a certain project that had very specific needs. It was not an easy task (The entire thing took me about 300 hours to implement correctly) and in the end the result was not as robust as a commercial muxer or demuxer from Elecard or MainConcept - However the off the shelf products would not do what we needed. I wrote them in C++ - used DirectShow (In C++) to write a source filter, decoded using Elecard (which worked better than MainConcept at the time) and wrote my own Renderer to display the actual video. The entire DirectShow chain was controlled from C# using interop.

Once you have selected you path you should then make some other decisions depending on what you do with the packets. If you want to send them to a decoder or a multiplexer then you can use DirectShow for that. You will have to put what ever you do into a source filter, transform filter or destination filter depending on where you receive the data. If you want to implement your filter in .NET you can use the "Pure .NET DirectShow Filters in C#" By Maxim Kartavenkov form http://www.codeproject.com/Articles/421167/Pure-NET-DirectShow-Filters-in-Csharp. (Or buy the Elecard .NET SDK if you need commercial support). There are some reference filters to get you started although you will have to read the DirectShow documentation as well. If you just look at the packets, maybe change them and write them back then it's possible you can write your own clean implementation for that or hack into the mpegts implementation of libavcodec it's not so complicated just hours of fun figuring out what is going on - Very instructive though. libavcodec has a very clean interface so you can easily get the changed packets back - You will have to read the docs for that as well though.

So, I'm not sure that's the answer you wanted but there is no easy path for what you want.

Answer 2

http://net7mma.codeplex.com has a class for reading them among other types of mpeg and non mpeg container formats.

All ts variations are also correctly supported including atsc.

(De) Muxing from one to another will also be ssupported soon.

Answer 3

Here's a .NET example processing HTTP Live Streaming playlists (.m3u8 files). It downloads .ts files in the playlist into memory and uses NAudio library to analyze and extract the audio stream.

Answer 4

You might find the traditional DirectShow filter development route more useful, for example using this component pack.

Answer 5

SDP files are amazing. I don't know if you need to handle this at a packet level, but I usually script out an SDP file. All the media players I have tested with--Windows Media, VLC, Quicktime-- supports them. Then if you need to embed the media into a web page, form, or into a recording this can be easily achieved from the apis for the respective player.

The SDP file will pass the container, encoding, and networking information necessary for the player to grab the stream itself.

I spent a ton of time playing around with direct show, until I realized that Windows Media player is a pretty damn good implementation and it will create all of your filters based on the SDP file. It is pretty easy. Here is the spec:

http://www.ietf.org/rfc/rfc4566.txt

VLC also has a really good api for this kind of thing.

Answer 6

I am not sure if I understand your question completely, if you are doing .NET and capturing MPEG packets and then need to do some processing, OpenCV would be fine but Would suggest to use Aforge.NET This will help you avoid writing interops for OpenCV. It contains a broad collection of video processing libraries and should be useful for you.