25

Can C# be used for developing a real-time application that involves taking input from web cam continuously and processing the input?

Nelson Rothermel
  • 8,471
  • 8
  • 56
  • 78
nishant
  • 2,383
  • 6
  • 35
  • 44
  • I'm not sure that as currently described, your app falls in the category of "real-time", as it would be unimportant if video were buffered when your app stalls. The only important issue would be to process the buffers faster than they fill up. – spender Sep 21 '10 at 16:01
  • 12
    Windows is not intended to be a "realtime" operating system. Consider the consequences of failure to process in real time: in your case, the failure mode is you miss a frame of video, probably not a big deal. A realtime system that, say, controls the positions of robot arms in a factory, or moves track switches on high speed rail lines, has very real human-life-safety implications if an operation goes over its time budget or control switches to another thread for a while. I don't think you actually need *realtime*; I think you just need *fast*. – Eric Lippert Sep 21 '10 at 16:10
  • 3
    @Eric: There is a distinction between soft real-time and hard real-time systems. Taking input from a web cam looks like a soft real-time scenario to me, and Windows should be fine. – Nemanja Trifunovic Sep 21 '10 at 20:28
  • @Nemanja: It is possible to do some measure of hard real-time with a Windows application (but not sure a CLR-based one). I worked on a C++ application that ran on Windows and was able to maintain real-time determinism to about 2ms - though we only counted on it to 10ms. We did all the RT from a separate RT critical thread that was locked into memory and was triggered using the Windows Multimedia Timer. You can get to full hard real-time (sub-ms) on a WIndows box by running a product like Venturcom RTX. – markshancock Oct 17 '13 at 22:42

5 Answers5

34

You cannot use any main stream garbage collected language for “hard real-time systems”, as the garbage collect will sometimes stop the system responding in a defined time. Avoiding allocating object can help, however you need a way to prove you are not creating any garbage and that the garbage collector will not kick in.

However most “real time” systems don’t in fact need to always respond within a hard time limit, so it all comes down do what you mean by “real time”.

Even when parts of the system needs to be “hard real time” often other large parts of the system like the UI don’t.

(I think your app needs to be fast rather than “real time”, if 1 frame is lost every 100 years how many people will get killed?)

Ian Ringrose
  • 49,271
  • 50
  • 203
  • 302
  • What about using this? Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.RealTime; – user8128167 Aug 25 '14 at 19:19
  • 1
    This should be the accepted answer!! +1 Garbage collection the objects makes a pause for a few milliseconds so the timing constraints will newer be fullfiled. – schoetbi Feb 16 '15 at 09:40
  • Unless your timing constraint is 1 second and the rest of your program runs in 500msec? Then you've got 500 spare milliseconds for garbage collection? If you're well below the timing constraint, does it actually matter that much? – Paul Oct 11 '15 at 17:47
  • “normal” garbage collectors don’t give a upper bound on how long they will stop your code running for. So they may be under 500msec the first 1000002 times they collect garbage, but then take a long time as they need to compact the heat due to fermentation. – Ian Ringrose Jul 12 '16 at 10:18
15

I've used C# to create multiple realtime, high speed, machine vision applications that run 24/7 and have moving machinery dependent on the application. If something goes wrong in the software, something immediately and visibly goes wrong in the real world.

I've found that C#/.Net provide pretty good functionality for doing so. As others have said, definitely stay on top of garbage collection. Break up to processing into several logical steps, and have separate threads working each. I've found the Producer Consumer programming model to work well for this, perhaps ConcurrentQueue for starters.

You could start with something like:

  • Thread 1 captures the camera image, converts it to some format, and puts it into an ImageQueue
  • Thread 2 consumes from the ImageQueue, processing the image and comes up with a data object that is put onto a ProcessedQueue
  • Thread 3 consumes from the ProcessedQueue and does something interesting with the results.

If Thread 2 takes too long, Threads 1 and 3 are still chugging along. If you have a multicore processor you'll be throwing more hardware at the math. You could also use several threads in place of any thread that I wrote above, although you'd have to take care of ordering the results manually.

Edit

After reading other peoples answers, you could probably argue my definition of "realtime". In my case, the computer produces targets that it sends to motion controllers which do the actual realtime motion. The motion controllers provide their own safety layers for things like timing, max/min ranges, smooth accel/decelerations and safety sensors. These controllers read sensors across an entire factory with a cycle time of less than 1ms.

Chiramisu
  • 4,451
  • 7
  • 42
  • 74
bufferz
  • 3,184
  • 4
  • 22
  • 35
  • What happens if your software is late processing an image once every few weeks, does the rest of the equipment just stop the production line as you catch up. – Ian Ringrose Jul 12 '16 at 10:20
8

Absolutely. The key will be to avoid garbage collection and memory management as much as possible. Try to avoid new-ing objects as much as possible, using buffers or object pools when you can.

David Pfeffer
  • 36,331
  • 28
  • 120
  • 198
  • can you please explain `avoid garbage collection & memory management` ? how does avoiding these help ? and how to avoid them :) ? – mrid Feb 28 '18 at 15:32
4
  • Of course, someone has even developed a library to do that: AForge.NET
  • As with any real-time application and not just C#, you'll have to manage the buffers well as @David suggested.
  • Not only that, there're also the XNA Framework (for things like 3D games) and you can program DirectX using C# as well which are very real-time.
  • And did you know that, if you want, you can do pointer manipulations in C# too?
chakrit
  • 57,172
  • 24
  • 125
  • 160
  • 1
    You know, Managed DirectX was dropped because it was too slow. – Puppy Sep 21 '10 at 20:34
  • that was a nice information. AForge.NET will be useful to me. thanks – Jsinh Sep 21 '10 at 20:38
  • [Here](http://www.mesta-automation.com/getting-started-with-computer-vision-in-c-sharp/) some useful documents exist to work with AForge framework. – GntS Dec 24 '16 at 10:08
1

It depends on how 'real-time' it needs to be; ie, what your timing constraints are, and how quickly you need to 'do something'.

If you can handle 'doing something' maybe every 300ms or so in .NET, say on a timer event, I've found Windows to work okay. Note that this is something I found true on multiple systems of different ages and different speeds. As always, YMMV.

But that number is awfully long for a lot of applications. Maybe not for yours.

Do some research, make sure your app responds quickly enough for your application.

J. Polfer
  • 11,281
  • 9
  • 52
  • 81