Video Processing Tutorial Using C#/VS.NET

Summary: How to quickly create a C# .NET application that performs image processing in real time on a file or live video stream using the Cogitance VideoCog .NET component. Duration: 20 minutes. Requisites: Visual Studio .NET 2005, Cogitance VideoCog .NET Component. Download: TennisTutorial.zip

Tutorial Outline: This tutorial contains the following steps:

1. Add the VideoCog Control to the Visual Studio Toolbox.
2. Add the VideoCog Control to Your Application's Form.
3. Stream Video from a File.
4. Process Video in the FrameReady() Event.
5. Process Images with the DVizion Image Processing Library.
6. Summary

Step 1: Add the VideoCog Control to the Visual Studio Toolbox

1.1 Create a new C# Windows Application project In Visual Studio .NET 2005.

1.2 Select "Choose Items..." from the Toolbox context menu:

1.3 From the "Choose Toolbox Items" dialog, select "Browse...":

1.4 From the VideoCog <installdir>\bin subdirectory, select VideoCog.dll and click the "Open" button :

1.5 The Cogitance VideoCogControl component has now been added to the ".NET Framework Components" list:

1.6 Select "OK" from the "Choose Toolbox Items" dialog. The control is now added to your toolbox and ready to use.

Step 1 is complete. The VideoCog control has now been added to your VS.NET 2005 Toolbox.

Step 2: Add the VideoCog Control to Your Application's Form

The VideoCog is a licensed control. At this point it will look for a license file in the same folder as the VideoCog.dll, which should have been sent to you when you purchased or downloaded a trial version of the product. If you need a trial license, please email Cogitance Techincal Support.

2.2 VideoCog properties are listed in the Visual Studio "Properties" Window. The "Video" section contains all properties specific to the VideoCog control. Most properties can be set either at "Design-Time" using the property editor, or at "Run-Time" programmatically.

Please refer to the VideoCog Reference Manual for a complete description of each property.

Step 2 is complete. The VideoCog control has now been added to your Windows Form and is ready to use.

Step 3: Stream Video from a File.

3.1 Let's begin adding code (C#) to our application.

The following code sets the video source to a file and begins playing it (media file in tutorial download package):

     private void Form1_Load(object sender, EventArgs e)
     {    
          videoCogControl1.Streams[0].Source = "tennis.wmv";
          videoCogControl1.Play();
     }

Note: To use a camera instead of a file, the following code will find the first camera connected to the host machine, and begin playing it upon application startup:

     private void Form1_Load(object sender, EventArgs e)
     {
          Collections.ArrayList cameras = videoCogControl1.EnumerateCameras();
          if (cameras.Count < 1)
          {
               Windows.Forms.MessageBox.Show("No cameras detected. Exiting.");
               return;
          }

          videoCogControl1.Streams[0].Source = cameras[0].ToString();
          videoCogControl1.Play(); 
     }

3.2 Build and execute the application (F5).

Step 3 is complete. The application is now streaming data from a file (or camera).

Step 4: Process Video in the FrameReady() Event.

4.1 Add the FrameReady event handler:

From the VS.Net "Properties" window, select the "Events" toolbar button. Scroll down to the Video section of the event list, and double-click on the FrameReady event to generate the videoCogControl1_FrameReady code entry point. The FrameReady event is the function the VideoCog will call for each video frame it processes.

4.2 Add the following code to the videoCogControl1_FrameReady event:

     private void videoCogControl1_FrameReady(double SampleTime, IntPtr pBuffer,
                                         int BufferSize, int Width, int Height)
     {
          unsafe
          {
               byte* p = (byte*)(void*)pBuffer;
               int nWidth = Width * 3;
               for (int y = 0; y < Height; y++)
               {
                    for (int x = 0; x < nWidth; x++)
                    {
                        p[0] = (byte)(255 - p[0]);
                        p++;
                    }
               }
           }	
     }							

The above code inverts the video image, pixel by pixel. This code uses pointers to access the image data. Pointers are considered unsafe in .NET, so for this code to compile, you will need to allow unsafe code in your project settings:

4.3 Enable frame grabbing on the VideoCog control. Before the event handler will be called, you must specifically enable the frame grabber for the specified stream:

     private void Form1_Load(object sender, EventArgs e)
     {   
          videoCogControl1.Streams[0].Source = "tennis.wmv";
          videoCogControl1.Streams[0].GrabberEnabled = true;
          videoCogControl1.Play();
     }

Frame grabbing slows down video performance considerably, so by default this property is set to false. You should only set the FrameGrabber property to true if you intend to process the video images.

4.4 Build and execute the application (F5). The video image should now appear inverted:

Step 4 is complete. The application is now processing individual video images.

Step 5: Process images with the DVizion image processing library.

5.1 The inversion code we did above was pretty fast. However the routine was relatively simple, and it was also unsafe. Since we now have a basic image buffer, we can instead integrate our application with safer, more complex image processing routines.

The VideoCog ships with DVizion, our simple image processing library specifically designed for real-time video processing. DVizion uses the Intel OpenComputer Vision library for many of its routines for fast performance, along with additional routines for image and text overlays.

5.2 To use the DVizion image processing library, we will first need to add a reference to it. From the VS.NET 2005 toolbar, select Project -> Add Reference:

5.3 From the "Add Reference" dialog, select the "Browse" tab, and browse to the VideoCog <installdir>\Bin folder:

5.4 Select the DVizion.dll and press "OK". This will add the DVizion library to your application.

5.5 Add the following variables the application:

     public partial class Form1 : Form
     {
          
          Cogitance.DVizion.dvImageM dvImage = new Cogitance.DVizion.dvImageM();
          Cogitance.DVizion.fltMMotion dvMotion =
                                             new Cogitance.DVizion.fltMMotion();
          
          public Form1()
          ...
          ...
     					

     private void videoCogControl1_FrameReady(double SampleTime, IntPtr pBuffer,
                                          int BufferSize, int Width, int Height)
     {
          byte[] Color = { 255, 255, 0, 0 };

          dvImage.Initialize(pBuffer, Width, Height);
          dvMotion.DrawRects = true;
          dvMotion.DrawContours = true;
          dvMotion.Apply(pBuffer, Width, Height);

          string nContours = "Contours Counted : " + dvMotion.ContourCount;
          string nPixels = "Pixels Changed : " + dvMotion.ChangedPixelCount;
          dvImage.GDIBegin();
          dvImage.GDIText(5, 0, nContours, "Times New Roman", 16, Color);
          dvImage.GDIText(5, 15, nPixels, "Times New Roman", 16, Color);
          dvImage.GDIEnd();		
     }							

Step 6: Summary

We've seen how easy it is to quickly get started processing digital video by using Microsoft Visual Studio 2005, C#, and the Cogitance VideoCog control. However, this tutorial has only scratched the surface. When processing digital video, there are many issues to consider, including:

• Camera location, movement, and lighting
• Object tracking
• Disk usage and image resolution tradeoffs
• Processor performance and multi-threading options
• Algorithm selection

To help with these issues, there are a multitude of open-souce and commercial image processing libraries available that can be quickly integrated with the VideoCog, including libraries from Intel and (OpenCV) and Microsoft (Microsoft Vision SDK). For examples, refer to the VideoCog Samples folder.

Good luck, and happy coding.
The Cogitance Team

Questions? Comments? Suggestions for improving this tutorial? Please don't hesitate - tell us! Email us at support@cogitance.com.