March 2007 - Image Processing
Real-time High-resolution GPU Chroma Keying
We were contacted by a company that needed to accelerate one of their applications. The application processes 8MP (megapixel) images of people in front of a green screen and applies a multi-point chroma key filter. This removes the green background so they can composite the person onto any other background image. Multi-point chroma key is an expensive operation, requiring multiple per-pixel calculations. When dealing with such high-resolution images, performance is never going to be great using the CPU. Even on modern processors, the original version of the application took upwards of 16 seconds to process a single image. This is not ideal when you have other dependent processes that are held up, or when you need to tweak the chroma key parameters and re-process the image multiple times to get a perfect result.
We produced a version for our client that runs on the GPU instead of the CPU, allowing the filter to run in real-time. This means that the effects of changing the chroma key parameters can be seen as you make them, instead of having to wait 16+ seconds for each change. For purposes of comparison, to multi-point chroma key a full 8MP image now takes 0.15 seconds - over a hundred times faster.
In addition to chroma key, we also produced a real-time filter for adjusting the levels of an image (the white and black points). It would be possible for our client to include further filters such as contrast, hue, saturation, RGB levels and others with no noticeable impact on performance.
This enhancement has been included in an application used in major sporting events during 2007.
Technical Details
GPU ChromaKey was written in C# using the .NET framework. We used Managed DirectX 9 to provide a context for custom pixel shaders written in HLSL. We had to convert the contents of the back buffer directly to a GDI-compatible bitmap in the fastest way possible. The work was integrated with our client's existing image processing interface. Development was completed in a week.
GPU Image Processing SDK - Available Now
If you are looking to include similar functionality in your own applications, we have available a faster, more comprehensive library for processing high resolution images and video on the GPU.
An early version of the SDK being used in a simple chroma key application
Please contact us for further details.
