Later when I concentrated on more serious work like developing high quality video filters, I rediscovered the colormatrix formalism. Now what is that all about and what's so exciting about that? If you assume each pixel to be a vector of color channels, multiplying the vector by a matrix is simply a linear transformation of each pixel. In homogenous coordinates and in RGBA or YUVA (i.e. 4 channel) colorspace, it can be completely described by a 4x5 matrix. Now there are lots of commonly used video filters, which can be described by a colormatrix multiplication:
|[||c||0||0||0||b - (1+c)/2||]|
Saturation/Hue rotation (Y'CbCrA)
Invert single RGB channels (RGBA)
Swap RGB channels (RGBA)
RGB gain (RGBA)
gr, gg and gb are in the range 0.0..2.0.
There are of course countless other filters possible, like generating an alpha channel from (inverted) luminance values etc. Now, what do you do if you want to make e.g. a brightness/contrast filter working on RGBA images? The naive method is to transform the RGBA values to Y'CbCrA, do the filtering and transform back to RGBA. And this is what can be optimized in a very elegant way by using some simple matrix arithmetics. Instead of performing 2 colorspace conversions in addition to the actual filter for each pixel, you can simply transform the colormatrix (M) from Y'CbCrA to RGBA:
|Untransformed pixel in RGBA:||(p)|
|Untransformed pixel in Y'CbCrA:||(RGBA->Y'CbCrA)*(p)|
|Transformed pixel in Y'CbCrA:||(M)*(RGBA->Y'CbCrA)*(p)|
|Transformed pixel in RGBA:||(Y'CbCrA->RGBA)*(M)*(RGBA->Y'CbCrA)*(p)|
(M)RGBA = (Y'CbCrA->RGBA)*(M)Y'CbCrA*(RGBA->Y'CbCrA)
If you have generic number-crunching routines, which to the vector-matrix multiplication for all supported pixelformats, you can reduce conversion overhead in your processing pipeline significantly.
Another trick is to combine multiple transformations in one matrix. The gmerlin video equalizer filter does this for brightness/contrast/saturation/hue rotation.
A number of gmerlin filters use the colormatrix method. Of course, for native pixelformats (image colorspace = matrix colorspace), the transformation is done directly, since it usually needs much less operations than a generic matrix-vector multiplication. But for foreign colorspaces, the colormatrix is transformed to the image colorspace like described above.
Some final remarks:
- The colormatrix multiplication needs all channels for each pixel. Therefore it doesn't work with subsampled chroma planes. An exception is the brightness/contrast/saturation/hue filter, because luminance and chroma operations are completely separated here.
- For integer pixelformats the floating point matrix is converted to an integer matrix, where the actual ranges/offsets for Y'CbCr are taken into account.
- In practically all cases, the color values can over- or underflow. The processing routines must do proper clipping.
Fantastic work man, keep your heads high you did it.
Post a Comment