Many CCD and CMOS cameras capture raw images in Bayer patterns, one of them may look like:
| G |
B |
G |
B |
G |
B |
| R |
G |
R |
G |
R |
G |
| G |
B |
G |
B |
G |
B |
| R |
G |
R |
G |
R |
G |
Each pixel just marks one of the RGB channels as a u8 or u16. Other channels are interpolated from adjacent pixels.
OpenCV provides a group of flags to convert a Bayer image to an RGB or Grayscale one, the specific algorithm are implemented in opencv/modules/imgproc/src/demosaicing.cpp.