Pixels, Channels, dtypes

By the end of this session you will be able to:

• Read an image's shape and dtype out loud and say what every number in them means
• Index a single pixel and a single channel, and convert between OpenCV's BGR and the RGB that every other library expects
• Predict whether an arithmetic operation on a uint8 image will wrap around or clamp, and pick the one you actually meant

An image is a rectangle of numbers

When you load a photograph, nothing image-like arrives. What arrives is a NumPy array, and nearly every bug you hit in this course is a question about that array's shape and dtype.

import numpy as np
import cv2
 
print(cv2.__version__)   # 5.0.0
print(np.__version__)    # 2.5.1
 
img = np.zeros((480, 640, 3), dtype=np.uint8)
print(img.shape)   # (480, 640, 3)
print(img.dtype)   # uint8
 
img[10, 20] = (0, 0, 255)   # indexing is [row, column], so [y, x]
print(img[10, 20])          # [  0   0 255]   one pixel, three channels
print(img[10, 20, 2])       # 255             one channel of one pixel
print(img[0:5, 0:5].shape)  # (5, 5, 3)       a 5x5 crop

Read (480, 640, 3) left to right: 480 rows, 640 columns, 3 channels. Height first. That ordering catches almost everyone, because every camera spec sheet and every file dialog says "640 by 480", width first. NumPy says the opposite, and it is not being awkward: an array is a list of rows, so the row count has to come first.

The practical consequence is that OpenCV functions taking a size want (width, height), while the array reports (height, width). So cv2.resize(img, (320, 240)) returns shape (240, 320, 3). Two orderings, one line of code.

A grayscale image has no third number at all. Its shape is (480, 640), so len(img.shape) == 2 is how you ask "is this grayscale", which beats trusting a variable named gray to be telling the truth.

OpenCV puts blue first

That (0, 0, 255) above is red, not blue. OpenCV orders channels blue, green, red.

The reason is history, not design. The camera and codec vendors OpenCV was built around in the late 1990s laid pixels out in that byte order, and by the time anyone objected, too much code depended on it.

Nothing breaks while the image stays inside OpenCV. imread, imwrite and everything between agree on BGR, so a load, process, save round trip is self-consistent and you can go a long time without noticing. It breaks the moment the array leaves. Matplotlib, Pillow, torchvision, every pretrained model and every annotation tool assume RGB. Hand them BGR and the sky comes out orange and skin comes out blue. Worse, a model fed BGR does not crash: it quietly loses accuracy while you blame your dataset.

Convert explicitly, at the boundary:

import numpy as np
import cv2
 
bgr = np.zeros((2, 2, 3), dtype=np.uint8)
bgr[:, :] = (0, 0, 255)             # red, written in BGR order
 
rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
print(bgr[0, 0])   # [  0   0 255]
print(rgb[0, 0])   # [255   0   0]

cvtColor returns a new array and leaves the original alone. The rule worth adopting today: treat BGR as an OpenCV-internal detail, and convert on the exact line where the array crosses out of OpenCV, not "somewhere near the top" of the file.

cv2.COLOR_BGR2GRAY collapses the channel axis entirely, and it is a weighted sum rather than an average, because your eye is far more sensitive to green than to blue. Pure green becomes 150, pure red 76, pure blue only 29.

uint8 stops at 255, then starts again at 0

uint8 is an unsigned 8-bit integer: whole numbers 0 to 255, one byte per channel per pixel. It is the default because it is what sensors produce and what files store, and because a 12 megapixel photo is already 36 MB.

What matters is the edge. Brighten an image the obvious way:

import numpy as np
 
img = np.full((2, 2, 3), 250, dtype=np.uint8)
print((img + 60)[0, 0])   # [54 54 54]

250 plus 60 is 310, there is no room for 310 in a byte, so it wraps: 310 minus 256 is 54. Your highlights become your darkest pixels. NumPy does this to arrays silently. No exception, no warning, just black holes punched through every bright region.

It does warn on a lone scalar, which is a confusing inconsistency to meet at 2am:

import numpy as np
 
print(np.uint8(250) + 10)   # 4, with RuntimeWarning: overflow encountered in scalar add
np.uint8(300)               # OverflowError: Python integer 300 out of bounds for uint8

Subtraction wraps the other way and is nastier, because the result looks plausible instead of broken. np.uint8(5) - np.uint8(10) is 251: a near-white pixel where you expected black. Every hand-rolled frame difference hits this.

Averaging hits it before you even divide:

import numpy as np
 
a = np.full((2, 2), 200, dtype=np.uint8)
b = np.full((2, 2), 100, dtype=np.uint8)
 
print(((a + b) // 2)[0, 0])                          # 22, wrong
print(((a.astype(np.uint16) + b) // 2)[0, 0])        # 150, right

The sum overflowed to 44 before the division ever ran.

Saturate, or leave uint8 entirely

Two fixes, for two situations.

OpenCV's arithmetic functions clamp at 0 and 255 instead of wrapping. Use these when the result is still an image:

import numpy as np
import cv2
 
img = np.full((2, 2, 3), 250, dtype=np.uint8)
print(cv2.add(img, 60)[0, 0])        # [255 255 255]
 
dark = np.full((2, 2), 5, dtype=np.uint8)
other = np.full((2, 2), 10, dtype=np.uint8)
print(cv2.subtract(dark, 10)[0, 0])  # 0,  clamped
print((dark - other)[0, 0])          # 251, wrapped
print(cv2.absdiff(dark, other)[0, 0])  # 5, magnitude, never wraps

cv2.absdiff is the one you want whenever you difference two images, because it gives magnitude and cannot go negative.

The second fix is to stop doing intermediate maths in uint8 at all. Once a multiply or a chain of steps is involved, go to float, finish, then clamp and come back:

import numpy as np
 
img = np.full((2, 2, 3), 250, dtype=np.uint8)
 
work = img.astype(np.float32) * 1.2 + 10             # 310.0, no ceiling in float32
print(np.clip(work, 0, 255).astype(np.uint8)[0, 0])  # [255 255 255]
print(work.astype(np.uint8)[0, 0])                   # [54 54 54], clipped too late

Note the order. astype truncates modulo 256, so clipping after the cast clips numbers that are already wrong.

Try it

Build a 4 by 6 red image, brighten it both ways, and count the damage.

import numpy as np
import cv2
 
img = np.zeros((4, 6, 3), dtype=np.uint8)
img[:, :, 2] = 255                      # channel 2 is red
 
print(img.shape, img.dtype)             # (4, 6, 3) uint8
print(img[0, 0])                        # [  0   0 255]
 
wrapped = img + 60
clamped = cv2.add(img, 60)
 
print(wrapped[0, 0])                    # ?
print(clamped[0, 0])                    # ?
print("pixels that got darker:", int(np.count_nonzero(wrapped < img)))

You succeed when you can state, before running the last line, exactly which number it prints and why. The answer is 24, one per pixel, and it is the red channel every time: 255 plus 60 wraps to 59, while blue and green climb harmlessly from 0 to 60. Now change 255 to 180 and predict the count again before running it.

Common mistakes

• Reading shape as width by height. img.shape is (height, width, channels), but cv2.resize and cv2.rectangle take (width, height). Mixing them gives a transposed image or an index error on the wrong axis.
• Converting BGR to RGB an even number of times. COLOR_BGR2RGB is its own inverse, so a stray second call puts you back in BGR with no error. Convert once, where the array leaves OpenCV.
• Brightening with +. Plain + on uint8 arrays wraps in silence and destroys exactly the highlights you were lifting. Use cv2.add, or go through float32 and np.clip.
• Clipping after the cast. np.clip(x, 0, 255).astype(np.uint8) is correct; np.clip(x.astype(np.uint8), 0, 255) is not, because the wraparound already happened inside astype.
• Assuming three uint8 channels. A PNG with transparency read using cv2.IMREAD_UNCHANGED has shape (h, w, 4), and a 16-bit depth scan comes back uint16 ranging to 65535. Both sail through code that assumes (h, w, 3) and 255. Print shape and dtype before trusting either.

Where this goes next

Next session, "Loading Things That Fail", you stop conjuring arrays with np.zeros and start reading real files, where cv2.imread hands back None instead of raising, imshow dies on a headless machine, and every frame needs checking before you index it.