Segmentation concepts

1. Motivation¶

All the segmentation functions that you will use in this lesson — training, inference and evaluation — operate on masks with specific shape and value conventions. Getting comfortable with these conventions now will save you time later.

2. Core idea¶

A segmentation model performs image-to-mask prediction: given an input image with height H and width W, it outputs a raster of the same H and W, where every pixel holds a predicted class index. This is fundamentally different from classification (one label for the whole image) and detection (a variable-length list of boxes); segmentation always produces exactly one prediction per pixel, with no separate step for describing objects or entire scenes.

3. Working with masks¶

A. Binary and multi-class masks¶

A binary mask has two possible pixel values: typically 0 for background and 1 for the target class (building, water, and so on).

0 = background  (not tree)
1 = target   (tree)

A multi-class mask extends this to more than two values, one integer per class, for example 0 through 12 for a 13-class land-cover scheme, with 0 conventionally reserved for background or “no data.”

For example:

0 = background
1 = water
2 = forest
3 = cropland
4 = impervious surface

B. Probability maps¶

In addition to a hard class prediction, a segmentation model can also produce a probability map: a multi-band raster where each band corresponds to one class, and pixel values range from 0 to 1, representing the model’s confidence that a given pixel belongs to that class. This is the pixel-level equivalent of the confidence score you used per-detection in L06.

For a two-class example:

pixel:
background → 0.08
tree   → 0.92

A probability map is useful for spotting uncertain regions, for example along class boundaries, and for setting a custom confidence threshold rather than accepting the model’s default hard classification.

C. Sliding-window inference¶

Real rasters are almost always larger than a model’s expected input size. Tiled inference (also called sliding-window inference in this context) solves this the same way it did in L06: a fixed-size window slides across the full raster with some overlap between neighboring windows, the model predicts each window independently, and the overlapping predictions are stitched into one seamless output raster. The overlap smooths out any inconsistency at window edges, the same purpose it served for detection tiling.

D. Raster predictions and vector outputs¶

A model’s direct output is always a raster: a grid of class values aligned with the input imagery. For many downstream uses, especially GIS workflows, vector polygons are more useful, for example to compute an area, join with other vector layers, or filter by shape. Converting a raster mask into vector polygons is a separate, deliberate post-processing step, not something the model does automatically; you will practice this conversion on the binary-workflow page. Vectorization does not make the original prediction more accurate. It only changes its representation.

4. Python reactivation¶

A mask, once loaded, is a NumPy array, the same data structure you have used for raster bands in earlier lessons. array.shape tells you its height and width; numpy.unique(array) lists the distinct values actually present, which is exactly the quick check recommended above before training on a new dataset.

You can count target pixels with a Boolean expression:

building_pixels = (mask == 1).sum()

building_pixels

The same logic scales to large raster masks.

For a multi-class mask:

classes, counts = np.unique(mask, return_counts=True)

list(zip(classes, counts))

This combines familiar array operations with the output of a deep-learning model.

5. Common pitfalls¶

• Assuming mask values without checking them. Different datasets use different conventions (0/1, 0/255, or arbitrary integers); verify before training rather than assuming.

• Confusing a probability map with the final class mask. The class mask is a single-band raster of class indices; the probability map is a separate, multi-band raster of per-class confidence. They serve different purposes and are not interchangeable.

• Treating tiled inference edges as guaranteed to be perfect. Overlap reduces boundary artifacts but does not eliminate them entirely; visually check a few tile boundaries in your output before trusting it fully.

• Forgetting that raster-to-vector conversion is a choice, not a default. Decide early whether your project deliverable needs vector output, since it changes what post-processing steps you plan for.

6. Mini task¶

A colleague shows you a segmentation mask array with array.shape == (512, 512) and numpy.unique(array) returning [0, 1, 2, 3, 4]. Without seeing the imagery, what can you conclude about the task, and what would you ask before trusting the mask?

7. Key takeaways¶

• A segmentation model outputs one class prediction per pixel, in a raster matching the input’s height and width, unlike a single scene label or a list of boxes.

• Binary masks use two values; multi-class masks use one integer per class, generally with 0 reserved for background.

• A probability map stores per-class confidence at every pixel and is distinct from the final hard class mask.

• Sliding-window (tiled) inference handles rasters larger than the model’s input size, the same principle used for detection in L06.

• Converting raster predictions to vector polygons is a deliberate post-processing step, not an automatic part of inference.