Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Choosing useful ways to assess spatial analytics results

Why evaluation matters

A result is not finished just because a model produced output. You still need to check whether that output actually supports your research question, communicate how good it is, and be honest about where it falls short. Evaluation is what turns a prediction into evidence you can use in a report.

Evaluation is project-specific

There is no single metric that fits every project. The right evaluation approach depends on your task type, your data, your method, and what validation information you actually have available. A metric that is standard for one task can be meaningless, or actively misleading, for another. Use the sections below to identify a starting point for your specific task, not a fixed requirement.

Evaluation by task type

Classification

Image or tabular classification assigns observations to discrete classes.

Useful evaluation tools include:

MeasureMain question
AccuracyWhat proportion of all predictions is correct?
PrecisionWhen the model predicts a class, how often is it correct?
RecallHow much of the true class does the model find?
F1-scoreHow well are precision and recall balanced?
Confusion MatrixWhich classes are being confused?

Accuracy can be misleading when classes are strongly imbalanced.

For example, if 95% of samples belong to one class, a model that nearly always predicts that class can have high accuracy while performing poorly on the minority class.

A confusion matrix helps reveal this problem because it shows the pattern of class-to-class errors.

Object detection

Object Detection evaluates both:

  1. what was detected,

  2. where it was detected.

A prediction is usually considered a True Positive when:

A False Positive is an unmatched or incorrect detection. A False Negative is a reference object that the model missed.

Bounding-box overlap

Intersection over Union is often abbreviated as IoU.

IoU compares the overlap between predicted and reference boxes:

IoU=predicted positive pixels∩reference positive pixelspredicted positive pixels∪reference positive pixels\text{IoU} = \frac{\text{predicted positive pixels} \cap \text{reference positive pixels}}{\text{predicted positive pixels} \cup \text{reference positive pixels}}
(1)

Higher IoU means stronger spatial agreement.

Precision and recall

Detection precision asks:

Of the objects predicted by the model, how many are correct?

Recall asks:

Of the real objects in the reference data, how many did the model find?

The confidence threshold creates a trade-off:

lower threshold
→ more predictions
→ often higher recall
→ potentially more false positives

higher threshold
→ fewer predictions
→ potentially higher precision
→ potentially more false negatives

The appropriate threshold depends on the application.

For example, a screening workflow followed by manual inspection may tolerate more false positives than a workflow that automatically counts objects.

Average precision

Average Precision summarizes the precision-recall relationship for one class across confidence thresholds.

Mean Average Precision averages performance across classes and, depending on the convention, may also summarize performance across several IoU thresholds.

Do not report mAP without explaining which definition or threshold convention you used.

Semantic segmentation

Semantic Segmentation assigns a class label to every pixel.

Evaluation therefore compares predicted and reference masks at pixel level.

Common tools include:

For a binary mask:

IoU=predicted positive pixels∩reference positive pixelspredicted positive pixels∪reference positive pixels\text{IoU} = \frac{\text{predicted positive pixels} \cap \text{reference positive pixels}}{\text{predicted positive pixels} \cup \text{reference positive pixels}}
(2)

IoU is particularly informative because it measures spatial overlap directly.

Multi-class segmentation

For several classes, overall performance may hide a weak minority class.

Check, where possible:

This matters when the project contains Class Imbalance.

A land-cover model that performs very well on common tree-canopy pixels but poorly on a rare wetland class may still have a high aggregate score.

Instance segmentation

Instance Segmentation combines object detection with pixel-level masks.

Evaluation can therefore consider two related questions:

  1. Was the individual object detected?

  2. How accurately does its predicted mask represent the object?

Useful checks can include:

For spatial projects, pay particular attention to:

The metric you emphasize should reflect whether your research question depends mainly on counts, locations, boundaries or some combination of them.

Change detection

Change Detection usually compares observations from different times.

For binary change/no-change output, familiar classification measures can be used:

But change detection has additional spatial and temporal challenges.

False change

A predicted change can result from:

Missed change

A true change may be missed because it is:

Visual comparison of the input dates remains useful because an aggregate metric cannot explain why a particular area was labelled as changed.

Also consider whether the two dates are genuinely comparable. Good model evaluation cannot repair inappropriate temporal input data.

Regression

Regression predicts continuous values.

Examples might include:

Mean absolute error

Mean Absolute Error is often abbreviated as MAE.

MAE measures the average absolute prediction error:

MAE=1n∑i=1n∣yi−y^i∣\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} \vert{}y_i - \hat{y}_i\vert{}
(3)

Because it uses the same unit as the target, it is usually easy to interpret.

Root mean squared error

Root Mean Squared Error is often abbreviated as RMSE.

RMSE gives relatively more weight to larger errors.

RMSE=1n∑i=1n(yi−y^i)2\text{RMSE} = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2}
(4)

Comparing MAE and RMSE can help reveal whether a relatively small number of large errors are important.

R-squared

R-squared is commonly written as (R^2).

It describes how much of the target variance is explained by the model.

Do not interpret a high (R^2) as proof that predictions are unbiased or spatially well distributed.

Residuals

A Residual is:

reference value - predicted value

Residual plots and maps can reveal patterns hidden by one summary number.

Look for:

Foundation models and embeddings

Foundation Model workflows do not always produce a direct prediction that can be evaluated with one standard metric.

Evaluation should match the downstream use.

Ask whether nearby items in Embedding Space are actually similar in ways relevant to your question.

Useful checks include:

Clustering

Clustering is unsupervised, so a cluster is not automatically a meaningful land-cover or environmental class.

Evaluate whether:

Lightweight classification

If embeddings are used as features for a classifier, evaluate the classifier using appropriate classification metrics and a defensible train/test strategy.

Zero-shot or prompted outputs

For a zero-shot model, qualitative inspection may be an important first step.

If the output becomes central to a quantitative conclusion, strengthen the evaluation with representative reference samples where feasible.

Spatial evaluation concerns

Spatial data raises evaluation issues that do not show up in the same way with non-spatial data:

Choosing an evaluation strategy

Work through this checklist for your own project:

Common pitfalls

Mini task

Choose one evaluation strategy for your own project, and write two or three sentences explaining why it fits your specific question, data, and validation situation. If you are unsure between two approaches, state what would help you decide.

Key takeaways

Spatial Data Analytics
Cheatsheets
Spatial Data Analytics
Common errors