A real USGS 3DEP LiDAR tile and a cloud-free Sentinel-2 scene, fused into per-pixel features (canopy/building height, NDVI, terrain slope, texture), trained against real OpenStreetMap building footprints, and evaluated at both the pixel level and the polygon level, over a dense rowhouse block on Capitol Hill, Washington DC.
Every layer below is resampled onto the same 2m grid in EPSG:32618 (UTM 18N). nDSM comes from the LiDAR point cloud (DSM minus DTM); NDVI comes from Sentinel-2's native 10m red/NIR bands upsampled onto the 2m grid, which is why NDVI looks visibly blockier than the LiDAR-derived layers, an honest illustration of the spatial-resolution tradeoff between the two sensors.
Two naive single-feature baselines, evaluated on the exact same held-out test region as the full model, make the case with real numbers rather than a marketing claim:
| Model | Precision | Recall | F1 | IoU |
|---|---|---|---|---|
| NDVI-only (low NDVI → guess building) | 77.3% | 43.7% | 55.8% | 38.7% |
| nDSM-only (height > 2.5m → guess building) | 35.7% | 99.7% | 52.5% | 35.6% |
| RandomForest, nDSM+NDVI+slope+texture | 79.3% | 80.2% | 79.7% | 66.3% |
NDVI-only is precise but misses most buildings (roads, parking lots, and bare soil also read as "not vegetation"). nDSM-only catches almost everything raised off the ground (99.7% recall) but drags in every tree along the way, cratering precision. Only the combined model gets both right.
Pixel metrics look strong (F1 79.7%), but a GIS reviewer cares about individual buildings, not pixels. Vectorizing the prediction and matching it against individual OSM building polygons in the test region tells a more honest story:
| Metric | Value |
|---|---|
| Ground-truth buildings in test region | 74 |
| Predicted building polygons | 72 |
| Matched at IoU ≥ 0.3 | 33 |
| Building-level detection rate | 44.6% |
| Mean matched-polygon IoU | 0.61 |
| Mean building area error | 36.1% |
| Mean centroid displacement | 6.5 m |
Three real misclassifications from the held-out test region, with Nova Lite explaining each one from its actual measured nDSM/NDVI/slope values, not a generic answer:
The false positive likely occurred due to the relatively low normalized surface height (nDSM) of 9.00 m, which might be near the classifier's threshold for distinguishing buildings from non-buildings. Additionally, the low NDVI value of 0.21 suggests sparse vegetation, which could cause the classifier to misinterpret the area as built-up rather than natural terrain. The gentle terrain slope of 1.0 degree further supports this, as it is unlikely to cause significant misclassification on its own.
The false negative is likely due to the moderate NDVI value of 0.65, indicating a significant presence of vegetation which may have confused the classifier, making it difficult to distinguish the building from the surrounding vegetation. Additionally, the terrain slope of 7.7 degrees might have contributed to the error by causing distortions in the LiDAR data, further complicating the classification.
The false negative classification is likely due to the moderate normalized surface height (nDSM) of 12.85 m, which may be near the threshold for building detection, combined with a relatively low NDVI of 0.43 indicating moderate vegetation cover that could obscure building features. Additionally, the gentle terrain slope of 1.6 degrees might reduce the distinctness of building structures in the LiDAR data.
GIS and remote sensing is a domain where an AI can sound confident and still be subtly wrong, so the pipeline turns that lens on itself: 30 domain questions put to Amazon Nova Lite, with the ones this project has direct ground truth for (DSM/DTM/nDSM, LiDAR returns) graded against that ground truth rather than taken at face value.
A geographic coordinate system (GCS) uses latitude and longitude to define locations on the Earth's surface in a three-dimensional space, referenced to the ellipsoid model of the Earth. In contrast, a projected coordinate system (PCS) transforms the curved surface of the Earth into a flat, two-dimensional surface using specific projections, which minimizes distortion for specific areas or purposes.
EPSG:4326 (WGS84) coordinates represent geographic latitude and longitude, which are angular measurements and not linear. Therefore, distances and areas calculated directly from these coordinates would be inaccurate because they do not account for the Earth's curvature and the distortion that occurs when projecting a spherical or ellipsoidal surface onto a flat plane.
The fundamental difference between raster and vector spatial data models lies in their representation of geographic data: raster data uses a grid of cells to store values (e.g., elevation, temperature), while vector data employs points, lines, and polygons to represent discrete objects. Vector data is preferred for precise boundary representation and attribute-rich information, whereas raster data is better suited for continuous data and seamless integration with imagery.
A Cloud-Optimized GeoTIFF (COG) differs from a regular GeoTIFF by incorporating tiled structure and efficient compression, such as JPEG2000 or DEFLATE, which allows for faster data access and streaming, significantly enhancing performance in cloud-based analysis by enabling rapid subsetting and parallel processing. This optimization is crucial for handling large datasets efficiently in cloud environments.
NDVI (Normalized Difference Vegetation Index) is calculated using the formula (NIR - Red) / (NIR + Red), where NIR is the near-infrared and Red is the visible red light reflected from vegetation. A value close to zero typically indicates non-vegetative surfaces like water or bare soil, while a value close to one indicates dense, healthy vegetation.
SAR imagery offers the advantages of all-weather and day-night imaging capabilities, providing high-resolution data that is effective through cloud cover and darkness. It is also sensitive to surface roughness and moisture content, which is beneficial for applications like land cover classification and monitoring. However, one major drawback is its relatively complex data interpretation and processing requirements compared to optical imagery.
Speckle noise in SAR imagery arises from coherent interference of backscattered signals from various scatterers within a resolution cell. It is typically mitigated using filters such as Lee, Frost, or wavelet-based methods, which smooth the image while preserving edges.
A Digital Terrain Model (DTM) represents the bare earth surface, excluding any objects like buildings or vegetation. A Digital Surface Model (DSM) includes all surfaces, such as buildings and vegetation, providing a comprehensive view of the Earth's surface. A Digital Elevation Model (DEM) is a type of DTM that specifically quantifies the elevation of the terrain.
A normalized Digital Surface Model (nDSM) is a raster dataset that separates ground and non-ground surface elements by subtracting a Digital Terrain Model (DTM) from a Digital Surface Model (DSM). It is computed by performing a pixel-wise subtraction of the DTM from the DSM, resulting in a model that highlights the height of above-ground features such as buildings and vegetation.
Orthorectification is the process of correcting aerial or satellite imagery to account for sensor movement and terrain-induced distortions, ensuring that the imagery accurately represents the Earth's surface in a uniform projection. It is a necessary step before mosaicking because it aligns and standardizes the geometric integrity of individual image tiles, which is crucial for creating a seamless and accurate composite image.
Photogrammetric structure-from-motion (SfM) uses overlapping images to triangulate point positions, creating a dense point cloud and subsequently a digital surface model (DSM), while LiDAR employs laser pulses to directly measure distances to the Earth's surface, providing a more accurate and direct representation of the terrain. SfM is generally less expensive but can be less precise compared to LiDAR.
Spatial resolution refers to the smallest object that can be detected on the Earth's surface, measured in meters per pixel. Temporal resolution indicates how frequently a satellite revisits the same location to capture data, typically in days. Spectral resolution denotes the sensor's ability to distinguish between different wavelengths of light, influencing the detail in the spectral information obtained.
A sensor with coarser spatial resolution but finer temporal resolution might be preferred for agricultural monitoring to capture frequent changes in crop conditions and growth stages over time, which is crucial for timely decision-making in crop management. This allows for more frequent updates on crop health, stress detection, and growth patterns, despite the lower detail in individual observations.
Semantic segmentation classifies each pixel in an image into predefined categories, such as "building" or "non-building," without distinguishing between individual objects, while instance segmentation not only classifies pixels but also differentiates between multiple instances of the same category, enabling the delineation of individual building footprints.
Pixel-level accuracy is insufficient for building extraction models because it does not account for the spatial arrangement and connectivity of pixels that form buildings. Instead, polygon-level metrics such as overall accuracy, precision, recall, and the F1 score should be used to evaluate how well the extracted building polygons match the reference data.
A spatial join is a GIS operation that combines attributes from one feature layer to another based on their spatial relationship. An "intersects" join identifies features that overlap or touch each other, while a "within" join identifies features that are completely contained within another feature.
In GIS, topology refers to the arrangement of geographic features in relation to one another, defining how they connect, adjacency, and proximity. A common topological error in a building footprint dataset is "overlaps," where two or more polygons intersect, indicating that the same area is incorrectly assigned to multiple buildings.
A spatial index like an R-tree is necessary for efficient geometric queries on large vector datasets because it significantly reduces the search space by organizing spatial data in a hierarchical structure, allowing for faster query response times through spatial partitioning and minimizing the number of data comparisons required.
A datum shift refers to the difference between two geodetic datums, which can cause positional discrepancies when converting coordinates. Even though NAD83 and WGS84 are both widely used datums, converting between them can introduce a small positional error due to their distinct reference points and ellipsoids, leading to potential misalignments of up to a few meters.
The classification field in a LAS/LAZ point cloud typically encodes the type of surface each point represents, such as ground, vegetation, or building. Ground classification is crucial before deriving a Digital Terrain Model (DTM) because it ensures that only ground points are used, thereby eliminating noise from non-ground features and improving the accuracy and reliability of the DTM.
First-return LiDAR points represent the first instance a laser pulse hits a surface, often capturing the canopy top for vegetation, while last-return points denote the final interaction, typically reaching the ground or building surface. This distinction is crucial as first-returns are more suitable for modeling vegetation canopies, whereas last-returns are essential for accurately modeling building surfaces, ensuring precise elevation data for different surface types.
'Points per square meter' in a LiDAR dataset refers to the number of laser pulses that hit a specific area, typically measured in square meters. Higher point density increases the likelihood of capturing detailed surface information, allowing for the reliable extraction of smaller buildings by ensuring sufficient data points to accurately delineate their dimensions.
Combining SAR or LiDAR height data with optical spectral data enhances building extraction by leveraging the strengths of each: SAR and LiDAR provide accurate elevation and structural information, while optical data offers detailed surface texture and material properties. This synergy allows for more precise differentiation between buildings and other land cover types.
Intersection over Union (IoU) is a metric used to evaluate the accuracy of object detection models by measuring the overlap between the predicted bounding box and the ground truth bounding box. A commonly used IoU threshold to consider an object detection a true positive is 0.5, meaning at least 50% overlap is required.
A spatial train/test split groups contiguous geographic blocks for training and testing, preserving spatial autocorrelation, which is crucial for capturing local patterns and dependencies in the data. In contrast, a random pixel split may include spatially correlated pixels in both training and test sets, potentially leading to overoptimistic accuracy estimates. This matters because it affects the generalizability of the classifier to unseen areas.
UTM is commonly used for local-area raster analysis because it minimizes distortion within a limited region, providing high accuracy for measurements and spatial analysis. Unlike global geographic CRS, UTM offers consistent scale and shape across smaller areas, which is crucial for precise local-scale applications.
GeoJSON is an open standard format for encoding a variety of geographic data structures in JSON, facilitating the use of vector data in web applications. One limitation of GeoJSON for very large vector datasets is its lack of support for efficient spatial indexing, which can result in slow query performance and high memory usage when handling extensive spatial data.
To detect new construction between two time periods, I would use multi-temporal optical imagery to identify changes in land cover and land use, then overlay the LiDAR-derived normalized Digital Surface Model (nDSM) to detect height differences indicative of new structures. This combination allows for accurate identification and verification of new constructions.
Terrain slope is typically derived from a Digital Terrain Model (DTM) by calculating the rate of change in elevation over a specified area. One common algorithm used for this is the finite difference method, which estimates slope by analyzing the elevation differences between a central cell and its neighboring cells.
Building extraction from SAR imagery poses additional challenges due to layover and shadow effects, where layover occurs when topography causes distant objects to appear closer, and shadows result from obstruction by buildings, both of which can distort the radar signal and complicate accurate building delineation. These effects are less prevalent in optical imagery, which relies on visible light and is less influenced by such geometric distortions.
Full pipeline, methodology, and limitations write-up: github.com/fborbon/gis-lidar. Longer narrative: Field Notes write-up.