Segmentation

Gemminis analyses sidewalks and roads from footage recorded by camera-equipped vehicles, and segmentation is what turns each frame into road, sidewalk, curb, poles, signs and the rest of the street. I trained and fine-tuned that model: EoMT on a DINOv3 backbone, adapted with LoRA, locally and in Google Colab.

Most of the gain came from data. I built training sets from the company’s own footage, with SAM 3 proposing each region’s class, SAM 2 refining its edges and every frame corrected by hand, and I evaluated as carefully as I trained: checkpoints compared on video as well as on metrics, IoU read class by class, and validation kept representative of the footage the model is meant for.

I then took the model from semantic to panoptic segmentation, where every object instance is kept separate: instance-level labels, EoMT’s native loss with Hungarian matching, and panoptic quality as the measure.

Defect detection

Gemminis also detects defects in sidewalks and roads, such as cracks, potholes, sinkage and broken curbs. I worked on the data and evaluation behind those detectors: measuring how far their scores can be trusted, building balanced sets of true and false positives to retrain from, labelling severity, and preparing data for a vision-language model that validates each detection in context. I also compared and retrained object detection models, using stereo depth to study how confidence changes with distance.

computer-visionsegmentationfine-tuning