Gabriele Caslini

Overview

This Master Thesis project, carried out within the AIDA (Artificial Intelligence Driving Autonomous) research group at Politecnico di Milano, tackles the challenge of automatically generating regulation-aware HD maps for autonomous driving in urban environments. The core problem: autonomous vehicles depend on pre-constructed HD maps for localization and planning, but manually annotating traffic regulations is slow and error-prone.

Pipeline Architecture

The system is composed of four main stages that process recorded drives:

• Detection: A YOLOv11-M detector, trained on a custom dataset of 20,930 images (34,000+ annotations) spanning 31 traffic sign classes. Hyperparameters were optimized via Bayesian Optimization (20 runs, 30 epochs each) targeting mAP50-95. Training achieved mAP50 of 0.917, Recall of 0.841, Precision of 0.923.
• Tracking: BoT-SORT multi-object tracker with appearance re-identification and global motion compensation. A custom multi-camera mosaic approach fuses detections from 3 cameras with a selection layer to avoid duplicates.
• 3D Localization & Pose Estimation: LiDAR points are projected onto camera frames and filtered per bounding box. Background point removal via statistical filtering, followed by 3D pose estimation (normal vector extraction) for vertical signs and side refinement with ROI extraction for horizontal markings.
• HD Map Route Annotation: Detected signs are converted into regulatory Special Nodes (stop, yield, crosswalk, speed limit) placed at precise distances along the vehicle's route with a hierarchical fallback system that prioritizes horizontal markings over vertical signs.

Results

Validated on 14 km of Italian roads. Recall ranges from 93% to 100% across classes. In the fallback configuration, only 3 missed events out of 161 and 13 false positives. Processing time: 5 minutes per kilometer.

Upcoming publication: Camera-LiDAR Fusion for Traffic Sign and Road Marking Extraction for Autonomous Driving (Feb 2026, CCTA).

Overview

The “Goodnight Moon” challenge, organized by Reach Every Reader (Harvard, MIT & Florida State University), asks teams to build ML models that automatically score children's early literacy assessments from audio recordings. Children's speech introduces greater acoustic variability due to smaller vocal tracts and unpredictable pronunciations, making standard ASR systems unreliable. Accurate automated scoring enables faster early intervention for students who may benefit from additional literacy support.

Approach

• Main model - Whisper Transformer: A custom encoder-decoder Transformer that takes the expected text (character-level tokenizer, 40-token vocab) and pre-computed Whisper-base audio embeddings as inputs. Four interleaved self-attention / cross-attention layers (S-C-S-C pattern) progressively fuse text and audio representations. A learnable CLS token aggregates the final sequence into a single vector, fed through a LayerNorm + MLP head to predict a score in [0, 1]. Training uses AdamW with linear warmup + cosine decay, EMA weight averaging (decay ≈ 0.983), and Bayesian hyperparameter search via W&B.
• Contrastive model: Dual-branch architecture (Wav2Vec2 for audio, FastText for text) projecting both modalities into a shared 256-d space and scoring via cosine similarity.
• Phonetic Transformer: Variant replacing continuous Whisper embeddings with discrete phoneme tokens, enabling a symbolic text-vs-phoneme comparison through a shared embedding space.
• Experiment tracking: Full lifecycle managed with Weights & Biases sweeps and HuggingFace model hub.

Results

The team ranked 8th out of 399 teams in the international competition, contributing to making automated early literacy screening more accessible and scalable. The project was carried out with Epoch V during the Erasmus exchange at TU Delft.

Overview

The Polimi Sailing Team is Politecnico di Milano's competitive sailing team that designs and builds high-performance moth-class sailboats. The moth is a single-handed foiling dinghy that lifts entirely out of the water on hydrofoils, requiring precise real-time control to maintain stable flight.

Technical Contribution

• Electronic Architecture: Designed and integrated the entire onboard electronic system from scratch, including distributed sensor nodes and communication infrastructure.
• Automated Height Control: Developed the real-time control algorithm for stable foil flight, managing ride height and pitch stability at high speeds.
• Sensor Fusion: Implemented multi-sensor data fusion for accurate state estimation, combining inertial measurements, height sensors, and speed data.
• CAN Bus & Embedded: Built a CAN bus network for communication between distributed nodes, handled PCB integration, debugging, and waterproofing of all electronics.

Results

The team achieved 1st place at the SuMoth Challenge 2024, a competitive event where university teams race self-designed moth sailboats. The electronic control system was a key differentiator for the team's performance.