A Multimodal Driver Drowsiness and Distraction Monitoring System: Design and Preliminary Evaluation

Abstract

Driver drowsiness and distraction have long been
recognised as major contributors to road-traffic crashes, and lowcost
driver-monitoring prototypes have frequently been restricted
to a single visual cue such as eye closure. In this work, a
multimodal, webcam-based driver-monitoring system has been
designed and preliminarily evaluated. Five complementary visual
signals—Eye Aspect Ratio (EAR), Percentage of Eye Closure
(PERCLOS), Mouth Aspect Ratio (MAR), head pose, and upperbody
posture—have been extracted in real time, and facial
and body landmarks have been obtained using MediaPipe.
The features have been normalised and combined through a
transparent weighted-linear fusion model, from which a Composite
Drowsiness Score has been derived and mapped to four
graduated alert levels. A driver-specific calibration stage has been
incorporated so that personalised baselines are established before
monitoring begins. Three alert-classification configurations—a
rule-based fusion system, a MobileNetV2 classifier, and a stacked
ensemble—have been compared on a subject-independent split
of the Driver Monitoring Dataset over 3,605 held-out frames.
Overall accuracies of about 46%, 57%, and 54% have been
obtained, macro ROC-AUC values of up to 0.69 have been
recorded, and an ablation study of the five cues has been reported.
The most severe alert category has been found difficult to classify
because of class imbalance and a face-only camera view. It
has therefore been concluded that the interpretable multimodal
pipeline is technically feasible, while systematic evaluation on
public benchmarks has been identified as the principal remaining
work.