Scene-Adaptive Token Budgeting for Urban Traffic Perception

Abstract

Urban traffic perception systems must process complex visual scenes under changing weather, illumination, density, and object scale. Transformer-based perception models provide strong contextual modeling, but their computational cost limits real-time deployment on vehicle-grade hardware. This study explores scene-adaptive token budgeting for efficient urban traffic perception. We develop the Scene Token Budgeting Transformer (STBT), which estimates the required token budget from scene complexity indicators, including object density, lane-boundary uncertainty, foreground motion, and low-light degradation. Simple road scenes are processed with a reduced token budget, while crowded intersections, nighttime frames, and scenes with small vulnerable road users retain more visual tokens. The evaluation was organized around driving conditions rather than only dataset averages. Experiments used Cityscapes, BDD100K, and nuScenes camera data, with 196,000 annotated frames, 1.84 million object boxes, and 25 semantic road categories. In clear daytime road scenes, STBT reduced token computation by 46.2% with only a 0.3-point decrease in detection mAP. In rainy-night scenes, the model automatically retained 71.5% of foreground-related tokens and limited mAP loss to 1.1 points. On an NVIDIA Orin automotive platform, end-to-end perception latency decreased from 62.8 ms to 41.6 ms per frame. Pedestrian and cyclist recall at distances above 35 meters improved by 3.4 points compared with a fixed-ratio pruning strategy, mainly because the token budget was preserved for small foreground objects. The results indicate that scene-level token budgeting can support real-time traffic perception while maintaining safety-sensitive detection performance.