A Uncertainty-Calibrated Transformer for Long-Horizon Forecasting with Missing and Irregular Observations

Abstract

The proliferation of Internet of Things (IoT) devices and distributed sensor networks has resulted in an explosion of time-series data, yet the utility of this data is frequently compromised by irregularities, such as missing observations, non-uniform sampling rates, and sensor failures. While Transformer-based architectures have established a new state-of-the-art in sequence modeling, standard implementations rely implicitly on fixed-interval discrete time steps, rendering them suboptimal for irregular temporal environments. Furthermore, long-horizon forecasting inherently involves accumulating errors, necessitating robust uncertainty quantification to support downstream decision-making processes. This paper introduces the Uncertainty-Calibrated Continuous Transformer (UCCT), a novel architecture designed to address these dual challenges simultaneously. We propose a continuous-time positional encoding mechanism that leverages the temporal timestamps directly, decoupling the model from the rigid index-based sequence assumption. Additionally, we integrate a probabilistic decoding head that outputs distributional parameters rather than point estimates, calibrated via a composite loss function balancing accuracy and aleatoric uncertainty estimation. Extensive experiments on real-world datasets, including energy consumption, meteorology, and healthcare telemetry, demonstrate that the proposed UCCT outperforms current deterministic and stochastic baselines. Specifically, the model achieves a reduction in Mean Squared Error by approximately 14% in scenarios with 50% missing data, while providing reliable confidence intervals that accurately capture the increasing variance over long forecast horizons.