CURTAIN: Towards a Foundation Model for Neutrino Telescopes

Neutrino telescopes record more than a trillion photon detections per year, yet there does not exist a \gls{ssl} task which fully satisfies all three conditions: improving as the pretraining set grows, transferring to a different detector, and outperforming the best model trained without pretraining at all. We introduce \textbf{\texttt{CURTAIN}}, a temporal masking scheme and the first \gls{ssl} task for neutrino telescopes that achieves all of these goals. Downstream performance monotonically improves with the number of pretraining events, as predicted by scaling laws. Our encoder builds on \texttt{DeepIce}, currently the leading model on the NuBench benchmarks direction reconstruction task. Pretraining it with \texttt{CURTAIN} reduces its median angular error by $45\,\%$ and its median energy error by $31\,\%$ compared to training from scratch. It also transfers to a detector geometry never seen in pretraining, reducing the median angular error by $36\,\%$ compared to training from scratch. At matched capacity and pretraining data, \texttt{CURTAIN}-pretrained encoders beat those pretrained by existing \gls{ssl} tasks on both tasks and at every pretraining set size.