Refining Dynamic Data Selection for Self-Supervised Learning via Semantic Gating

Aaltodoc (Aalto University)

Abstract

Self-supervised learning (SSL) can learn visual representations without class labels during pretraining, but repeatedly processing large data pools is computationally expensive. Dynamic Data Selection via Coarse-to-Fine Refinement (DySCoF) reduces this cost by prioritizing samples with high proxy-estimated optimization gain. This thesis studies a limitation of gain-only selection: corrupted or out-of-distribution (OOD) noisy samples can also receive high gain scores and remain in subsets selected by DySCoF. The study diagnoses this behavior in noisy settings, then uses a label-dependent nearest-neighbour support score only as oracle analysis, and finally introduces Semantic-Gated DySCoF (SG-DySCoF), a refinement based on cosine similarity to neighbours in the representation space. The method is evaluated on CIFAR-10, CIFAR-100, EuroSAT, and STL-10 across blur and OOD noise settings. SG-DySCoF produced modest but consistent accuracy improvements in the matched-seed CIFAR and STL-10 experiments, while a larger gain was observed in long-tailed EuroSAT setting. reductions in noise retention are generally larger than the accuracy improvements, suggesting that cleaner subset composition does not translate proportionally into higher downstream accuracy. The results overall suggest that semantic gating can provide a lightweight refinement to gain-based dynamic selection in noisy settings, though its effect varies with dataset structure, noise type,and representation quality.

Authors 1

1. Xinrui Peng corresponding Aachen Collaborative partner

   RWTH Aachen University

   Affiliation as printed

   Collaborative partner RWTH Aachen Chair of Information Theory and Data Analytics