Taraneh Younesian (TU Delft)*; Zilong Zhao (Delft University of Technology); Amirmasoud Ghiassi (TU Delft); Robert Birke (ABB Research); Lydia Chen (TU Delft)PMLR Page
Abstract
Noisy labeled data is more a norm than a rarity for self-generated content that is continuously published on the web and social media from non-experts.
Active querying experts are conventionally adopted to provide labels for the informative samples which don't have labels, instead of possibly incorrect labels. The new challenge that arises here is how to discern the informative and noisy labels which benefit from expert cleaning.
In this paper, we aim to leverage the stringent oracle budget to robustly maximize learning accuracy. We propose a noise-aware active learning framework, QActor, and a novel measure CENT, which considers both cross-entropy and entropy to select informative and noisy labels for an expert cleansing. QActor iteratively cleans samples via quality models and actively querying an expert on those noisy yet informative samples.
To adapt to learning capacity per iteration, QActor dynamically adjusts the query limit according to the learning loss for each learning iteration. We extensively evaluate different image datasets with noise label ratios ranging between 30% and 60%.
Our results show that QActor can nearly match the optimal accuracy achieved using only clean data at the cost of only an additional 10% of ground truth data from the oracle.