Catálogo de artículos
2 artículos
Self-supervised pretraining for label-efficient medical imaging
Annotating medical images is costly and requires expert radiologists. We evaluate a contrastive self-supervised pretraining scheme on chest radiographs and demonstrate that downstream classifiers reach expert-level AUROC with only 10% of the labels required by supervised baselines. We analyze failure modes on rare pathologies and propose a curriculum that mitigates them.
Attention sparsity for low-resource neural machine translation
Low-resource translation suffers from overfitting in standard transformer architectures. We introduce a structured attention-sparsity regularizer that constrains the effective context window during training and relaxes it at inference. Across eight low-resource language pairs, the method improves BLEU by an average of 2.4 points and reduces hallucination rates measured by a fact-consistency metric. Ablations isolate the contribution of head-level sparsity.