小伍的游乐场【中文】论文速递|VCSD:无需教师模型的视觉对比自蒸馏,免费涨点(马里兰大学)
马里兰大学提出 VCSD,一种仅靠输入条件差分驱动的在线自蒸馏方法:让 EMA 教师分别在原图与内容抹除输入下预测,两者的逐 token 概率差揭示哪些候选真正由视觉内容支撑,以此作为蒸馏信号,无需外部教师或特权信息。在 6 个模型、7 个基准上稳定提升 1.76 至 5.33 分(如 Qwen3-VL-8B 从 72.51% 提至 76.26%),且推理时零额外开销。
【EN】Paper Brief | Visual Contrastive Self-Distillation: Free Gains Without an External Teacher (UMD)
UMD proposes VCSD, an on-policy self-distillation method driven purely by input conditioning: an EMA teacher predicts once with the original image and once with content-erased input, and the token-wise probability differences reveal which candidates are actually supported by visual evidence, forming a distillation signal that needs no external teacher or privileged information. It delivers consistent gains of +1.76 to +5.33 points across six models and seven benchmarks (e.g., Qwen3-VL-8B rises from 72.51% to 76.26%) with zero additional inference-time cost.
来源 Source:
留言
NO REGISTRATION · 临时网名 + 邮箱即可开聊