Researchers propose a self-distillation fix for ‘catastrophic forgetting’ in LLMs

LLMs tend to lose prior skills when fine-tuned for new tasks. A new self-distillation approach aims to reduce regression and ...
IEEE

SSD: Self-Supervised Distillation for Heterophilic Graph Representation Learning

Abstract: Graph Knowledge Distillation (GKD) has made remarkable progress in graph representation learning in recent years. Despite its great success, GKD often obeys the label-dependence manner, ...
IEEE

A Self-Supervised Learning Framework for Soft Robot Proprioception

Abstract: The inherent compliant nature of soft robots can offer remarkable advantages over their rigid counterparts in terms of safety to human users and adaptability in unstructured environments.