Abstract

Multi-task learning has shown large benefits in Natural Language Understanding (NLU). However, current state-of-the-arts (SOTAs) like MT-DNN and MMoE do not model task relationships explicitly and fail to obtain effective task alignment. In this paper, we propose a Collaborative Multi-Task Representation (CMTR) framework to tackle this problem. We capture instance-level task relations through a task interaction layer, which helps guide the fusion of task-oriented representations into the final representation. Moreover, tailored loss functions are proposed to facilitate the learning of task alignment. Specifically, we leverage knowledge distillation as an auxiliary loss to assist the adaptation layers in generating task-oriented representations. We also introduce a regularization loss to learn better gating functions for multi-task fusion. Empirically, CMTR outperforms SOTA multi-task learning frameworks on most natural …

Date

2024

Authors

Yaming Yang, Defu Cao, Ming Zeng, Jing Yu, Xiang Ren, Yunhai Tong, Yujing Wang

Conference

2024 International Joint Conference on Neural Networks (IJCNN)

Pages

1-8

Publisher

IEEE