国产亚洲AV自拍|av中文字幕一区|资源在线观看一区二区|亚洲影视久久亚洲特级性交|一级做一级a做片爱免费观看|欧美另类亚洲色婷婷精品无码|亚洲青青草免费一区|青青草免费成人网|91久久国内视频|五月天丁香久久

    2024年伊始，團(tuán)隊(duì)論文“Learning from Human Educational Wisdom: A Student-Centered Knowledge Distillation Method ”被人工智能領(lǐng)域國(guó)際頂刊IEEE Transactions on Pattern Analysis and Machine Intelligence（CCF A，IF=23.6）錄用，學(xué)生為第一作者，本人為通訊作者，主要通過(guò)知識(shí)蒸餾技術(shù)使得輕量級(jí)小模型在智能手機(jī)、攝像頭、機(jī)器人和無(wú)人車（機(jī)）等資源受限的終端設(shè)備上具有云端大模型的性能。

    Abstract——Existing studies on knowledge distillation typically focus on teacher-centered methods, in which the teacher network is trained according to its own standards before transferring the learned knowledge to a student one. However, due to differences in network structure between the teacher and the student, the knowledge learned by the former may not be desired by the latter. Inspired by human educational wisdom, this paper proposes a Student-Centered Distillation (SCD) method that enables the teacher network to adjust its knowledge transfer according to the student network’s needs. We implemented SCD based on various human educational wisdom, e.g., the teacher network identified and learned the knowledge desired by the student network on the validation set, and then transferred it to the latter through the training set. To address the problems of current deficiency knowledge, hard sample learning and knowledge forgetting faced by a student network in the learning process, we introduce and improve Proportional-Integral-Derivative (PID) algorithms from automation fields to make them effective in identifying the current knowledge required by the student network. Furthermore, we propose a curriculum learning-based fuzzy strategy and apply it to the proposed PID control algorithm, such that the student network in SCD can actively pay attention to the learning of challenging samples after with certain knowledge. The overall performance of SCD is verified in multiple tasks by comparing it with state-of-the-art ones. Experimental results show that our student-centered distillation method outperforms existing teacher-centered ones.

    Keywords——Knowledge distillation, human educational wisdom, student-centered, fuzzy PID, curriculum learning