1. Αποθετήριο Παραδοτέων Υποτρόφων Δράσεων ΕΣΠΑ Τριτοβάθμιας Εκπαίδευσης 2017-2023
2. Όλο το Αποθετήριο
3. Ερευνητικές ομάδες

Τύπος:	Άρθρο σε επιστημονικό περιοδικό
Τίτλος:	Probabilistic knowledge transfer for lightweight deep representation learning
Συγγραφέας:	[EL] Πασσαλής, Νικόλαος[EN] Passalis, Nikolaos [EL] Τζελέπη, Μαρία[EN] Tzelepi, Maria [EL] Τέφας, Αναστάσιος[EN] Tefas, Anastasios
Ημερομηνία:	01/06/2020
Περίληψη:	Knowledge-transfer (KT) methods allow for transferring the knowledge contained in a large deep learning model into a more lightweight and faster model. However, the vast majority of existing KT approaches are designed to handle mainly classification and detection tasks. This limits their performance on other tasks, such as representation/metric learning. To overcome this limitation, a novel probabilistic KT (PKT) method is proposed in this article. PKT is capable of transferring the knowledge into a smaller student model by keeping as much information as possible, as expressed through the teacher model. The ability of the proposed method to use different kernels for estimating the probability distribution of the teacher and student models, along with the different divergence metrics that can be used for transferring the knowledge, allows for easily adapting the proposed method to different applications. PKT outperforms several existing state-of-the-art KT techniques, while it is capable of providing new insights into KT by enabling several novel applications, as it is demonstrated through extensive experiments on several challenging data sets.
Γλώσσα:	Αγγλικά
Σελίδες:	10
DOI:	10.1109/TNNLS.2020.2995884
EISSN:	2162-2388
Θεματική κατηγορία:	[EL] Τεχνητή νοημοσύνη[EN] Artificial Intelligence
Λέξεις-κλειδιά:	Lighweight Deep Learning; Knowledge Transfer; Neural Network Distillation; Representation Learning; Metric Learning
Κάτοχος πνευματικών δικαιωμάτων:	© 2020, IEEE
Διατίθεται ανοιχτά στην τοποθεσία:	https://cidl.csd.auth.gr/resources/journal_pdfs/%CE%94.121.pdf
Ηλεκτρονική διεύθυνση του τεκμηρίου στον εκδότη:	https://ieeexplore.ieee.org/abstract/document/9104915
Ηλεκτρονική διεύθυνση περιοδικού:	https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=5962385
Τίτλος πηγής δημοσίευσης:	IEEE Transactions on Neural Networks and Learning Systems
Τεύχος:	5
Τόμος:	32
Σελίδες τεκμηρίου (στην πηγή):	2030 - 2039
Σημειώσεις:	This research is co-financed by Greece and the European Union (European Social FundESF) through the Operational Programme “Human Resources Development, Education and Lifelong Learning 2014-2020” in the context of the project “Lighweight Deep Learning Models for Signal and Information Analysis” (MIS 5047925).
Εμφανίζεται στις συλλογές:	Ερευνητικές ομάδες