1. Αποθετήριο Παραδοτέων Υποτρόφων Δράσεων ΕΣΠΑ Τριτοβάθμιας Εκπαίδευσης 2017-2023
2. Όλο το Αποθετήριο
3. Ερευνητικές ομάδες

Τύπος:	Άρθρο σε επιστημονικό περιοδικό
Τίτλος:	Online Subclass Knowledge Distillation
Συγγραφέας:	[EL] Τζελέπη, Μαρία[EN] Tzelepi, Maria [EL] Πασσαλής, Νικόλαος[EN] Passalis, Nikolaos [EL] Τέφας, Αναστάσιος[EN] Tefas, Anastasios
Ημερομηνία:	04/05/2021
Περίληψη:	Knowledge Distillation has been established as a highly promising approach for training compact and faster models by transferring knowledge from more heavyweight and powerful models, so as to satisfy the computation and storage requirements of deploying state-of-the-art deep neural models on embedded systems. However, conventional knowledge distillation requires multiple stages of training rendering it a computationally and memory demanding procedure. In this paper, a novel single-stage self knowledge distillation method is proposed, namely Online Subclass Knowledge Distillation (OSKD), that aims at revealing the similarities inside classes, improving the performance of any deep neural model in an online manner. Hence, as opposed to existing online distillation methods, we are able to acquire further knowledge from the model itself, without building multiple identical models or using multiple models to teach each other, rendering the OSKD approach more effective. The experimental evaluation on five datasets indicates that the proposed method enhances the classification performance, while comparison results against existing online distillation methods validate the superiority of the proposed method.
Γλώσσα:	Αγγλικά
Σελίδες:	13
DOI:	10.1016/j.eswa.2021.115132
ISSN:	0957-4174
Θεματική κατηγορία:	[EL] Τεχνητή νοημοσύνη[EN] Artificial Intelligence
Λέξεις-κλειδιά:	Knowledge Distillation; Lightweight Deep Learning; Online distillation; Subclass knowledge distillation; Self distillation; Deep neural networks
Κάτοχος πνευματικών δικαιωμάτων:	© 2021 Elsevier Ltd. All rights reserved.
Διατίθεται ανοιχτά στην τοποθεσία:	https://cidl.csd.auth.gr/resources/journal_pdfs//Online%20Subclass%20Knowledge%20Distillation.pdf
Ηλεκτρονική διεύθυνση του τεκμηρίου στον εκδότη:	https://www.sciencedirect.com/science/article/abs/pii/S095741742100573X
Ηλεκτρονική διεύθυνση περιοδικού:	https://www.sciencedirect.com/journal/expert-systems-with-applications
Τίτλος πηγής δημοσίευσης:	Expert Systems with Applications
Τόμος:	181
Σελίδες τεκμηρίου (στην πηγή):	Article no 115132
Σημειώσεις:	This research is co-financed by Greece and the European Union (European Social Fund - ESF) through the Operational Programme “Human Resources Development, Education and Lifelong Learning 2014–2020” in the context of the project “Lightweight Deep Learning Models for Signal and Information Analysis” (MIS 5047925).
Εμφανίζεται στις συλλογές:	Ερευνητικές ομάδες