Παρακαλώ χρησιμοποιήστε αυτό το αναγνωριστικό για να παραπέμψετε ή να δημιουργήσετε σύνδεσμο προς αυτό το τεκμήριο:
https://hdl.handle.net/123456789/1121| Τύπος: | Άρθρο σε επιστημονικό περιοδικό |
| Τίτλος: | Geometric regularization of local activations for knowledge transfer in Convolutional Neural Networks |
| Συγγραφέας: | [EL] Θεοδωρακόπουλος, Ηλίας[EN] Theodorakopoulos, Ilias [EL] Φωτοπούλου, Φωτεινή[EN] Fotopoulou, Foteini [EL] Οικονόμου, Γεώργιος[EN] Economou, George |
| Ημερομηνία: | 19/08/2021 |
| Περίληψη: | In this work, we propose a mechanism for knowledge transfer between Convolutional Neural Networks via the geometric regularization of local features produced by the activations of convolutional layers. We formulate appropriate loss functions, driving a “student” model to adapt such that its local features exhibit similar geometrical characteristics to those of an “instructor” model, at corresponding layers. The investigated functions, inspired by manifold-to-manifold distance measures, are designed to compare the neighboring information inside the feature space of the involved activations without any restrictions in the features’ dimensionality, thus enabling knowledge transfer between different architectures. Experimental evidence demonstrates that the proposed technique is effective in different settings, including knowledge-transfer to smaller models, transfer between different deep architectures and harnessing knowledge from external data, producing models with increased accuracy compared to a typical training. Furthermore, results indicate that the presented method can work synergistically with methods such as knowledge distillation, further increasing the accuracy of the trained models. Finally, experiments on training with limited data show that a combined regularization scheme can achieve the same generalization as a non-regularized training with 50% of the data in the CIFAR-10 classification task. |
| Γλώσσα: | Αγγλικά |
| Σελίδες: | 22 |
| DOI: | 10.3390/info12080333 |
| EISSN: | 2078-2489 |
| Θεματική κατηγορία: | [EL] Επιστήμη ηλεκτρονικών υπολογιστών και Πληροφορική, άλλοι τομείς[EN] Computer and Information sciences, miscellaneous |
| Λέξεις-κλειδιά: | manifold regularization; knowledge transfer; knowledge distillation; deep learning with limited data |
| Κάτοχος πνευματικών δικαιωμάτων: | © 2021 by the authors. Licensee MDPI, Basel, Switzerland. |
| Όροι και προϋποθέσεις δικαιωμάτων: | This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). |
| Ηλεκτρονική διεύθυνση του τεκμηρίου στον εκδότη: | https://www.mdpi.com/2078-2489/12/8/333 |
| Ηλεκτρονική διεύθυνση περιοδικού: | https://www.mdpi.com/journal/information |
| Τίτλος πηγής δημοσίευσης: | Information |
| Τεύχος: | 8 |
| Τόμος: | 12 |
| Σελίδες τεκμηρίου (στην πηγή): | Article no 333 |
| Σημειώσεις: | This research is co-financed by Greece and the European Union (European Social Fund— ESF) through the Operational Programme “Human Resources Development, Education and Lifelong Learning 2014–2020” in the context of the project “New knowledge-transfer and regularization techniques for training Convolutional Neural Networks with limited data” (MIS 5047164). |
| Εμφανίζεται στις συλλογές: | Ερευνητικές ομάδες |
Αρχεία σε αυτό το τεκμήριο:
Το πλήρες κείμενο αυτού του τεκμηρίου δεν διατίθεται προς το παρόν από το αποθετήριο