Παρακαλώ χρησιμοποιήστε αυτό το αναγνωριστικό για να παραπέμψετε ή να δημιουργήσετε σύνδεσμο προς αυτό το τεκμήριο:
https://hdl.handle.net/123456789/1120| Τύπος: | Πρακτικά συνεδρίου |
| Τίτλος: | Local manifold regularization for knowledge transfer in Convolutional Neural Networks |
| Συγγραφέας: | [EL] Θεοδωρακόπουλος, Ηλίας[EN] Theodorakopoulos, Ilias [EL] Φωτοπούλου, Φωτεινή[EN] Fotopoulou, Foteini [EL] Οικονόμου, Γεώργιος[EN] Economou, George |
| Ημερομηνία: | 15/07/2020 |
| Περίληψη: | In this work we present a method for local manifold-based regularization, as a mechanism for knowledge transfer during training of Convolutional Neural Networks. The proposed method aims at regularizing local features produced in intermediate layers of a “student” CNN through an appropriate loss function that encourages the model to adapt such that the local features to exhibit similar geometrical characteristics to those of an “instructor” model, at corresponding layers. To that purpose we formulate a computationally efficient function, loosely encoding the neighboring information in the feature space of the involved feature sets. Experimental evaluation demonstrates the effectiveness of the proposed scheme under various scenarios involving knowledge-transfer, even for difficult tasks where it proves more efficient than the established technique of knowledge distillation. We demonstrate that the presented regularization scheme, utilized in combination with distillation improves the performance of both techniques in most tested configurations. Furthermore, experiments on training with limited data, demonstrate that a combined regularization scheme can achieve the same generalization as an un-regularized training with 50% of the data. |
| Γλώσσα: | Αγγλικά |
| Σελίδες: | 8 |
| DOI: | 10.1109/IISA50023.2020.9284400 |
| ISBN: | 978-1-6654-2228-4 |
| Θεματική κατηγορία: | [EL] Επιστήμη ηλεκτρονικών υπολογιστών και Πληροφορική, άλλοι τομείς[EN] Computer and Information sciences, miscellaneous |
| Λέξεις-κλειδιά: | Deep Learning with Limited Data; Manifold Regularization; Knowledge Transfer; Knowledge Distillation |
| Κάτοχος πνευματικών δικαιωμάτων: | ©2020 IEEE |
| Ηλεκτρονική διεύθυνση του τεκμηρίου στον εκδότη: | https://ieeexplore.ieee.org/document/9284400 |
| Ηλεκτρονική διεύθυνση περιοδικού: | https://ieeexplore.ieee.org/xpl/conhome/9284145/proceeding |
| Τίτλος πηγής δημοσίευσης: | 2020 11th International Conference on Information, Intelligence, Systems and Applications (IISA) Proceedings |
| Όνομα εκδήλωσης: | 2020 11th International Conference on Information, Intelligence, Systems and Applications (IISA) |
| Τοποθεσία εκδήλωσης: | Piraeus, Greece |
| Ημ/νία έναρξης εκδήλωσης: | 15/07/2020 |
| Ημ/νία λήξης εκδήλωσης: | 17/07/2020 |
| Σημειώσεις: | This research is co-financed by Greece and the European Union (European Social Fund- ESF) through the Operational Programme «Human Resources Development, Education and Lifelong Learning 2014-2020» in the context of the project “New knowledge-transfer and regularization techniques for training Convolutional Neural Networks with limited data” (MIS 5047164). |
| Εμφανίζεται στις συλλογές: | Ερευνητικές ομάδες |
Αρχεία σε αυτό το τεκμήριο:
Το πλήρες κείμενο αυτού του τεκμηρίου δεν διατίθεται προς το παρόν από το αποθετήριο