Mathematical optimizations for deep learning
Yükleniyor...
Tarih
2018
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Springer International Publishing
Erişim Hakkı
info:eu-repo/semantics/openAccess
Özet
Deep neural networks are often computationally expensive, during both the training stage and inference stage. Training is always expensive, because back-propagation requires high-precision floating-pointmultiplication and addition. However, various mathematical optimizations may be employed to reduce the computational cost of inference. Optimized inference is important for reducing power consumption and latency and for increasing throughput. This chapter introduces the central approaches for optimizing deep neural network inference: pruning "unnecessary" weights, quantizing weights and inputs, sharing weights between layer units, compressing weights before transferring from main memory, distilling large high-performance models into smaller models, and decomposing convolutional filters to reduce multiply and accumulate operations. In this chapter, using a unified notation, we provide a mathematical and algorithmic description of the aforementioned deep neural network inference optimization methods. © Springer Nature Switzerland AG 2018.
Açıklama
Koç, Çetin Kaya (isu author)
Anahtar Kelimeler
Kaynak
Cyber-Physical Systems Security
WoS Q Değeri
Scopus Q Değeri
N/A
Cilt
Sayı
Künye
Green S., Vineyard C.M., Koç Ç.K. (2018) Mathematical Optimizations for Deep Learning. In: Koç Ç.K. (eds) Cyber-Physical Systems Security. Springer, Cham. https://doi.org/10.1007/978-3-319-98935-8_4
Green, S., Vineyard, C. M., & Koç, Ç. K. (2018). Mathematical Optimizations for Deep Learning. In Cyber-Physical Systems Security (pp. 69-92). Springer, Cham.
Green, S., Vineyard, C. M., & Koç, Ç. K. (2018). Mathematical Optimizations for Deep Learning. In Cyber-Physical Systems Security (pp. 69-92). Springer, Cham.
