Auxiliary learning of non-monotonic hyperparameter scheduling system via grid search

Yükleniyor...
Küçük Resim

Tarih

2022

Dergi Başlığı

Dergi ISSN

Cilt Başlığı

Yayıncı

Erişim Hakkı

info:eu-repo/semantics/openAccess

Özet

Recent advancements in advanced neural networks have given rise to new adaptive learning strategies. Conventional learning strategies suffer from many issues, such as slow convergence and lack of robustness. To fully exploit its potential, these issues must be resolved. Both issues are related to the step-size, and momentum term, which is generally fixed and remains uniform for all weights associated with each network layer. In this study, the recently published Back-Propagation Algorithm with Variable Adaptive Momentum (BPVAM) algorithm has been proposed to overcome these issues and improve effectiveness for classification. The study was conducted on various hyperparameters based on the grid search approach, then the optimal values of hyperparameters have trained these algorithms. Six cases were considered with varying values of the hyperparameter to evaluate the impact of the hyperparameter on the training models. It is empirically proven that the convergence behavior of the model is improved in terms of the mean and standard deviation for accuracy and the sum of squared error (SSE). A comprehensive set of experiments indicated that the BPVAM is a robust and highly efficient algorithm.

Açıklama

Anahtar Kelimeler

Uyarlanabilir Sinir Ağları, Hiperparametre, Kararlı Durum Hatası, Optimizasyon

Kaynak

Zeki sistemler teori ve uygulamaları dergisi (Online)

WoS Q Değeri

Scopus Q Değeri

Cilt

5

Sayı

2

Künye

HAMİTOĞLU, A. Auxiliary Learning of Non-Monotonic Hyperparameter Scheduling System Via Grid Search. Journal of Intelligent Systems: Theory and Applications, 5(2), 168-177.