This paper presents AMAdam, an innovative adaptive modifier gradient descent optimization algorithm that aims to overcome the challenges faced by traditional optimization methods in the field of artificial intelligence. The core of AMAdam’s contribution is its capacity to dynamically adjust the learning rate according to subtle gradient variations, resulting in an acceleration of the convergence speed of the optimization process. Concurrently, it ensures robust stabilization, guaranteeing that the algorithm converges reliably and efficiently. In addition, AMAdam efficiently reduces memory usage and hyperparameter complexity, distinguishing it from standard methods. A reliable comprehensive convergence analysis is provided. Extensive testing on multiple datasets, such as MNIST, IMDB movie reviews, CIFAR-10, and CIFAR-100, verifies that AMAdam consistently outperforms well-known optimizers including SGD(M), Adam, Adamax, RMSProp, Adagrad, AdaDelta, AdamW, and Radam. These outcomes demonstrate the effectiveness of AMAdam in optimization tasks while advancing computational efficiency, representing an important breakthrough in gradient descent optimization. Code is available at https://github.com/thchi/AMad.

AMAdam: adaptive modifier of Adam method

Gabriella Casalino
2024-01-01

Abstract

This paper presents AMAdam, an innovative adaptive modifier gradient descent optimization algorithm that aims to overcome the challenges faced by traditional optimization methods in the field of artificial intelligence. The core of AMAdam’s contribution is its capacity to dynamically adjust the learning rate according to subtle gradient variations, resulting in an acceleration of the convergence speed of the optimization process. Concurrently, it ensures robust stabilization, guaranteeing that the algorithm converges reliably and efficiently. In addition, AMAdam efficiently reduces memory usage and hyperparameter complexity, distinguishing it from standard methods. A reliable comprehensive convergence analysis is provided. Extensive testing on multiple datasets, such as MNIST, IMDB movie reviews, CIFAR-10, and CIFAR-100, verifies that AMAdam consistently outperforms well-known optimizers including SGD(M), Adam, Adamax, RMSProp, Adagrad, AdaDelta, AdamW, and Radam. These outcomes demonstrate the effectiveness of AMAdam in optimization tasks while advancing computational efficiency, representing an important breakthrough in gradient descent optimization. Code is available at https://github.com/thchi/AMad.
File in questo prodotto:
File Dimensione Formato  
s10115-023-02052-9 (1).pdf

non disponibili

Descrizione: Versione Editoriale
Tipologia: Documento in Versione Editoriale
Licenza: NON PUBBLICO - Accesso privato/ristretto
Dimensione 4.43 MB
Formato Adobe PDF
4.43 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
AMAdam.pdf

accesso aperto

Descrizione: Pre-print
Tipologia: Documento in Pre-print
Licenza: Creative commons
Dimensione 8.09 MB
Formato Adobe PDF
8.09 MB Adobe PDF Visualizza/Apri
AMAdam_Final 1.pdf

accesso aperto

Tipologia: Documento in Post-print
Licenza: Creative commons
Dimensione 12.05 MB
Formato Adobe PDF
12.05 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11586/502582
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 3
social impact