AMAdam: adaptive modifier of Adam method

This paper presents AMAdam, an innovative adaptive modifier gradient descent optimization algorithm that aims to overcome the challenges faced by traditional optimization methods in the field of artificial intelligence. The core of AMAdam’s contribution is its capacity to dynamically adjust the learning rate according to subtle gradient variations, resulting in an acceleration of the convergence speed of the optimization process. Concurrently, it ensures robust stabilization, guaranteeing that the algorithm converges reliably and efficiently. In addition, AMAdam efficiently reduces memory usage and hyperparameter complexity, distinguishing it from standard methods. A reliable comprehensive convergence analysis is provided. Extensive testing on multiple datasets, such as MNIST, IMDB movie reviews, CIFAR-10, and CIFAR-100, verifies that AMAdam consistently outperforms well-known optimizers including SGD(M), Adam, Adamax, RMSProp, Adagrad, AdaDelta, AdamW, and Radam. These outcomes demonstrate the effectiveness of AMAdam in optimization tasks while advancing computational efficiency, representing an important breakthrough in gradient descent optimization. Code is available at https://github.com/thchi/AMad.