research
∙
07/25/2023
Relationship between Batch Size and Number of Steps Needed for Nonconvex Optimization of Stochastic Gradient Descent using Armijo Line Search
Stochastic gradient descent (SGD) is the simplest deep learning optimize...
share
research
∙
07/18/2023
Modified memoryless spectral-scaling Broyden family on Riemannian manifolds
This paper presents modified memoryless quasi-Newton methods based on th...
research
∙
08/21/2022
Critical Bach Size Minimizes Stochastic First-Order Oracle Complexity of Deep Learning Optimizer using Hyperparameters Close to One
Practical results have shown that deep learning optimizers using small c...
research
∙
06/27/2022
Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness
Convergence and convergence rate analyses of adaptive methods, such as A...
research
∙
03/28/2022
Conjugate Gradient Method for Generative Adversarial Networks
While the generative model has many advantages, it is not feasible to ca...
research
∙
01/28/2022
Using Constant Learning Rate of Two Time-Scale Update Rule for Training Generative Adversarial Networks
Previous numerical results have shown that a two time-scale update rule ...
research
∙
12/14/2021
Minimization of Stochastic First-order Oracle Complexity of Adaptive Methods for Nonconvex Optimization
Numerical evaluations have definitively shown that, for deep learning op...
research
∙
08/26/2021
The Number of Steps Needed for Nonconvex Optimization of a Deep Learning Optimizer is a Rational Function of Batch Size
Recently, convergence as well as convergence rate analyses of deep learn...
research
∙
02/29/2020