research
∙
05/08/2023
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Gradient preconditioning is a key technique to integrate the second-orde...
share
research
∙
11/25/2022
PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices
Pipeline parallelism enables efficient training of Large Language Models...
research
∙
10/06/2022
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Gradient regularization (GR) is a method that penalizes the gradient nor...
research
∙
09/20/2022
Neural Graph Databases
Graph databases (GDBs) enable processing and analysis of unstructured, c...
research
∙
09/14/2022
Efficient Quantized Sparse Matrix Operations on Tensor Cores
The exponentially growing model size drives the continued success of dee...
research
∙
10/02/2020
Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks
Natural Gradient Descent (NGD) helps to accelerate the convergence of gr...
research
∙
02/13/2020
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Large-scale distributed training of deep neural networks results in mode...
research
∙
06/06/2019
Practical Deep Learning with Bayesian Principles
Bayesian methods promise to fix many shortcomings of deep learning, but ...
research
∙
11/29/2018