research
∙
05/23/2023
Selective Pre-training for Private Fine-tuning
Suppose we want to train text prediction models in email clients or word...
share
research
∙
11/29/2022
Similarity Distribution based Membership Inference Attack on Person Re-identification
While person Re-identification (Re-ID) has progressed rapidly due to its...
research
∙
06/27/2022
Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimization
By ensuring differential privacy in the learning algorithms, one can rig...
research
∙
06/09/2022
Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks
Adversarial examples, which are usually generated for specific inputs wi...
research
∙
06/06/2022
Per-Instance Privacy Accounting for Differentially Private Stochastic Gradient Descent
Differentially private stochastic gradient descent (DP-SGD) is the workh...
research
∙
05/22/2022
Robust Quantity-Aware Aggregation for Federated Learning
Federated learning (FL) enables multiple clients to collaboratively trai...
research
∙
11/01/2021
Indiscriminate Poisoning Attacks Are Shortcuts
Indiscriminate data poisoning attacks, which add imperceptible perturbat...
research
∙
10/13/2021
Differentially Private Fine-tuning of Language Models
We give simpler, sparser, and faster algorithms for differentially priva...
research
∙
10/08/2021
Momentum Doesn't Change the Implicit Bias
The momentum acceleration technique is widely adopted in many optimizati...
research
∙
06/29/2021
Regularized OFU: an Efficient UCB Estimator forNon-linear Contextual Bandit
Balancing exploration and exploitation (EE) is a fundamental problem in ...
research
∙
06/17/2021
Large Scale Private Learning via Low-rank Reparametrization
We propose a reparametrization scheme to address the challenges of apply...
research
∙
05/31/2021
Adversarial Training with Rectified Rejection
Adversarial training (AT) is one of the most effective strategies for pr...
research
∙
02/25/2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
The privacy leakage of the model about the training data can be bounded ...
research
∙
01/08/2021
BN-invariant sharpness regularizes the training model to better generalization
It is arguably believed that flatter minima can generalize better. Howev...
research
∙
08/04/2020
Well-Conditioned Methods for Ill-Conditioned Systems: Linear Regression with Semi-Random Noise
Classical iterative algorithms for linear system solving and regression ...
research
∙
07/21/2020
Membership Inference with Privately Augmented Data Endorses the Benign while Suppresses the Adversary
Membership inference (MI) in machine learning decides whether a given ex...
research
∙
06/29/2020
Adai: Separating the Effects of Adaptive Learning Rate and Momentum Inertia
Adaptive Momentum Estimation (Adam), which combines Adaptive Learning Ra...
research
∙
02/12/2020
On Layer Normalization in the Transformer Architecture
The Transformer is widely used in natural language processing tasks. To ...
research
∙
11/26/2019
Gradient Perturbation is Underrated for Differentially Private Convex Optimization
Gradient perturbation, widely used for differentially private optimizati...
research
∙
05/29/2019
Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data
Stochastic variance reduced methods have gained a lot of interest recent...
research
∙
03/17/2019
Training Over-parameterized Deep ResNet Is almost as Easy as Training a Two-layer Network
It has been proved that gradient descent converges linearly to the globa...
research
∙
01/02/2019
SGD Converges to Global Minimum in Deep Learning via Star-convex Path
Stochastic gradient descent (SGD) has been found to be surprisingly effe...
research
∙
09/19/2018
Capacity Control of ReLU Neural Networks by Basis-path Norm
Recently, path norm was proposed as a new capacity measure for neural ne...
research
∙
02/27/2018
Train Feedfoward Neural Network with Layer-wise Adaptive Rate via Approximating Back-matching Propagation
Stochastic gradient descent (SGD) has achieved great success in training...
research
∙
02/19/2018
Generalization Error Bounds with Probabilistic Guarantee for SGD in Nonconvex Optimization
The success of deep learning has led to a rising interest in the general...
research
∙
12/20/2017
Block-diagonal Hessian-free Optimization for Training Neural Networks
Second-order methods for neural network optimization have several advant...
research
∙
09/23/2017
Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descent
Recent work has demonstrated the effectiveness of gradient descent for d...
research
∙
05/25/2016
Reshaped Wirtinger Flow and Incremental Algorithm for Solving Quadratic System of Equations
We study the phase retrieval problem, which solves quadratic system of e...
research
∙
03/11/2016