research
∙
07/14/2023
HYTREL: Hypergraph-enhanced Tabular Data Representation Learning
Language models pretrained on large collections of tabular data have dem...
share
research
∙
06/06/2023
Large Language Models of Code Fail at Completing Code with Potential Bugs
Large language models of code (Code-LLMs) have recently brought tremendo...
research
∙
06/01/2023
Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion
Pretrained code language models have enabled great progress towards prog...
research
∙
11/08/2022
Parameter and Data Efficient Continual Pre-training for Robustness to Dialectal Variance in Arabic
The use of multilingual language models for tasks in low and high-resour...
research
∙
09/30/2022
Differentially Private Optimization on Large Model at Small Cost
Differentially private (DP) optimization is the standard paradigm to lea...
research
∙
09/30/2022
Differentially Private Bias-Term only Fine-tuning of Foundation Models
We study the problem of differentially private (DP) fine-tuning of large...
research
∙
06/14/2022
Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger
Per-example gradient clipping is a key algorithmic step that enables pra...
research
∙
04/23/2022
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning
Recent work has found that multi-task training with a large number of di...
research
∙
10/15/2021
Meta-learning via Language Model In-context Tuning
The goal of meta-learning is to learn to adapt to a new task with only a...
research
∙
06/24/2020
Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes
BERT has recently attracted a lot of attention in natural language under...
research
∙
08/28/2019
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
Statistical natural language inference (NLI) models are susceptible to l...
research
∙
07/09/2019
GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing
We present GluonCV and GluonNLP, the deep learning toolkits for computer...
research
∙
04/26/2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources
With an increasing demand for training powers for deep learning algorith...
research
∙
04/16/2019
Just-in-Time Dynamic-Batching
Batching is an essential technique to improve computation efficiency in ...
research
∙
04/06/2018