research
∙
07/19/2023
Gradient Sparsification For Masked Fine-Tuning of Transformers
Fine-tuning pretrained self-supervised language models is widely adopted...
share
research
∙
07/12/2023
Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models
We investigate the effects of post-training quantization and quantizatio...
research
∙
04/25/2022
A Survey on Word Meta-Embedding Learning
Meta-embedding (ME) learning is an emerging approach that attempts to le...
research
∙
04/04/2022
Aligned Weight Regularizers for Pruning Pretrained Neural Networks
While various avenues of research have been explored for iterative pruni...
research
∙
09/30/2021
Deep Neural Compression Via Concurrent Pruning and Self-Distillation
Pruning aims to reduce the number of parameters while maintaining perfor...
research
∙
04/14/2021
I Wish I Would Have Loved This One, But I Didn't – A Multilingual Dataset for Counterfactual Detection in Product Reviews
Counterfactual statements describe events that did not or cannot take pl...
research
∙
02/12/2021
Semantically-Conditioned Negative Samples for Efficient Contrastive Learning
Negative sampling is a limiting factor w.r.t. the generalization of metr...
research
∙
01/22/2021
k-Neighbor Based Curriculum Sampling for Sequence Prediction
Multi-step ahead prediction in language models is challenging due to the...
research
∙
07/29/2020
Compressing Deep Neural Networks via Layer Fusion
This paper proposes layer fusion - a model compression technique that di...
research
∙
06/05/2020
An Overview of Neural Network Compression
Overparameterized networks trained to convergence have shown impressive ...
research
∙
05/12/2020
Do not let the history haunt you – Mitigating Compounding Errors in Conversational Question Answering
The Conversational Question Answering (CoQA) task involves answering a s...
research
∙
09/09/2019
Transfer Reward Learning for Policy Gradient-Based Text Generation
Task-specific scores are often used to optimize for and evaluate the per...
research
∙
01/21/2019
Error-Correcting Neural Sequence Prediction
In this paper we propose a novel neural language modelling (NLM) method ...
research
∙
11/02/2018
Analysing Dropout and Compounding Errors in Neural Language Models
This paper carries out an empirical analysis of various dropout techniqu...
research
∙
09/16/2018
Curriculum-Based Neighborhood Sampling For Sequence Prediction
The task of multi-step ahead prediction in language models is challengin...
research
∙
09/16/2018
Semi-Supervised Multi-Task Word Embeddings
Word embeddings have been shown to benefit from ensembling several word ...
research
∙
08/13/2018
Angular-Based Word Meta-Embedding Learning
Ensembling word embeddings to improve distributed word representations h...
research
∙
05/18/2018
Siamese Capsule Networks
Capsule Networks have shown encouraging results on defacto benchmark com...
research
∙
04/23/2018
Dropping Networks for Transfer Learning
In natural language understanding, many challenges require learning rela...
research
∙
10/04/2017