research
∙
05/24/2023
Self-Evolution Learning for Discriminative Language Model Pretraining
Masked language modeling, widely used in discriminative language model (...
share
research
∙
05/24/2023
Revisiting Token Dropping Strategy in Efficient BERT Pretraining
Token dropping is a recently-proposed strategy to speed up the pretraini...
research
∙
05/22/2023
Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks
Text classification tasks often encounter few shot scenarios with limite...
research
∙
03/24/2023
Towards Making the Most of ChatGPT for Machine Translation
ChatGPT shows remarkable capabilities for machine translation (MT). Seve...
research
∙
03/01/2023
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Sharpness aware minimization (SAM) optimizer has been extensively explor...
research
∙
02/19/2023
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT
Recently, ChatGPT has attracted great attention, as it can generate flue...
research
∙
02/18/2023
Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE
This technical report briefly describes our JDExplore d-team's submissio...
research
∙
12/04/2022
Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE
This technical report briefly describes our JDExplore d-team's Vega v2 s...
research
∙
10/11/2022
Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models
Fine-tuning large pretrained language models on a limited training corpu...
research
∙
08/22/2022
PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation
Prompt-tuning, which freezes pretrained language models (PLMs) and only ...
research
∙
05/30/2022
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation
Sequence-to-sequence (seq2seq) learning has become a popular trend for p...
research
∙
04/16/2022
A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysis
Aspect-Based Sentiment Analysis is a fine-grained sentiment analysis tas...
research
∙
01/13/2022
Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis
Aspect-based sentiment analysis (ABSA) is a fine-grained task of sentime...
research
∙
10/26/2021