research
∙
07/15/2023
Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph
Large language models (LLMs) have made significant strides in various ta...
share
research
∙
06/15/2023
CMMLU: Measuring massive multitask language understanding in Chinese
As the capabilities of large language models (LLMs) continue to advance,...
research
∙
05/24/2023
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
Large language models are powerful text processors and reasoners, but ar...
research
∙
05/24/2023
BeamSearchQA: Large Language Models are Strong Zero-Shot QA Solver
Open-domain question answering is a crucial task that often requires acc...
research
∙
05/23/2023
Query Rewriting for Retrieval-Augmented Large Language Models
Large Language Models (LLMs) play a powerful Reader of the Retrieve-then...
research
∙
05/16/2023
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Diffusion models have gained significant attention in the realm of image...
research
∙
05/11/2023
PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization
Based on the remarkable achievements of pre-trained language models in a...
research
∙
04/23/2023
Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models
Large language models (LLMs) can achieve highly effective performance on...
research
∙
03/29/2023
AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators
Many natural language processing (NLP) tasks rely on labeled data to tra...
research
∙
02/01/2023
Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models
Large language models can perform various reasoning tasks by using chain...
research
∙
12/22/2022
GENIE: Large Scale Pre-training for Text Generation with Diffusion Model
In this paper, we propose a large-scale language pre-training for text G...
research
∙
12/18/2022
Curriculum Sampling for Dense Retrieval with Document Expansion
The dual-encoder has become the de facto architecture for dense retrieva...
research
∙
12/15/2022
MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrievers
Dense retrieval aims to map queries and passages into low-dimensional ve...
research
∙
12/14/2022
APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning
Long-form numerical reasoning in financial analysis aims to generate a r...
research
∙
12/10/2022
LEAD: Liberal Feature-based Distillation for Dense Retrieval
Knowledge distillation is often used to transfer knowledge from a strong...
research
∙
11/18/2022
GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation
We introduce GENIUS: a conditional text generation model using sketches ...
research
∙
10/21/2022
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Sampling proper negatives from a large document pool is vital to effecti...
research
∙
10/21/2022
Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning
Commonsense generation aims to generate a realistic sentence describing ...
research
∙
10/18/2022
Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis
Most existing pre-trained language representation models (PLMs) are sub-...
research
∙
10/18/2022
Soft-Labeled Contrastive Pre-training for Function-level Code Representation
Code contrastive pre-training has recently achieved significant progress...
research
∙
09/27/2022
PROD: Progressive Distillation for Dense Retrieval
Knowledge distillation is an effective way to transfer knowledge from a ...
research
∙
06/28/2022
Joint Generator-Ranker Learning for Natural Language Generation
Due to exposure bias, most existing natural language generation (NLG) mo...
research
∙
05/23/2022
A Self-Paced Mixed Distillation Method for Non-Autoregressive Generation
Non-Autoregressive generation is a sequence generation paradigm, which r...
research
∙
04/27/2022
DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation
Dialog response generation in open domain is an important research topic...
research
∙
04/01/2022
Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings
Vector quantization (VQ) based ANN indexes, such as Inverted File System...
research
∙
01/26/2022
CodeRetriever: Unimodal and Bimodal Contrastive Learning
In this paper, we propose the CodeRetriever model, which combines the un...
research
∙
10/07/2021
Adversarial Retriever-Ranker for dense text retrieval
Current dense text retrieval models face two typical challenges. First, ...
research
∙
09/24/2021
Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
We study the problem of coarse-grained response selection in retrieval-b...
research
∙
09/14/2021
KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning
Pre-trained language models have led to substantial gains over a broad r...
research
∙
06/08/2021
FastSeq: Make Sequence Generation Faster
Transformer-based models have made tremendous impacts in natural languag...
research
∙
05/11/2021
EL-Attention: Memory Efficient Lossless Attention for Generation
Transformer model with multi-head attention requires caching intermediat...
research
∙
05/10/2021
Poolingformer: Long Document Modeling with Pooling Attention
In this paper, we introduce a two-level attention schema, Poolingformer,...
research
∙
04/16/2021
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
Now, the pre-training technique is ubiquitous in natural language proces...
research
∙
03/25/2021
Mask Attention Networks: Rethinking and Strengthen Transformer
Transformer is an attention-based neural network, which consists of two ...
research
∙
12/19/2020
Uncertainty-Aware Label Refinement for Sequence Labeling
Conditional random fields (CRF) for label decoding has become ubiquitous...
research
∙
12/01/2020
An Enhanced Knowledge Injection Model for Commonsense Generation
Commonsense generation aims at generating plausible everyday scenario de...
research
∙
10/21/2020
ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
In a sponsored search engine, generative retrieval models are recently p...
research
∙
10/04/2020
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
In this paper, we propose a novel data augmentation method, referred to ...
research
∙
04/30/2020
RikiNet: Reading Wikipedia Pages for Natural Question Answering
Reading long documents to answer open-domain questions remains challengi...
research
∙
04/08/2020
Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
News headline generation aims to produce a short sentence to attract rea...
research
∙
04/03/2020
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
In this paper, we introduce XGLUE, a new benchmark dataset to train larg...
research
∙
01/13/2020
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
In this paper, we present a new sequence-to-sequence pre-training model ...
research
∙
09/12/2019
Neural Semantic Parsing in Low-Resource Settings with Back-Translation and Meta-Learning
Neural semantic parsing has achieved impressive results in recent years,...
research
∙
02/25/2019