research
∙
08/28/2023
MEMORY-VQ: Compression for Tractable Internet-Scale Memory
Retrieval augmentation is a powerful but expensive method to make langua...
share
research
∙
06/17/2023
GLIMMER: generalized late-interaction memory reranker
Memory-augmentation is a powerful approach for efficiently incorporating...
research
∙
05/22/2023
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Multi-query attention (MQA), which only uses a single key-value head, dr...
research
∙
03/17/2023
CoLT5: Faster Long-Range Transformers with Conditional Computation
Many natural language processing tasks benefit from long inputs, but pro...
research
∙
01/25/2023
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
Retrieval-augmented language models such as Fusion-in-Decoder are powerf...
research
∙
12/15/2022
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model...
research
∙
10/18/2022
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models
Decoding methods for large language models often trade-off between diver...
research
∙
09/29/2022
Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsing
A common recent approach to semantic parsing augments sequence-to-sequen...
research
∙
10/12/2021
Mention Memory: incorporating textual knowledge into Transformers through entity mention attention
Natural language understanding tasks such as open-domain question answer...
research
∙
05/10/2021
ReadTwice: Reading Very Large Documents with Memories
Knowledge-intensive tasks such as question answering often require assim...
research
∙
02/26/2021
DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections
This paper explores learning rich self-supervised entity representations...
research
∙
08/21/2018