research
∙
09/15/2023
Headless Language Models: Learning without Predicting with Contrastive Weight Tying
Self-supervised pre-training of language models usually consists in pred...
share
research
∙
08/22/2023
SONAR: Sentence-Level Multimodal and Language-Agnostic Representations
We introduce SONAR, a new multilingual and multimodal fixed-size sentenc...
research
∙
06/13/2023
Is Anisotropy Inherent to Transformers?
The representation degeneration problem is a phenomenon that is widely o...
research
∙
06/09/2023
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?
Recent advances in natural language processing (NLP) have led to the dev...
research
∙
06/02/2023
Data-Efficient French Language Modeling with CamemBERTa
Recent advances in NLP have significantly improved the performance of la...
research
∙
05/23/2023
A Simple Method for Unsupervised Bilingual Lexicon Induction for Data-Imbalanced, Closely Related Language Pairs
Existing approaches for unsupervised bilingual lexicon induction (BLI) o...
research
∙
12/20/2022
Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation
One of the major challenges of machine translation (MT) is ambiguity, wh...
research
∙
12/14/2022
MANTa: Efficient Gradient-Based Tokenization for Robust End-to-End Language Modeling
Static subword tokenization algorithms have been an essential component ...
research
∙
11/08/2022
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-...
research
∙
06/22/2022
DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon
Finding word boundaries in continuous speech is challenging as there is ...
research
∙
05/24/2022
MaskEval: Weighted MLM-Based Evaluation for Text Summarization and Simplification
In text summarization and simplification, system outputs must be evaluat...
research
∙
05/24/2022
T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translation
We present a new approach to perform zero-shot cross-modal transfer betw...
research
∙
04/11/2022
Speech Sequence Embeddings using Nearest Neighbors Contrastive Learning
We introduce a simple neural encoder architecture that can be trained us...
research
∙
03/30/2022
Generative Spoken Dialogue Language Modeling
We introduce dGSLM, the first "textless" model able to generate audio sa...
research
∙
03/11/2022
Are discrete units necessary for Spoken Language Modeling?
Recent work in spoken language modeling shows the possibility of learnin...
research
∙
02/18/2022
From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French
Language models for historical states of language are becoming increasin...
research
∙
01/17/2022
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus
The need for raw large raw corpora has dramatically increased in recent ...
research
∙
12/20/2021
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
What are the units of text that we want to model? From bytes to multi-wo...
research
∙
10/26/2021
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?
Recent impressive improvements in NLP, largely based on the success of c...
research
∙
04/15/2021
Rethinking Automatic Evaluation in Sentence Simplification
Automatic evaluation remains an open research question in Natural Langua...
research
∙
03/22/2021
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
With the success of large-scale pre-training and multilingual modeling i...
research
∙
01/26/2021
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
Multilingual pretrained language models have demonstrated remarkable zer...
research
∙
10/24/2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models
Transfer learning based on pretraining language models on a large amount...
research
∙
10/23/2020
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
Coupled with the availability of large scale datasets, deep learning arc...
research
∙
07/27/2020
Evaluating the reliability of acoustic speech embeddings
Speech embeddings are fixed-size acoustic representations of variable-le...
research
∙
06/11/2020
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
We use the multilingual OSCAR corpus, extracted from Common Crawl via la...
research
∙
05/27/2020
Establishing a New State-of-the-Art for French Named Entity Recognition
The French TreeBank developed at the University Paris 7 is the main sour...
research
∙
05/01/2020
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations
In order to simplify a sentence, human editors perform multiple rewritin...
research
∙
05/01/2020
Multilingual Unsupervised Sentence Simplification
Progress in Sentence Simplification has been hindered by the lack of sup...
research
∙
05/01/2020
Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi
Building natural language processing systems for non standardized and lo...
research
∙
11/30/2019
Modeling German Verb Argument Structures: LSTMs vs. Humans
LSTMs have proven very successful at language modeling. However, it rema...
research
∙
11/10/2019
CamemBERT: a Tasty French Language Model
Pretrained language models are now ubiquitous in Natural Language Proces...
research
∙
10/07/2019
Controllable Sentence Simplification
Text simplification aims at making a text easier to read and understand ...
research
∙
01/30/2019
Reference-less Quality Estimation of Text Simplification Systems
The evaluation of text simplification (TS) systems remains an open chall...
research
∙
06/12/2016