research
∙
05/11/2022
RITA: a Study on Scaling Up Generative Protein Sequence Models
In this work we introduce RITA: a suite of autoregressive generative mod...
share
research
∙
10/16/2021
PAGnol: An Extra-Large French Generative Model
Access to large pre-trained models of varied architectures, in many diff...
research
∙
09/24/2021
Is the Number of Trainable Parameters All That Actually Matters?
Recent work has identified simple empirical scaling laws for language mo...
research
∙
07/25/2021
LightOn Optical Processing Unit: Scaling-up AI and HPC with a Non von Neumann co-processor
We introduce LightOn's Optical Processing Unit (OPU), the first photonic...
research
∙
06/07/2021
Photonic Differential Privacy with Direct Feedback Alignment
Optical Processing Units (OPUs) – low-power photonic chips dedicated to ...
research
∙
04/29/2021
Photonic co-processors in HPC: using LightOn OPUs for Randomized Numerical Linear Algebra
Randomized Numerical Linear Algebra (RandNLA) is a powerful class of met...
research
∙
02/08/2021
Contrastive Embeddings for Neural Architectures
The performance of algorithms for neural architecture search strongly de...
research
∙
01/06/2021
Adversarial Robustness by Design through Analog Computing and Synthetic Gradients
We propose a new defense mechanism against adversarial attacks inspired ...
research
∙
12/11/2020
Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment
The scaling hypothesis motivates the expansion of models past trillions ...
research
∙
06/23/2020
Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures
Despite being the workhorse of deep learning, the backpropagation algori...
research
∙
06/15/2020
Online Change Point Detection in Molecular Dynamics With Optical Random Features
Proteins are made of atoms constantly fluctuating, but can occasionally ...
research
∙
06/02/2020
Light-in-the-loop: using a photonics co-processor for scalable training of neural networks
As neural networks grow larger and more complex and data-hungry, trainin...
research
∙
06/11/2019
Principled Training of Neural Networks with Direct Feedback Alignment
The backpropagation algorithm has long been the canonical training metho...
research
∙
05/21/2018