research
∙
09/07/2023
Large Language Models as Optimizers
Optimization is ubiquitous. While derivative-based algorithms have been ...
share
research
∙
08/07/2023
Simple synthetic data reduces sycophancy in large language models
Sycophancy is an undesirable behavior where models tailor their response...
research
∙
05/29/2023
Brainformers: Trading Simplicity for Efficiency
Transformers are central to recent successes in natural language process...
research
∙
05/17/2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
The mixture proportions of pretraining data domains (e.g., Wikipedia, bo...
research
∙
05/15/2023
Symbol tuning improves in-context learning in language models
We present symbol tuning - finetuning language models on in-context inpu...
research
∙
03/07/2023
Larger language models do in-context learning differently
We study how in-context learning (ICL) in language models is affected by...
research
∙
02/13/2023
Symbolic Discovery of Optimization Algorithms
We present a method to formulate algorithm discovery as program search, ...
research
∙
02/03/2023
PyGlove: Efficiently Exchanging ML Ideas as Code
The increasing complexity and scale of machine learning (ML) has led to ...
research
∙
04/15/2022
Resource-Constrained Neural Architecture Search on Tabular Datasets
The best neural architecture for a given machine learning problem depend...
research
∙
01/21/2021
PyGlove: Symbolic Programming for Automated Machine Learning
Neural networks are sensitive to hyper-parameter and architecture choice...
research
∙
01/27/2020
Towards a Human-like Open-Domain Chatbot
We present Meena, a multi-turn open-domain chatbot trained end-to-end on...
research
∙
03/07/2018