Contrastive Language-Image Pretraining
Connecting text and images.
Read MoreResearch Notebooks
Key Ideas From
Important Papers
Sharing articles and tutorials on the finer details of training and tuning deep neural networks for maximum performance.
2024-04-06
Mode Connectivity
Local minima in loss landscapes are connected by high accuracy pathways.
Read More2024-03-19
Gradient Boosting
Ensembles where new members are trained to correct previous mistakes.
Read More2024-03-08
Knowledge Distillation
Training a small model on the outputs of a larger and more accurate model.
Read More2024-02-26
Double Descent
A phenomena where generalization gets worse then better with larger models and bigger datasets.
Read More2024-02-15
Denoising Diffusion
A class of generative latent variable models inspired by nonequilibrium thermodynamics.
Read More2024-01-29
Optimal Brain Damage
An early method for pruning networks according to parameter saliency.
Read More2024-01-22
Low-Rank Adaptation
Reducing the storage requirements for fine tuned task specific networks.
Read More2024-01-21
Snapshot Ensembles
A low-cost method that leverages checkpoints throughout the training trajectory.
Read More2024-01-20
Proximal Policy Optimization
A computationally efficient on-policy reinforcement learning algorithm.
Read More2024-01-19
Natural Evolution Strategies
A family of algorithms for evolving the parameters of search distributions.
Read More2024-01-17
World Models
Dreaming with generative models of reinforcement learning environments.
Read More2024-01-16
Deep Q-Learning
A foundational off-policy algorithm that kickstarted deep reinforcement learning.
Read More2024-01-14
Stochastic Weight Averaging
An optimization trick for the final phases of training with SGD.
Read More2024-01-13
Lottery Ticket Hypothesis
Finding sparse subnetworks that train as well as dense networks from scratch.
Read More2024-01-13
Trust Region Policy Optimization
The monotonic on-policy reinforcement learning algorithm.
Read More