ML Research And Development
A Stochastic Gradient
Descent Into Madness
Deep learning is advancing rapidly as thousands of new papers are published every year. Exploring best practices and state of the art techniques can often feel like drinking from a fire hose. Dead Neuron serves as a concise guide to research, where our collection of notebooks distill key ideas and implementation details from influential papers to help you learn how to build better neural networks.
Featured Research
Subnetwork Ensembles
Neural network ensembles have been effectively used to improve generalization by combining the predictions of multiple independently trained models. However, the growing scale and complexity of deep neural networks have led to these methods becoming prohibitively expensive and time consuming to implement. Low-cost ensemble methods have become increasingly important as they can alleviate the need to train multiple models from scratch while retaining the generalization benefits that...
Read More