learned_optimization reference documentation
learned_optimization is a research codebase for training learned optimizers. It implements hand designed and learned optimizers, tasks to meta-train and meta-test them on, and outer-training algorithms such as ES, gradients and PES.
- No dependency introduction to learned optimizers in JAX
- What is a learned optimizer?
- The inner problem
- Optimizers
- Learned optimizers
- Meta-loss: Measuring the performance of the learned optimizer.
- Meta-training with Gradients
- Vectorization: Speeding up Meta-training
- Evolutionary Strategies (ES): Meta-training without meta-gradients
- Meta-training with Truncated backprop through time
- Meta-training truncated ES
- Meta-training with truncations with less bias: Persistent Evolution Strategies (PES)
- Exercises
- Conclusion and relations to the
learned_optimization
package
- Summary tutorial: Getting metrics out of your models