Original title: Fast ODE-based Sampling for Diffusion Models in Around 5 Steps
Authors: Zhenyu Zhou, Defang Chen, Can Wang, Chun Chen
In this article, the authors explore the topic of sampling from diffusion models, which involves solving ordinary differential equations (ODEs) to obtain accurate solutions with a minimal number of function evaluations (NFE). While recent advancements in higher-order ODE solvers have led to improved sampling performance, these methods still introduce approximation errors that greatly reduce sample quality when using a small NFE, such as around 5. To address this issue, the authors propose a new approach called Approximate MEan-Direction Solver (AMED-Solver). By observing that each sampling trajectory mostly lies in a two-dimensional subspace, AMED-Solver directly learns the mean direction for faster diffusion sampling, thereby eliminating truncation errors. Furthermore, this method can easily be incorporated as a plugin to enhance existing ODE-based samplers. The authors present extensive experiments on image synthesis, demonstrating the effectiveness of their method across different resolutions. Remarkably, with only 5 NFE, they achieve excellent FID scores on CIFAR-10, ImageNet 64×64, and LSUN Bedroom datasets. The authors provide their code for easy access and implementation.
Original article: https://arxiv.org/abs/2312.00094