# minddiffusion

**Repository Path**: mindspore-lab/minddiffusion

## Basic Information

- **Project Name**: minddiffusion
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: deploy
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-07-13
- **Last Updated**: 2024-07-13

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Awesome MindDiffusion Models

## Introduction

This repo is an open source collection of a series of classic and new SoTA diffusion models based on MindSpore. We also provide the awesome list of diffusion models.

## Released now
These models are implemented in MindSpore and run on Ascend.
| Model Name | Task | Link |
|---|---|---|
|Taichu-GLIDE | Vision - Text To Image|[Link](./vision/Taichu-GLIDE/) |
|Wukong-Huahua| Vision - Text To Image|[Link](./vision/wukong-huahua/) |
|Stable-Diffusionv2| Vision - Text To Image|[Link](vision/stablediffusionv2/) |

## Awesome Model List
<table>
<thead>
<tr>
<th>Model</th>
<th>Paper</th>
<th>Institution</th>
<th>Date</th>
<th>Conference</th>
<th>Support</th>
</tr>
</thead>
<tbody>
<tr>
<td>DDPM</td>
<td><a href="https://proceedings.neurips.cc/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf">DDPM: Denoising Diffussion Probalisitic Model</a></td>
<td>UC Berkeley</td>
<td>Jun 2020</td>
<td>NeurIPS 2020</td>
<td>To do</td>
</tr>
<tr>
<td colspan=6 align="center"><b>Vision - Image generation<b></td>
</tr>
<tr>
<td>Improved diffusion</td>
<td><a href="https://proceedings.mlr.press/v139/nichol21a.html">Improved Denoising Diffusion Probabilistic Models</a></td>
<td>OpenAI</td>
<td>Feb 2021</td>
<td>PMLR 2021</td>
<td></td>
</tr>
<tr>
<td>Guided diffusion</td>
<td><a href="https://proceedings.neurips.cc/paper/2021/hash/49ad23d1ec9fa4bd8d77d02681df5cfa-Abstract.html">Diffusion Models Beat Gans on Image Synthesis</a></td>
<td>OpenAI</td>
<td>Apr 2021</td>
<td>NeurIPS 2021</td>
<td></td>
</tr>
<tr>
<td>ADM</td>
<td><a href="https://arxiv.org/abs/2105.05233">Diffusion Models Beat GANs on Image Synthesis</a></td>
<td>OpenAI</td>
<td>Apr 2021</td>
<td>NeurIPS 2021</td>
<td></td>
</tr>
<tr>
<td>FastDPM</td>
<td><a href="">On Fast Sampling of Diffusion Probabilistic Models</a></td>
<td>NVIDIA</td>
<td>May 2021</td>
<td>ICLR Workshop 2021</td>
<td></td>
</tr>
<tr>
<td>LSGM</td>
<td><a href="https://arxiv.org/abs/2106.05931">Score-based Generative Modeling in Latent Space</a></td>
<td>NVIDIA</td>
<td>Jun 2021</td>
<td>NeurIPS 2021</td>
<td></td>
</tr>
<tr>
<td>Distilled-DM</td>
<td><a href="https://arxiv.org/abs/2202.00512">Progressive Distillation for Fast Sampling of Diffusion Models</a></td>
<td>Google Brain</td>
<td>Feb 2022</td>
<td>ICLR 2022</td>
<td></td>
</tr>
<tr>
<td>GGDM</td>
<td><a href="http://arxiv.org/abs/2202.05830">Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality</a></td>
<td>Google Brain</td>
<td>Feb 2022</td>
<td>ICLR 2022</td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Vision -  Text to Image<b></td>
</tr>
<tr>
<td>Stable Diffusion/LDM</td>
<td><a href="https://openaccess.thecvf.com/content/CVPR2022/html/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.html">High-Resolution Image Synthesis with Latent Diffusion Models</a></td>
<td>Stability.AI</td>
<td>Dec 2021</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Glide</td>
<td><a href="https://arxiv.org/abs/2112.10741">Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models</a></td>
<td>OpenAI</td>
<td>Dec 2021</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Dalle-2</td>
<td><a href="https://arxiv.org/abs/2204.06125">Hierarchical Text-conditional Image Generation with Clip Latents</a></td>
<td>OpenAI</td>
<td>Apr 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>KNN Diffusion</td>
<td><a href="https://arxiv.org/abs/2204.02849">Image Generation via Large-Scale Retrieval</a></td>
<td>Meta AI</td>
<td>Apr 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Imagen</td>
<td><a href="https://arxiv.org/abs/2205.11487">Imagen: Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding</a></td>
<td>Google Brain</td>
<td>May 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>LAION-RDM</td>
<td><a href="https://arxiv.org/abs/2207.13038">Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models</a></td>
<td>Ludwig-Maximilian University of Munich</td>
<td>Jul 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>DreamBooth</td>
<td><a href="https://arxiv.org/abs/2208.12242">DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation</a></td>
<td>Google Research</td>
<td>Aug 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>DreamFusion</td>
<td><a href="https://arxiv.org/abs/2209.14988">DreamFusion: Text-to-3D using 2D Diffusion</a></td>
<td>Google Research</td>
<td>29 Sep 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Vision - Image Editing</b></td>
</tr>
<tr>
<td>SDEdit</td>
<td><a href="https://arxiv.org/abs/2108.01073">SDEdit: Image Synthesis and Editing with Stochastic Differential Equations</a></td>
<td>Stanford U &amp; CMU</td>
<td>Aug 2021</td>
<td>ICLR 2022</td>
<td></td>
</tr>
<tr>
<td>RePaint</td>
<td><a href="https://arxiv.org/abs/2201.09865">RePaint: Inpainting using Denoising Diffusion Probabilistic Models</a></td>
<td>ETH Zurich</td>
<td>Jan 2022</td>
<td>CVPR 2022</td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Vision -  Video Genereation</bs></td>
</tr>
<tr>
<td>Video diffusion models</td>
<td><a href="https://arxiv.org/abs/2204.03458">Video diffusion models</a></td>
<td>Google Brain</td>
<td>Apr 2022</td>
<td>ICLR 2022 Workshop</td>
<td></td>
</tr>
<tr>
<td>MCVD</td>
<td><a href="https://arxiv.org/abs/2205.09853">MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation</a></td>
<td>University of Montreal</td>
<td>May 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Make-A-Video</td>
<td><a href="https://arxiv.org/abs/2209.14792">Make-A-Video: Text-to-Video Generation without Text-Video Data</a></td>
<td>Meta AI</td>
<td>29 Sep 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Imagen Video</td>
<td><a href="https://arxiv.org/abs/2210.02303">Imagen Video: High Definition Video Generation with Diffusion Models</a></td>
<td>Google Brain</td>
<td>5 Oct 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Natural language</b></td>
</tr>
<tr>
<td>Diffusion-LM</td>
<td><a href="https://arxiv.org/abs/2205.14217">Diffusion-LM Improves Controllable Text Generation</a></td>
<td>Stanford University</td>
<td>May 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Audio - Audio Generation</b></td>
</tr>
<tr>
<td>DiffWave</td>
<td><a href="https://arxiv.org/abs/2009.09761">DiffWave: A Versatile Diffusion Model for Audio Synthesis</a></td>
<td>Nvidia &amp; Baidu</td>
<td>Jun 2020</td>
<td>ISMIR 2021</td>
<td></td>
</tr>
<tr>
<td>WaveGrad</td>
<td><a href="https://arxiv.org/abs/2009.00713">WaveGrad: Estimating Gradients for Waveform Generation</a></td>
<td>Google Brain</td>
<td>Sep 2020</td>
<td>ICLR 2021</td>
<td></td>
</tr>
<tr>
<td>Symbolic Music Generation</td>
<td><a href="https://arxiv.org/abs/2103.16091">Symbolic Music Generation with Diffusion Models</a></td>
<td>Google Brain</td>
<td>Mar 2021</td>
<td>ISMIR 2021</td>
<td></td>
</tr>
<tr>
<td>DiffSinger</td>
<td><a href="https://arxiv.org/abs/2105.02446">DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism</a></td>
<td>Zhejiang University</td>
<td>May 2021</td>
<td>AAAI 2022</td>
<td></td>
</tr>
<tr>
<td>VDM</td>
<td><a href="https://arxiv.org/abs/2107.00630">Variational Diffusion Models</a></td>
<td>Google Brain</td>
<td>Jul 2021</td>
<td>NeurIPS 2021</td>
<td></td>
</tr>
<tr>
<td>FastDiff</td>
<td><a href="https://arxiv.org/abs/2204.09934">FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis</a></td>
<td>Tencent AI Lab</td>
<td>Apr 2022</td>
<td>IJCAI 2022</td>
<td></td>
</tr>
<tr>
<td>BDDMs</td>
<td><a href="https://arxiv.org/abs/2203.13508">BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis</a></td>
<td>Tencent AI Lab</td>
<td>May 2022</td>
<td>ICLR 2022</td>
<td></td>
</tr>
<tr>
<td>SawSing</td>
<td><a href="https://arxiv.org/abs/2208.04756">DDSP-based Singing Vocoders: A New Subtractive-based Synthesizer and A Comprehensive Evaluation</a></td>
<td></td>
<td>AUG 2022</td>
<td>ISMIR 2022</td>
<td></td>
</tr>
<tr>
<td>Prodiff</td>
<td><a href="https://arxiv.org/abs/2207.06389">ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech</a></td>
<td>Zhejiang University</td>
<td>JUL 2022</td>
<td>ACM Multimedia 2022</td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Audio -  Audio Conversion</b></td>
</tr>
<tr>
<td>DiffVC</td>
<td><a href="https://arxiv.org/abs/2109.13821">Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme</a></td>
<td>Huawei Noah</td>
<td>Sep 2021</td>
<td>ICLR 2022</td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Audio -  Audio Enhancement</b></td>
</tr>
<tr>
<td>NU-Wave</td>
<td><a href="https://arxiv.org/abs/2104.02321">NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling</a></td>
<td>MINDSLAB</td>
<td>Apr 2021</td>
<td>Interspeech 2021</td>
<td></td>
</tr>
<tr>
<td>CDiffSE</td>
<td><a href="https://arxiv.org/abs/2202.05256">Conditional Diffusion Probabilistic Model for Speech Enhancement</a></td>
<td>CMU</td>
<td>Feb 2022</td>
<td>IEEE 2022</td>
<td></td>
</tr>
<tr>
<td colspan=6 align="center"><b>Audio -  Text to Speech</b></td>
</tr>
<tr>
<td>Grad-TTS</td>
<td><a href="https://arxiv.org/abs/2111.11755">Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech</a></td>
<td>Huawei Noah</td>
<td>May 2021</td>
<td></td>
<td></td>
</tr>
<tr>
<td>EdiTTS</td>
<td><a href="https://arxiv.org/abs/2110.02584.pdf">EdiTTS: Score-based Editing for Controllable Text-to-Speech</a></td>
<td>Yale University</td>
<td>Oct 2021</td>
<td></td>
<td></td>
</tr>
<tr>
<td>DiffGAN-TTS</td>
<td><a href="https://arxiv.org/abs/2201.11972">DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs</a></td>
<td>Tencent AI Lab</td>
<td>Jan 2022</td>
<td></td>
<td></td>
</tr>
<tr>
<td>Diffsound</td>
<td><a href="https://arxiv.org/abs/2207.09983">Diffsound: Discrete Diffusion Model for Text-to-sound Generation</a></td>
<td>Tencent AI Lab</td>
<td>Jul 2022</td>
<td></td>
<td></td>
</tr>
</tbody>
</table>



## Contributing

We welcom all contributions to improve this project! Please fork this repo and submit a pull request to contribute your diffusion models.