diff --git a/README.md b/README.md index 5ce9701d..a6413c2b 100644 --- a/README.md +++ b/README.md @@ -13,12 +13,14 @@ threestudio is a unified framework for 3D content creation from text prompts, si
threestudio
-threestudio -threestudio +threestudio +threestudio +threestudio
-threestudio -threestudio +threestudio +threestudio +threestudio
threestudio @@ -31,7 +33,7 @@ threestudio is a unified framework for 3D content creation from text prompts, si ๐Ÿ‘† Results obtained from methods implemented by threestudio ๐Ÿ‘†
| ProlificDreamer | DreamFusion | Magic3D | SJC | Latent-NeRF | Fantasia3D | TextMesh |
-| Zero-1-to-3 | Magic123 | HiFA | +| Zero-1-to-3 | Magic123 | HiFA | SDI |
| InstructNeRF2NeRF | Control4D | @@ -68,6 +70,7 @@ threestudio is a unified framework for 3D content creation from text prompts, si ## News +- 08/11/2024: Thank [Artem Lukoianov](https://github.com/ottogin) for implementation of [Score Distillation via Reparametrized DDIM](https://lukoianov.com/sdi)! Text-to-3D module is added to Threestudio as well as a notebook with 2D score distillation experiments. - 21/10/2024: Thank [Amir Barda](https://github.com/amirbarda) for implementation of [MagicClay](https://github.com/amirbarda/MagicClay)! Follow the instructions on its website to give it a try. - 12/03/2024: Thank [Matthew Kwak](https://github.com/mskwak01) and [Inรจs Hyeonsu Kim](https://github.com/Ines-Hyeonsu-Kim) for implementation of [3DFuse](https://github.com/KU-CVLAB/3DFuse-threestudio)! Follow the instructions on its website to give it a try. - 08/03/2024: Thank [Xinhua Cheng](https://github.com/cxh0519/) for implementation of [GaussianDreamer](https://github.com/cxh0519/threestudio-gaussiandreamer)! Follow the instructions on its website to give it a try. @@ -241,6 +244,36 @@ For feature requests, bug reports, or discussions about technical problems, plea ## Supported Models +### Score Distillation via Reparametrized DDIM (SDI) [![arXiv](https://img.shields.io/badge/arXiv-2405.15891-b31b1b.svg?style=flat-square)](https://arxiv.org/abs/2405.15891) + +SDI suggests to reconsider the approach to sampling the noise term in Dreamfusion. The paper demonstrates that score distillation process can be seen as a reparametrization of 2D image sampling algorithms. In that case the noise added on each step of score distillation should be of a very particular form. Noise in Dreamfusion (SDS), however, is sampled randomly, what causes over-blurring. SDI approximates the correct noise term by inverting the DDIM process. + +Notable differences from the paper: N/A. + +Pros: +* High quality of the textures +* Sharp geometric details + +Cons: +* Slower than SDS (1.5x times) due to additional inversion. Still faster then prolific dreamer due to lower number of steps +* Requires more VRAM than SDS due to higher resolution rendering. Decrease the resolution to fit to smaller GPUs. + +**Results obtained in threestudio (Stable Diffusion, 512x512)** + +A_DSLR_photo_of_a_freshly_baked_round_loaf_of_sourdough_bread +a_photograph_of_a_knight + +**Example running commands** +```sh +python launch.py --config configs/sdi.yaml --train --gpu 0 system.prompt_processor.prompt="pumpkin head zombie, skinny, highly detailed, photorealistic" + +python launch.py --config configs/sdi.yaml --train --gpu 1 system.prompt_processor.prompt="a photograph of a ninja" + +python launch.py --config configs/sdi.yaml --train --gpu 2 system.prompt_processor.prompt="a zoomed out DSLR photo of a hamburger" + +python launch.py --config configs/sdi.yaml --train --gpu 3 system.prompt_processor.prompt="bagel filled with cream cheese and lox" +``` + ### ProlificDreamer [![arXiv](https://img.shields.io/badge/arXiv-2305.16213-b31b1b.svg?style=flat-square)](https://arxiv.org/abs/2305.16213) **This is an unofficial experimental implementation! Please refer to [https://github.com/thu-ml/prolificdreamer](https://github.com/thu-ml/prolificdreamer) for official code release.**