Welcome to Unreal-StableDiffusionTools Discussions!

[Mystfit]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Feature requests or ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

[danke1]

Hi Mystfit~!

I'm happy to see your great work!

While trying my own, I have a question.

https://youtube.com/shorts/C7f3dAKIOJE?feature=share

As i see your video above, each frames turns to the next frame very smoothly!

they are changing gradually and continuosly! not suddenly switched between frame and frame!

but not my own.

my own looks like just the combination of several simular frames.( slowly changed also, but between each frame, there is no conjuction)

what makes it possible is related to the stable diffusion option setting like iteration, strengh ext?
If it's right, Could you share how to set it?

else, just skill for making video with frames?

I hope you understand what i mean..!

Thanks~!!!

[Mystfit]

Thanks for the question! It mostly comes down to what parameters you use. Firstly you need to be use a fixed seed value so the generated noise pattern Stable Diffusion uses as a start point remains consistent every frame. Next, I slowly increased the strength value over time starting from 0, finally, I used prompts that tried to describe the content of the 3D scene so that the generated output had something to latch onto. In that video, I started with the prompt "two orange chairs" first. You can see in the animation when I dropped that prompt from the timeline the scene turned into a kitchen instead. I'm working on adding frame interpolation to the plugin which may help with making transitions between frames, but the biggest key is the seed and strength values.

…

On Tue, Nov 1, 2022, 12:45 AM danke1 ***@***.***> wrote: Hi Mystfit~! I'm happy to see your great work! While trying my own, I have a question. https://youtube.com/shorts/C7f3dAKIOJE?feature=share As i see your video above, each frames turns to the next frame very smoothly! they are changing gradually and continuosly! not suddenly switched between frame and frame! but not my own. my own looks like just the combination of several simular frames.( slowly changed also, but between each frame, there is no conjuction) what makes it possible is related to the stable diffusion option setting like iteration, strengh ext? else, just skill for making video with frames? I hope you understand what i mean..! Thanks~!!! — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGCJS4TM4V6O47FJRJNR33WF6WPJANCNFSM6AAAAAARJAJS6I> . You are receiving this because you authored the thread.Message ID: <Mystfit/Unreal-StableDiffusionTools/repo-discussions/9/comments/4017900@ github.com>

[danke1]

Amazing~!
You answer is so detailed and helpful! Thank you for your help!

The frame interpolation that you mentioned, I can’t find that…Could you let me know where and how to do that?!

[Mystfit]

The frame interpolation is not quite ready yet as I'm looking into integrating another model to handle the interpolation as my first attempt isn't working so well.

…

On Tue, Nov 1, 2022 at 4:12 PM danke1 ***@***.***> wrote: Amazing~~! You answer is so detailed and helpful! Thank you for your help! The frame interpolation that you mentioned, Where can i find that?! — Reply to this email directly, view it on GitHub <#9 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGCJS4GDAWJ3B7V2MSMGJDWGCDABANCNFSM6AAAAAARJAJS6I> . You are receiving this because you authored the thread.Message ID: <Mystfit/Unreal-StableDiffusionTools/repo-discussions/9/comments/4023909@ github.com>

[Cosmicwiz21]

Thank you Mystfit for all you hard work man!! :)) great tool!!

[thibaudthibaud]

Hi all !

I am super excited to try out thi great project, I am just running into an issue and was hoping you could help me out, it is probably very simple.
I could load the plugin correctly inside UE and generate the tokens but apparently it doesn't load any models and in the tutos
videos I can't find any hints on how to do that within this version of the plugin. By the way I am in Unreal Engine 5.2.
If you have any tips that would be fantastic and thanks for the hard work !

[Mystfit]

Glad to hear you're giving the plugin a go! In your content browser settings button, you might need to make sure that plugin contents are visible. This will expose all of the preset assets included with the pipeline.

…

On Fri, 8 Mar 2024, 10:47 pm thibaudthibaud, ***@***.***> wrote: load.model.png (view on web) <https://github.com/Mystfit/Unreal-StableDiffusionTools/assets/162691740/da8995e3-1a59-4a00-8426-c02e6cd995b7> Hi all ! I am super excited to try out thi great project, I am just running into an issue and was hoping you could help me out, it is probably very simple. I could load the plugin correctly inside UE and generate the tokens but apparently it doesn't load any models and in the tutos videos I can't find any hints on how to do that within this version of the plugin. By the way I am in Unreal Engine 5.2. If you have any tips that would be fantastic and thanks for the hard work ! — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGCJS2WWVOFFDP4GEAINFTYXGCLVAVCNFSM6AAAAAARJAJS6KVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DOMJYGAZTK> . You are receiving this because you authored the thread.Message ID: <Mystfit/Unreal-StableDiffusionTools/repo-discussions/9/comments/8718035@ github.com>

[thibaudthibaud]

Hi thanks for your super fast response !

It is enabled and I can perfectly see the plugin content as shown on the pictures here :

But when I go in projects settings and look for the "model dowload path" it drives me to an empty folder :

Should I download any models somewhere ? or should it be done automatically by itself when updating all the dependencies ?

Thanks again !

[Mystfit]

When a diffusers model is loaded (at generation time usually - and all of the provided plugin models are diffusers based) it will download the model from huggingface.co as long as your token is set. If the presets are visible, try and generate an image and the progress bar at the bottom of the plugin UI should turn purple as the model is downloaded.

…

On Fri, 8 Mar 2024, 11:37 pm thibaudthibaud, ***@***.***> wrote: Hi thanks for your super fast response ! It is enabled and I can perfectly see the plugin content as shown on the pictures here : plugin.content.png (view on web) <https://github.com/Mystfit/Unreal-StableDiffusionTools/assets/162691740/677682cd-9cf2-4af2-b2ef-c40bf25738e6> But when I go in projects settings and look for the "model dowload path" it drives me to an empty folder : model.download.path.png (view on web) <https://github.com/Mystfit/Unreal-StableDiffusionTools/assets/162691740/8e40dd77-645d-4fe3-8b50-54857c3a838a> Should I download any models somewhere ? or should it be done automatically by itself when updating all the dependencies ? Thanks again ! — Reply to this email directly, view it on GitHub <#9 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGCJSYUTVKHASMTZZAT3YTYXGIIDAVCNFSM6AAAAAARJAJS6KVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DOMJYGYYDG> . You are receiving this because you authored the thread.Message ID: <Mystfit/Unreal-StableDiffusionTools/repo-discussions/9/comments/8718603@ github.com>

[thibaudthibaud]

Thank you for your time and advices, it did work !