Skip to content

prompt to prompt edit method to control the generation in images and videos

Notifications You must be signed in to change notification settings

jianshu-qin/p2p

Repository files navigation

The code should only requires diffusers==0.18.2.My enviroment is same as https://github.com/haohantianchen/Animate.

directory

p2p is the directory for Prompt2Prompt codes. src is the directory for AnimateDiff codes. xt_unc is the directory to save some tensors. tools is the directory to do some preprocess.

Install

Install AnimateDiff-PromptTravel follow https://github.com/haohantianchen/Animate.
Then in the root dir, replace the files in src dir, copy the p2p dir into the root dir .

Usage

bash infer.sh

The default config will generate 16 frames video , loading bg latents from xt_unc/bg_latents.pt and fg latents from xt_unc/rec_girl_fg_latents.pt, then blend bg latents with weights w1=[0.9,0.1] and warp fg latents.

The bg and fg latents are generated by test_nti.py. fg latents are reconstructed from ref_image.png using null-text inversion. More details can be seen in readme_image.md.

More args can be seen in cli.py, generate() function.

About

prompt to prompt edit method to control the generation in images and videos

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages