Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to obtain neutral mesh during audio2mesh training? #190

Open
fredkingdom opened this issue Jul 26, 2024 · 0 comments
Open

How to obtain neutral mesh during audio2mesh training? #190

fredkingdom opened this issue Jul 26, 2024 · 0 comments

Comments

@fredkingdom
Copy link

Thanks for open-sourcing such a wonderful work!

I notice that in Fig.1 of the technique report, the neutral mesh is used to calculated the mesh offset during audio2lmk training. It seems that you use mediapipe to obtain the mesh vertices in this step, would you please share how you obtained neutral mesh vertices in this step?

I also notice that during audio2vid inference, you add the predicted mesh offset to the non-neutral mesh vertices obtained from the reference image, will this effect the performance of lip sync?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant