How to use SD 2.1?

[DustinBrett]

Excited to see support for Stable Diffusion v2.1 in #29. As I don't have the hardware to build this, would it be possible to run the GitHub Action to create the needed files?

I am not sure I understand what would need to change in order to move from v1.5 to v2.1. I imagine I would need a different .wasm file and tokenizer.json, as well as to point cacheUrl somewhere else.

Any help would be much appreciated as I would love to see the quality of images with the newer model. Thanks!

FYI: @masahi, @spectrometerHBH & @tqchen

PS: Sorry for this issue sounding so similar to mlc-ai/web-llm#89, but it's the same asks so I figured it didn't need much changing.

[masahi]

For now only the model import is supported for v2.1. The compilation pipeline, including some js code, might need some change.

[DustinBrett]

For now only the model import is supported for v2.1. The compilation pipeline, including some js code, might need some change.

Ah ok, understood. Thanks for the info and getting it 1 step closer.

[sunzj]

i tried build and deploy SD 2.1, it runs normal, but the image just mosaic.

modified two places:

1. model_trace.py func concat_embeddings: change tensor dim 768->1024.

2. deploy.py tokenizer=CLIPTokenizer.from_pretrained("stabilityai/stable-diffusion-2-1", subfolder="tokenizer"),