error with flash attention

[staple-pi]

hi,
I'm trying to run amg_example.py , but meet an Userwarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:253.) return torch.nn.functional.scaled_dot_product_attention(q_, k_, v_, attn_mask=attn_bias) in lash_4.py:line369. how can i solve it?

[staple-pi]

i have already set os.environ['SEGMENT_ANYTHING_FAST_USE_FLASH_4'] = '0', but it looks like flash_4 was still used anyway

[cpuhrsch]

Hello @staple-pi, thanks for opening this issue. The error you're reporting doesn't seem related to flash_4, but your installation of torch. Can you share a bit more detail on what version of torch and on what platform you're using it?
Thanks!

[bigmover]

Hello @staple-pi, thanks for opening this issue. The error you're reporting doesn't seem related to flash_4, but your installation of torch. Can you share a bit more detail on what version of torch and on what platform you're using it? Thanks!

torch2.0.1cu117 will reproduce the error! sam-fast is specified to match torch2.2?

[staple-pi]

Hello @staple-pi, thanks for opening this issue. The error you're reporting doesn't seem related to flash_4, but your installation of torch. Can you share a bit more detail on what version of torch and on what platform you're using it? Thanks!

Thank you for your reply！ Actually, i have already install the preview version of torch(2.2.0.dev20231123+cu121). And i'm using it in the environment created by anaconda with python 3.10.13 on Windows

[cpuhrsch]

@staple-pi Hm, interesting. This might be a general issue with PyTorch on Windows. Can you use scaled_dot_product_attention otherwise or just not in the context of this project?

[staple-pi]

@staple-pi Hm, interesting. This might be a general issue with PyTorch on Windows. Can you use scaled_dot_product_attention otherwise or just not in the context of this project?

After test, scaled_dot_product_attention can work in elsewhere base the same environment.

[cpuhrsch]

@staple-pi - I landed some optimizations for the AMG. Can you try again?

[staple-pi]

@staple-pi - I landed some optimizations for the AMG. Can you try again?

unfortunately, the error still appear

[drisspg]

Are you using the sdp_kernel to only enable "flash_attention". Flash Attention2 is not supported on windows and we don't have an ETA on when/if this support will be added.