This repository has been archived by the owner on Aug 30, 2024. It is now read-only.

sync SYCL code #312

Closed

luoyu-intel wants to merge 2 commits into main from sync_sycl

Contributor

luoyu-intel commented Jul 12, 2024 •

edited

Loading

Type of Change

update the SYCL performance.

llama2-7b int4, sym, g128, comp_dtype=fp32, scale_dtype=fp32, KV_dtype=fp32

Max1100: 8.6ms/token
A770: 14.5ms/token
A770m: 15.8ms/token
A750: 15.4ms/token
155H:  51.6ms/token

cmake .. -DNS_SYCL=ON -DCMAKE_CXX_COMPILER=icx
run_llama xxx -ngl 33

luoyu-intel added 2 commits

July 12, 2024 15:06


          sync

fb7fc64


          fix linux compile

8bce8a0

luoyu-intel closed this

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet