End-to-end code generation example #610

kurapov-peter · 2023-06-14T13:39:11Z

I was able to come up with an end-to-end pipeline of code generation and execution that typically occurs in a JIT engine (our use case is the analytical query compilation in HDK). It creates a simple function (adding +1 to an array and storing the result in a second one) via llvm api, converts it to spirv, and submits it to the runtime.

The example is a bit complex in terms of dependencies: it uses llvm api, the spirv translator library, and intel gpu stack for execution. I found managing those in conda was the easiest way (I'm a bit biased though :) ). Nevertheless, I'm attaching the exact environment in scripts/deps.yml for reproduction simplicity (doesn't include the toolchain since any dev env for UR would already have it). Use as:

conda env create -f scripts/deps.yml
conda activate examples

Configure with:

cmake -DUR_BUILD_ADAPTER_L0=on ..

Ideally, we would want to be able to have the same flow for all the devices (including nvidia for which we would also convert to spirv if needed).

Corresponding L0-specific examples can be found here.

kbenzie · 2023-06-15T13:11:11Z

I've converted this to draft for now until we resolve getting access to hardware.

kbenzie · 2023-08-03T10:41:22Z

@kurapov-peter which versions of LLVM is this known to work with?

kurapov-peter · 2023-08-03T14:59:53Z

@kurapov-peter which versions of LLVM is this known to work with?

This one is for llvm 10..14 (I used 14 to test it). I can update to support higher version if you like.

kbenzie · 2023-08-03T16:12:40Z

@kurapov-peter which versions of LLVM is this known to work with?

This one is for llvm 10..14 (I used 14 to test it). I can update to support higher version if you like.

Ah okay, that probably explains why LLVM 16 wasn't working for me locally.

I feel like the CMake for finding the necessary packages will need to be more forgiving before we merge this, maybe disable building example if LLVM/SPIRV-LLVM-Translator aren't found instead of an error.

No rush though, I was mainly looking at this locally because we've introduced some changes to how UR is initalized recently and wanted to see if things still work with some modifications.

kurapov-peter · 2023-08-03T20:46:58Z

@kbenzie, sure, will do.

kurapov-peter · 2023-08-04T16:38:38Z

@kbenzie, I tested with llvm-16, should work now.

I feel like the CMake for finding the necessary packages will need to be more forgiving before we merge this, maybe disable building example if LLVM/SPIRV-LLVM-Translator aren't found instead of an error.

Done. I had to add extra parenthesis for the LLVMSPIRVLib_FOUND. Funny :)

kbenzie · 2023-08-21T13:18:58Z

Done. I had to add extra parenthesis for the LLVMSPIRVLib_FOUND. Funny :)

Thanks. Yeah that's very odd - CMake 🤷

I've been testing this rebased on a more recent commit on the adapters branch, this requires some changes to the loader/platform initialization which I'm happy to contribute. I am however seeing a SEGFAULT in the exit handlers which I'm still debugging, so will hold off until I've figured out what's going on there.

kbenzie · 2023-08-30T11:23:10Z

I am however seeing a SEGFAULT in the exit handlers which I'm still debugging, so will hold off until I've figured out what's going on there.

I reinstalled my Level Zero driver and this went away. I've been successfully running the example in Debug mode but get SEGFAULT's in Release or RelWithDebInfo modes.

I've created kurapov-peter#1 to update the example code with the adapter handles.

kurapov-peter · 2023-08-31T13:09:47Z

Sounds good! I wasn't able to rebase without conflicts, so will need to resolve them first. Should I just merge the patch or do I need to rebase the current branch on top of the latest adapters branch first?

kbenzie · 2023-08-31T15:21:07Z

I reinstalled my Level Zero driver and this went away. I've been successfully running the example in Debug mode but get SEGFAULT's in Release or RelWithDebInfo modes.

I figure these out. There's no dependency on building the adapters for the examples so a full build is required, then the SEGFAULT's went away.

Sounds good! I wasn't able to rebase without conflicts, so will need to resolve them first.

We have some guidance on dealing with merge conflicts in the Forks and Pull Requests section of the contirbution guide. Hopefully that will help resolving those.

Should I just merge the patch or do I need to rebase the current branch on top of the latest adapters branch first?

Whatever works for you, if you manage to rebase on the adapters branch its probably easiest to cherry-pick the top commit from my patch branch and ignore the merge in from the adapters branch in my PR.

kbenzie · 2023-08-31T15:50:31Z

I'm going on holiday now so I've assiged this to @veselypeta to work on getting this merged with you @kurapov-peter

examples/CMakeLists.txt

kurapov-peter · 2023-09-05T10:53:45Z

@veselypeta, I've rebased to the latest adapters state and cherry-picked the commit for API changes from kurapov-peter#1. I also added the missing copyrights and added the unlicense license to the original repo.

veselypeta

LGTM!

pbalcer

I added a few minor comments. I don't want to put more work on you, so I'm OK with leaving these as todo for later.

pbalcer · 2023-09-05T13:46:54Z

scripts/deps.yml

@@ -0,0 +1,37 @@
+name: examples


scripts is where we keep the specification and associated tooling. The third_party directory might be a better place for this (and a short README.md would be very useful about how to use this)

pbalcer · 2023-09-05T13:52:48Z

examples/codegen/codegen.cpp

+
+    auto adapters = get_adapters();
+    auto platforms = get_platforms(adapters);
+    auto gpus = get_gpus(platforms.front());


Would it make sense to filter out any platforms that aren't level-zero (or platforms that don't support spir64)? How would this behave on CUDA/HIP?

pbalcer · 2023-09-05T13:56:19Z

examples/codegen/codegen.cpp

+
+    ur_check(urQueueFinish(queue));
+
+    for (int i = 0; i < a_size; ++i) {


This prints out the result but never prints out the initial array. We should either check for the expected value programmatically or be consistent with how the data is printed.

pbalcer · 2023-09-05T14:01:03Z

examples/codegen/codegen.cpp

+    return devices;
+}
+
+template <typename T, size_t N> struct alignas(4096) AlignedArray {


is this for page alignment?
getpagesize() ?

pbalcer · 2023-09-05T14:04:44Z

examples/CMakeLists.txt

@@ -6,6 +6,9 @@
 include_directories(${CMAKE_CURRENT_SOURCE_DIR}/include)

 add_subdirectory(hello_world)
+if(UR_BUILD_EXAMPLE_CODEGEN)
+    add_subdirectory(codegen)


Now that we have GPU-equipped runners, it should be possible to create a workflow where this example is built and run as part of tests. Otherwise, it will be easy to accidentally break it.

kbenzie · 2023-09-14T09:51:48Z

I added a few minor comments. I don't want to put more work on you, so I'm OK with leaving these as todo for later.

I've added #859 to track this.

kbenzie marked this pull request as draft June 15, 2023 13:07

kurapov-peter mentioned this pull request Jul 3, 2023

Unified runtime evalutaion intel/hdk#156

Open

kurapov-peter force-pushed the pakurapo/codegen-example branch from ecf3c71 to 1a723c2 Compare August 4, 2023 16:36

kbenzie assigned veselypeta Aug 31, 2023

kurapov-peter and others added 5 commits September 5, 2023 10:20

Add codegen example

544ded5

Add conda dependencies

41883a9

Update codegen to support llvm16

aa88398

Disable building codegen example if the necessary packages are not found

cabcd98

[ur] Add adapter handle support to codegen example

80f6eb3

kurapov-peter force-pushed the pakurapo/codegen-example branch from 1a723c2 to 80f6eb3 Compare September 5, 2023 10:32

kurapov-peter added 2 commits September 5, 2023 10:43

Add missing copyright to the codegen example

60372d3

Change spir to spir64 to allow for correct execution on PVC

115abc2

kurapov-peter marked this pull request as ready for review September 5, 2023 10:47

veselypeta reviewed Sep 5, 2023

View reviewed changes

examples/CMakeLists.txt Outdated Show resolved Hide resolved

Add UR_BUILD_EXAMPLE_CODEGEN cmake option

4797c71

kurapov-peter requested review from veselypeta and pbalcer September 5, 2023 12:39

veselypeta approved these changes Sep 5, 2023

View reviewed changes

pbalcer approved these changes Sep 5, 2023

View reviewed changes

kbenzie mentioned this pull request Sep 14, 2023

Followup review feedback on e2e codegen example #859

Closed

kbenzie merged commit 1d706f6 into oneapi-src:adapters Sep 18, 2023
36 checks passed

omarahmed1111 mentioned this pull request Sep 20, 2023

Followup review feedback on e2e codegen example #876

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

End-to-end code generation example #610

End-to-end code generation example #610

kurapov-peter commented Jun 14, 2023

kbenzie commented Jun 15, 2023

kbenzie commented Aug 3, 2023

kurapov-peter commented Aug 3, 2023

kbenzie commented Aug 3, 2023

kurapov-peter commented Aug 3, 2023

kurapov-peter commented Aug 4, 2023

kbenzie commented Aug 21, 2023

kbenzie commented Aug 30, 2023

kurapov-peter commented Aug 31, 2023

kbenzie commented Aug 31, 2023

kbenzie commented Aug 31, 2023

kurapov-peter commented Sep 5, 2023

veselypeta left a comment

pbalcer left a comment

pbalcer Sep 5, 2023

pbalcer Sep 5, 2023

pbalcer Sep 5, 2023

pbalcer Sep 5, 2023

pbalcer Sep 5, 2023

kbenzie commented Sep 14, 2023


		ur_check(urQueueFinish(queue));

		for (int i = 0; i < a_size; ++i) {

End-to-end code generation example #610

End-to-end code generation example #610

Conversation

kurapov-peter commented Jun 14, 2023

kbenzie commented Jun 15, 2023

kbenzie commented Aug 3, 2023

kurapov-peter commented Aug 3, 2023

kbenzie commented Aug 3, 2023

kurapov-peter commented Aug 3, 2023

kurapov-peter commented Aug 4, 2023

kbenzie commented Aug 21, 2023

kbenzie commented Aug 30, 2023

kurapov-peter commented Aug 31, 2023

kbenzie commented Aug 31, 2023

kbenzie commented Aug 31, 2023

kurapov-peter commented Sep 5, 2023

veselypeta left a comment

Choose a reason for hiding this comment

pbalcer left a comment

Choose a reason for hiding this comment

pbalcer Sep 5, 2023

Choose a reason for hiding this comment

pbalcer Sep 5, 2023

Choose a reason for hiding this comment

pbalcer Sep 5, 2023

Choose a reason for hiding this comment

pbalcer Sep 5, 2023

Choose a reason for hiding this comment

pbalcer Sep 5, 2023

Choose a reason for hiding this comment

kbenzie commented Sep 14, 2023