fix issue#35, update ort to v2 #106
Conversation
|
just a draft to see how the CI test results go |
|
onnxruntime needs to be update to v1.17.1 as ort v2 requires |
|
Thanks for the contribution @broccoliSpicy , I will do some tests on CUDA to make sure everything works correctly and merge this in next week! |
Thank you so much! @var77 |
| @@ -306,7 +309,8 @@ impl EncoderService { | |||
|
|
|||
| let num_cpus = num_cpus::get(); | |||
|
|
|||
| let encoder = SessionBuilder::new(environment)? | |||
| let _ = &ONNX_ENV; | |||
| let encoder = Session::builder()? | |||
| .with_parallel_execution(true)? | |||
| .with_intra_threads(num_cpus as i16)? | |||
| .with_optimization_level(GraphOptimizationLevel::Level3)? | |||
There was a problem hiding this comment.
Just tested with GPU server, the default env seems to not work, but I have put
.with_execution_providers([
CUDAExecutionProvider::default().build(),
OpenVINOExecutionProvider::default().build(),
CPUExecutionProvider::default().build(),
])?
on Session::builder()? and removed the ONNX_ENV completely and it started to use the GPU
There was a problem hiding this comment.
thank you so much!
feel free to edit it in my branch
|
Quick update on this: I am waiting ORT to have v2 stable release, make sure that nothing breaks our integration in that release and merge this PR with stable v2 version. |
That makes sense, thank you for your update! |
#35