-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: Refactor string input check #101
Conversation
src/backend_common.cc
Outdated
*const_cast<char*>(buffer) = 0; | ||
} | ||
|
||
if (*element_idx != expected_element_cnt) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here slightly different condition checks from original tensorflow.cc
and libtorch.cc
. Removing (*response != nullptr)
.
I don't think we want to skip sending this error or skip FillStringTensor
if RESPOND_AND_SET_NULL_IF_ERROR
is called before SetStringInputTensor
in tensorflow.cc
and libtorch.cc
. I believe it was a mistake because other string format checks before this line do not require if (*response != nullptr)
to send error or FillStringTensor
. Open to discussions.
// Code from tensorflow.cc and libtorch.cc
if ((*response != nullptr) && (element_idx != request_element_cnt)) {
RESPOND_AND_SET_NULL_IF_ERROR(
response, TRITONSERVER_ErrorNew(
TRITONSERVER_ERROR_INTERNAL,
std::string(
"expected " + std::to_string(request_element_cnt) +
" strings for inference input '" + name + "', got " +
std::to_string(element_idx))
.c_str()));
FillStringTensor(
tensor, tensor_offset + element_idx, request_element_cnt - element_idx);
}
Co-authored-by: Tanmay Verma <[email protected]>
What does the PR do?
Move each individual check of string input in model backend repos to the bakend repo.
Checklist
<commit_type>: <Title>
Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
backend: #101
tensorflow_backend: triton-inference-server/tensorflow_backend#104
pytorch_backend: triton-inference-server/pytorch_backend#136
onnxruntime_backend: triton-inference-server/onnxruntime_backend#263
python_backend: triton-inference-server/python_backend#370
Test plan:
17021222 and 17057045
Background
Initially just to add missing string input tensor checks to python_backend. Refactored because duplicate code from other model backends were found.