[yaml] add RunInference support with VertexAI #33406
Conversation
Signed-off-by: Jeffrey Kinard <[email protected]>
|
R: @robertwb |
|
R: @damccorm |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
| def underlying_handler(self): | ||
| return self._handler | ||
|
|
||
| def preprocess_fn(self, row): |
There was a problem hiding this comment.
This won't actually get called until runtime, right? Perhaps we should call this default_preprocess_fn() and have it return an lambda (or raise an error). And then change postprocess_fn for symmetry.
There was a problem hiding this comment.
Great suggestion, I ended up making the preprocess parameter required on the VertexAI handler, so the error is redundant in this case... but I think moving forward it could be useful, and the refactor in general is cleaner.
Signed-off-by: Jeffrey Kinard <[email protected]>
| pass | ||
|
|
||
| def inference_output_type(self): | ||
| return RowTypeConstraint.from_fields([('example', Any), ('inference', Any), |
There was a problem hiding this comment.
Could we ask the handler for the type of inference? (Presumably the example type is that of the input as well.) Some handlers may not be able to provide this, but some can.
There was a problem hiding this comment.
We don't generally today, but we could for a very limited set of handlers. The ones I can think of with predictable output types are hugging face pipelines and vLLM; the rest are all dependent on the model.
I think this is probably worth doing when we can, it probably requires some slight modification to the PredictionResult type, though, and might be worth scoping into a follow on PR.
| pass | ||
|
|
||
| def inference_output_type(self): | ||
| return RowTypeConstraint.from_fields([('example', Any), ('inference', Any), |
There was a problem hiding this comment.
We don't generally today, but we could for a very limited set of handlers. The ones I can think of with predictable output types are hugging face pipelines and vLLM; the rest are all dependent on the model.
I think this is probably worth doing when we can, it probably requires some slight modification to the PredictionResult type, though, and might be worth scoping into a follow on PR.
Signed-off-by: Jeffrey Kinard <[email protected]>
Signed-off-by: Jeffrey Kinard <[email protected]>
This PR adds support for calling RunInference with preliminary support for VertexAI through the use of the
VertexAIModelHandlerJSONModelHandler.In order to implement more robust validation for the transform, the
SafeLineLoaderwas refactored out ofyaml_transform.pyinto a separateyaml_utils.pyfile to avoid import cycles.Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.