Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add error message when explicit load model doesn't exist #244

Merged
merged 1 commit into from
Aug 17, 2023

Conversation

rmccorm4
Copy link
Contributor

@rmccorm4 rmccorm4 commented Aug 17, 2023

Previously we would fail if an explicit load model didn't exist, but we didn't log any error messages describing it. This adds an error message if the model doesn't exist in any specified model repository.

Assert log error is printed in test: triton-inference-server/server#6204


$ tritonserver --model-repos models1 --model-repos models2 --model-control-mode explicit --load-model dne
...
# Adds this error message to logs
E0817 19:18:51.786824 2997733 model_repository_manager.cc:1265] failed to poll model 'dne': model not found in any model repository.
...

I0817 19:18:51.786955 2997733 server.cc:674]
+-------+---------+--------+
| Model | Version | Status |
+-------+---------+--------+
+-------+---------+--------+

I0817 19:18:51.840315 2997733 metrics.cc:810] Collecting metrics for GPU 0: NVIDIA GeForce RTX 3090
I0817 19:18:51.840634 2997733 metrics.cc:703] Collecting CPU metrics
I0817 19:18:51.840879 2997733 tritonserver.cc:2435]
+----------------------------------+----------------------------------------------------------------------------------------------------------------------+
| Option                           | Value                                                                                                                |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------+
| server_id                        | triton                                                                                                               |
| server_version                   | 0.0.0                                                                                                                |
| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration sys |
|                                  | tem_shared_memory cuda_shared_memory binary_tensor_data parameters statistics logging                                |
| model_repository_path[0]         | models1                                                                                                              |
| model_repository_path[1]         | models2                                                                                                              |
| model_control_mode               | MODE_EXPLICIT                                                                                                        |
| startup_models_0                 | dne                                                                                                                  |
| strict_model_config              | 0                                                                                                                    |
| rate_limit                       | OFF                                                                                                                  |
| pinned_memory_pool_byte_size     | 268435456                                                                                                            |
| cuda_memory_pool_byte_size{0}    | 67108864                                                                                                             |
| min_supported_compute_capability | 6.0                                                                                                                  |
| strict_readiness                 | 1                                                                                                                    |
| exit_timeout                     | 30                                                                                                                   |
| cache_enabled                    | 0                                                                                                                    |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------+

I0817 19:18:51.840919 2997733 server.cc:305] Waiting for in-flight requests to complete.
I0817 19:18:51.840935 2997733 server.cc:321] Timeout 30: Found 0 model versions that have in-flight inferences
I0817 19:18:51.840948 2997733 server.cc:336] All models are stopped, unloading models
I0817 19:18:51.840958 2997733 server.cc:343] Timeout 30: Found 0 live models and 0 in-flight non-inference requests
error: creating server: Internal - failed to load all models

@rmccorm4 rmccorm4 requested review from kthui and GuanLuo August 17, 2023 19:15
@rmccorm4 rmccorm4 merged commit e456d2a into main Aug 17, 2023
1 check passed
@rmccorm4 rmccorm4 deleted the rmccormick-dne branch August 18, 2023 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

5 participants