README_Docker.md

Vald ONNX Ingress Filter

vald-onnx-ingress-filter is one of the official ingress filter components provided by Vald.

Its custom logic requires the input of the ONNX model as a request and outputs the result from the ONNX model as the request of the Vald Agent.

Using this component lets users vectorize various data such as text and images using the ONNX model only inside the Vald cluster without external APIs.

Requirement

linux/amd64

• Libraries: kubectl, Helm, grpcio, numpy, onnxruntime, vald-client-python
• Others: Vald cluster

linux/arm64

• Libraries: kubectl, Helm, grpcio, numpy, onnxruntime, vald-client-python
• Others: Vald cluster

Get Started

vald-onnx-ingress-filter is used for ingress filter component of the Vald cluster, which means it should be used on the Kubernetes cluster, not the local environment or Docker.

The steps are following:

1. Deploy vald-onnx-ingress-filter

   git clone https://github.com/vdaas/vald-onnx-ingress-filter.git
   kubectl apply -f vald-onnx-ingress-filter/k8s

   NOTE: The example manifest files use ResNet50-v2 from ONNX Model Zoo as the ONNX model. You can change the model by editing k8s/deployment.yaml.

   ...
     - |
       curl -L "https://github.com/onnx/models/raw/main/vision/classification/resnet/model/resnet50-v2-7.onnx" -o /model/sample.onnx  #FIXME

2. Deploy the Vald cluster with filter gateway

   Please edit the example/helm/values.yaml in the Vald repository to make vald-filter-gateway available and use it for deployment.

   git clone https://github.com/vdaas/vald.git
   cd vald
   
   vim example/helm/values.yaml
   ---
   ...
   gateway:
   ...
       filter:
           enabled: true
   ...
   agent:
       ngt:
           dimension: 1000

   After editing, let’s try to deploy the Vald cluster by the helm install command.

   helm install vald vald/vald --values example/helm/values.yaml

Sample code with vald-client-python

We use a random vector as an example of the insert object.

import grpc
import numpy as np
from vald.v1.payload import payload_pb2
from vald.v1.vald import (
    filter_pb2_grpc,
    search_pb2_grpc,
)

channel = grpc.insecure_channel("localhost:8081")

// Insert
stub = filter_pb2_grpc.FilterStub(channel)
sample = np.random.random((1, 3, 224, 224)).astype(np.float32)
resize_vector = payload_pb2.Object.ReshapeVector(
    object=sample.tobytes(),
    shape=[1, 3, 224, 224],
)
resize_vector = resize_vector.SerializeToString()

req = payload_pb2.Insert.ObjectRequest(
    object=payload_pb2.Object.Blob(
        id="0",
        object=resize_vector
    ),
    config=payload_pb2.Insert.Config(skip_strict_exist_check=False),
    vectorizer=payload_pb2.Filter.Target(
        host="vald-onnx-ingress-filter",
        port=8081,
    )
)
stub.InsertObject(req)

// Search
sstub = search_pb2_grpc.SearchStub(channel)
scfg = payload_pb2.Search.Config(
    num=10, radius=-1.0, epsilon=0.01, timeout=3000000000
)
sstub.Search(
    payload_pb2.Search.Request(
        vector=np.zeros((1000)),
        config=scfg
    )
)

Versions

tag	linux/amd64	linux/arm64	description
latest	✅	✅	the latest image is the same as the latest version of vdaas/vald repository version.
vX.Y.Z	✅	✅	the vX.Y.Z image applies the source code of the vdaas/vald repository.
pr-XXX	✅	❌	the pr-XXX image applies the source code of the pull request XXX of the vdaas/vald repository.

Dockerfile

The Dockerfile of this image is here.

About Vald Project

The information about the Vald project, please refer to the following:

• Official website
• GitHub

Contacts

We're love to support you! Please feel free to contact us anytime with your questions or issue reports.

• Official Slack WS
• GitHub Issue

License

This product is under the terms of the Apache License v2.0; refer LICENSE file.