Skip to content

Instantly share code, notes, and snippets.

@israel-hdez
Created November 10, 2023 05:14
Show Gist options
  • Save israel-hdez/af374562ef9e5b9d80890aa6f0bce20d to your computer and use it in GitHub Desktop.
Save israel-hdez/af374562ef9e5b9d80890aa6f0bce20d to your computer and use it in GitHub Desktop.
KServe basic test
#!/usr/bin/env bash
# Deploy to new namespace
oc new-project kserve-test
# Create a ServingRuntime
curl -s https://raw.githubusercontent.com/opendatahub-io/kserve/master/config/runtimes/kserve-sklearnserver.yaml | \
sed 's/ClusterServingRuntime/ServingRuntime/' | \
sed "s|kserve-sklearnserver:replace|docker.io/kserve/sklearnserver:latest|" | \
oc apply -f -
# Create the InferenceService, which will deploy a sample model
cat <<EOF | oc apply -f - $@
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: "sklearn-v2-iris"
annotations:
serving.knative.openshift.io/enablePassthrough: "true"
sidecar.istio.io/inject: "true"
spec:
predictor:
model:
modelFormat:
name: sklearn
protocolVersion: v2
runtime: kserve-sklearnserver
storageUri: "gs://kfserving-examples/models/sklearn/1.0/model"
EOF
# Wait for the pod of the model to fully start. Then fetch the URL:
MODEL_ENDPOINT=$(kubectl get inferenceservice sklearn-v2-iris -o jsonpath='{.status.url}')
# Create sample data file
cat <<EOF > iris-input-v2.json
{
"inputs": [
{
"name": "input-0",
"shape": [2, 4],
"datatype": "FP32",
"data": [
[6.8, 2.8, 4.8, 1.4],
[6.0, 3.4, 4.5, 1.6]
]
}
]
}
EOF
# Test that the model replies correctly.
curl -v \
-H "Content-Type: application/json" \
-d @./iris-input-v2.json \
${MODEL_ENDPOINT}/v2/models/sklearn-v2-iris/infer
# Expected reply:
#{
# "id": "823248cc-d770-4a51-9606-16803395569c",
# "model_name": "sklearn-v2-iris",
# "model_version": "v1.0.0",
# "outputs": [
# {
# "data": [1, 1],
# "datatype": "INT64",
# "name": "predict",
# "parameters": null,
# "shape": [2]
# }
# ]
#}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment