Flask API requests tensorflow serving minimum example

README.md

References

https://www.tensorflow.org/tfx/serving/api_rest
tensorflow/serving#1317
https://towardsdatascience.com/image-classification-on-tensorflow-serving-with-grpc-or-rest-call-for-inference-fd3216ebd4f3

app.py

import json

from flask import Flask, request, jsonify
import requests

app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    if request.method == 'POST':
        headers = {"content-type": "application/json"}
        data = {
          'instances': np.random.rand(8, 224, 224, 3).tolist()
        }
        data = json.dumps(data)
        predictions = requests.post('http://localhost:8501/v1/models/yourmodel:predict', data=data, headers=headers)

        return jsonify(predictions)

if __name__ == '__main__':
    app.run()

run.sh

docker pull tensorflow/serving
docker run -p 8500:8500 -p 8501:8501 \
  --mount type=bind,source=/abusolute/path/to/saved_model/,target=/models/yourmodel \
  -e MODEL_NAME=yourmodel -t tensorflow/serving