model_serving_preprocess.md

Preprocessing before model serving

Get the input and output tensor name from the model

Create a tensorflow model builder

# Create SavedModelBuilder class
# defines where the model will be exported
export_path_base = MODEL_PATH
export_path = os.path.join(
    tf.compat.as_bytes(export_path_base),
    tf.compat.as_bytes(str(VERSION_NUMBER)))
print('Exporting trained model to', export_path)
builder = tf.saved_model.builder.SavedModelBuilder(export_path)

Set signature definition

 # Example
 # Creates the TensorInfo protobuf objects that encapsulates the input/output tensors
 input_images = tf.placeholder(tf.float32, shape=[None, None, None, 3], name='input_images')
 tensor_info_input = tf.saved_model.utils.build_tensor_info(input_images) 
 tensor_info_height = tf.saved_model.utils.build_tensor_info(tf.constant('height'))
 tensor_info_width = tf.saved_model.utils.build_tensor_info(tf.constant('width'))
 model_output = get_model()
 # output tensor info
 tensor_info_output = tf.saved_model.utils.build_tensor_info(model_output)

 # Defines the DeepLab signatures, uses the TF Predict API
 # It receives an image and its dimensions and output the segmentation mask
 prediction_signature = (
     tf.saved_model.signature_def_utils.build_signature_def(
         inputs={'images': tensor_info_input, 'height': tensor_info_height, 'width': tensor_info_width},
         outputs={'segmentation_map': tensor_info_output},
         method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

Export

 builder.add_meta_graph_and_variables(
 sess, [tf.saved_model.tag_constants.SERVING],
 signature_def_map={
     'predict_images':
         prediction_signature,
 })

 # export the model
 builder.save(as_text=True)
 print('Done exporting!')

Check if signature has been set for the model using 'saved_model_cli' (preinstalled with tensorflow)
```
saved_model_cli show --dir PATH_TO_MODEL --all
```

bendangnuksung/model_serving_preprocess.md

Preprocessing before model serving

Reference