# **Deploy Keras model to SageMaker**

###### tags: `Dragon Cloud AI`

## Step 1. Transform Keras model to Tensorflow pb file
Read the .h5 file and transform it to .pb file with specific file structure.
Note: The Tensorflow and Keras versions should match those which the model was trained on.

import keras
from keras.models import model_from_json
json_file = open('merge_softmax_202004201930.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
loaded_model.load_weights('merge_softmax_202004201930_weights.h5')
print("Loaded model from disk")

from tensorflow.python.saved_model import builder
from tensorflow.python.saved_model.signature_def_utils import predict_signature_def
from tensorflow.python.saved_model import tag_constants

model_version = '1'
export_dir = 'model/export/Servo/' + model_version

builder = builder.SavedModelBuilder(export_dir)

signature = predict_signature_def(
    inputs={"inputs": loaded_model.input}, outputs={"score": loaded_model.output})

from keras import backend as K

with K.get_session() as sess:
    # Save the meta graph and variables
    builder.add_meta_graph_and_variables(
        sess=sess, tags=[tag_constants.SERVING], signature_def_map={"serving_default": signature})
    builder.save()

After executing this code, you will have a directory named model, compress (.tar.gz) and upload it to s3 bucket (default bucket of SageMaker).

## Step 2. Define the pre/post processing and the environment requirements

Create a directory called `my_src` and store two files: `requirements.txt` and `inference.py`

The `requirements.txt` contains a list of environment requirements, and `inference.py` contains pre/post-processing methods.

### Notes for writing `inference.py`
1. The file name should be `inference.py`, using other names will cause errors in later steps.
2. The file should contain two function called `input_handler` and `output_handler`, the former is for pre-processing while the later is for post-processing.

## Step 3. Deployment
Open Jupyter Notebook on SageMaker, and start the deployment.

import boto3, re
from sagemaker import get_execution_role

role = get_execution_role()

from sagemaker.tensorflow.serving import Model
sagemaker_model = Model(model_data = 's3://' + sagemaker_session.default_bucket() + '/model/merge_softmax_202004201930.tar.gz',
                        role = role,
                        # assign the Tensorflow version
                        framework_version = '1.14',
                        # source directory
                        source_dir='my_src',
                        # environment requirements
                        env={'SAGEMAKER_REQUIREMENTS': 'requirements.txt'},
                        # pre/post-processing program
                        entry_point = 'inference.py')
                        
%%time
predictor = sagemaker_model.deploy(initial_instance_count = 1,
                                   instance_type='ml.t2.medium')

## Reference
For details of deploying Tensorflow models on SageMaker, please check this [website](https://sagemaker.readthedocs.io/en/stable/frameworks/tensorflow/using_tf.html#deploy-tensorflow-serving-models).