HANDS ON GUIDE TO GENERATE CUSTOM TENSORFLOW LITE MODEL USING TENSORFLOW/KERAS AND QUANTIZATION FOR EDGE DEVICES

NEELESH SINHA
NEELESH SINHA

Jr. Data Scientist

INTRODUCTION

Tensorflow Lite is a cross platform production ready framework for deploying machine learning models on mobile devices and embedded systems.Tensorflow lite models operate on low latency optimization which can process high volume of data with minimum delay and since the model is deployed on an edge device like a smartphone it offers a good privacy with respect to data, use case and predictions for model and you can also use its pre built models for classification, object detection, pose estimation etc. check out examples. The google translate app has a feature that can translate text using your smartphone camera in real time is a good example of tensorflow lite in action in the real world.

LINK TO DOWNLOAD – https://play.google.com/store/apps/details?id=com.google.android.apps.translate&hl=en_IN&

But we can also create our custom models you In this hands on guide. We will generate a custom model in tensorflow/keras and convert it into a tensorflow lite model and see how much size we can save without compromising the accuracy of the tensorflow lite model for using it on edge devices.

Lets import the required modules –

import os
import numpy as np
import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dense
from tensorflow.keras.losses import SparseCategoricalCrossentropy
from sklearn.metrics import accuracy_score

We are using tensorflow version 2.0.

#To return the model size of both models we are using these two functions get_file_size and convert_bytes.
def get_file_size(file_path):
size = os.path.getsize(file_path)
return size
def convert_bytes(size, unit=None):
if unit == "KB":
return print('File size: ' + str(round(size / 1024, 3)) + '
Kilobytes')
elif unit == "MB":
return print('File size: ' + str(round(size / (1024 * 1024),
3)) + ' Megabytes')
else:
return print('File size: ' + str(size) + ' bytes')

For this guide we’ll be using the classic fashion_mnist dataset available on tensorflow’s official website which has images consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28×28 grayscale image, associated with a label
from 10 classes.
Let’s go ahead and load the dataset-

#To use this data we need call keras function and save it to a variable as below
fashion_mnist = keras.datasets.fashion_mnist
#Now separate the data under training images and training labels and testing images and testing labels.
(train_images, train_labels), (test_images, test_labels) =
fashion_mnist.load_data()

#create a class with 10 values corresponding to the dataset

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress',
'Coat','Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
#Run these commands for data exploration

we can see the shape, number of labels and classes now lets preprocess and look at a sample image.

To bring all the image intensity values to one scale we have normalized the values from 0 to 1 by dividing it by 255 as shown above.
#Let’s start building our first model using tensorflow/keras
Create a sequential class so the input images that we get will be in grid structure of 28×28 and flatten it out into an one dimensional array by using the flatten function and the first dense layer with rectified linear unit (relu) with 128 output units and second with 10 output units.

model = keras.Sequential([
Flatten(input_shape=(28, 28)),
Dense(128, activation='relu'),
Dense(10)
])

#define optimizer, loss and accuracy metrics

model.compile(optimizer='adam',
loss= SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
#Generate weight values for corresponding models
#run it for 10 epochs
model.fit(train_images, train_labels, epochs=10)

Output-

Epoch 1/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.4972
- accuracy: 0.8242
Epoch 2/10
1875/1875 [==============================] - 4s 2ms/step - loss: 0.3789
- accuracy: 0.8629
Epoch 3/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.3391
- accuracy: 0.8762
Epoch 4/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.3140
- accuracy: 0.8850Epoch 5/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2958
- accuracy: 0.8910
Epoch 6/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2830
- accuracy: 0.8955
Epoch 7/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2688
- accuracy: 0.8993
Epoch 8/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2595
- accuracy: 0.9036
Epoch 9/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2484
- accuracy: 0.9076
Epoch 10/10
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2389
- accuracy: 0.9110
<tensorflow.python.keras.callbacks.History at 0x7f9ef37fb7b8>

We have got an accuracy of 91.10 percent.

#get file size

convert_bytes(get_file_size(KERAS_MODEL_NAME), "MB")

Output-

File size: 1.19 Megabytes

#test accuracy

test_loss, test_acc = model.evaluate(test_images, test_labels,
verbose=2)
print('\nTest accuracy:', test_acc)

Output-

313/313 - 0s - loss: 0.3446 - accuracy: 0.8802
Test accuracy: 0.8802000284194946

#save the model in h5 format

KERAS_MODEL_NAME = "tf_model_fashion_mnist.h5"model.save(KERAS_MODEL_NAME)

Finally, we will convert our model to tensorflow lite

TF_LITE_MODEL_FILE_NAME = "tf_lite_model.tflite"

#Define the converter and call the convert function

tf_lite_converter = tf.lite.FLite Converter.from_keras_model(model)
tflite_model = tf_lite_converter.convert()
#save the model in the created file name
tflite_model_name = TF_LITE_MODEL_FILE_NAME
open(tflite_model_name, "wb").write(tflite_model)

#get the file size

convert_bytes(get_file_size(TF_LITE_MODEL_FILE_NAME), "KB")

Output-

File size: 398.93 Kilobytes

As compared to tensorflow keras this file size is 1/3 of it.
Both the models have been created and now we have to evaluate the models in terms of their accuracy for that we have to set up the interpreter by defining the input values and output values.

interpreter = tf.lite.Interpreter(model_path = TF_LITE_MODEL_FILE_NAME)
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
print("Input Shape:", input_details[0]['shape'])
print("Input Type:", input_details[0]['dtype'])
print("Output Shape:", output_details[0]['shape'])
print("Output Type:", output_details[0]['dtype'])

Output-

Input Shape: [ 1 28 28]
Input Type: <class 'numpy.float32'>Output Shape: [ 1 10]
Output Type: <class 'numpy.float32'>
We have to modify the input shape and resize the tensors
interpreter.resize_tensor_input(input_details[0]['index'], (10000, 28,
28))
interpreter.resize_tensor_input(output_details[0]['index'], (10000,
10))
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
print("Input Shape:", input_details[0]['shape'])
print("Input Type:", input_details[0]['dtype'])
print("Output Shape:", output_details[0]['shape'])
print("Output Type:", output_details[0]['dtype'])

Output –

Input Shape: [10000 28 28]
Input Type: <class 'numpy.float32'>
Output Shape: [10000 10]
Output Type: <class 'numpy.float32'>
test_images.dtype
dtype('float64')
Change the individual variable type to float64 to numpy.float64
test_imgs_numpy = np.array(test_images, dtype=np.float32)
#Set the tensor to an array and get the predictions
interpreter.set_tensor(input_details[0]['index'], test_imgs_numpy)
interpreter.invoke()
tflite_model_predictions =
interpreter.get_tensor(output_details[0]['index'])
print("Prediction results shape:", tflite_model_predictions.shape)
prediction_classes = np.argmax(tflite_model_predictions, axis=1)

Output –

Prediction results shape: (10000, 10)

#Call the accuracy_score from sklearn package and pass the prediction and test labels

(acc = accuracy_score(prediction_classes, test_labels)

#print accuracy results

print('Test accuracy TFLITE model :', acc)
Test accuracy TFLITE model : 0.8802

We got an accuracy of 88.02 which accurately matches tensorflow keras models accuracy.

CONCLUSION
We have successfully converted the tensorflow/keras model to tensorflow lite model with the exact same accuracy and smaller model file size to be used for edge devices.

Leave a Comment

Your email address will not be published. Required fields are marked *