How CEVA uses TensorFlow Lite for Always-On Speech Recognition on the Edge

$ python3 -m tensorflow_docs.tools.nbfmt [options] notebook.ipynb

```
converter = tf.lite.TFLiteConverter.from_keras_model(keras_model)
converter.experimental_new_converter = True
tflite_model = converter.convert()
open("converted_to_tflite_model.tflite", "wb").write(tflite_model)
```

Used :

$ python3 -m tensorflow_docs.tools.nbfmt [options] notebook.ipynb
  

```
$> xxd –I model.tflite > model.cc
```

Here we found that some of the model layers (for example, GRU) were not properly supported (at the time) by TFLM. It is very reasonable to assume that, as TFLM continues to mature and Google and the TFLM community invest more in it, issues like this will become rarer. 
In our case, though, we opted to re-implement the GRU layers in terms of Fully Connected layers, which was surprisingly easy. 

Integration

The next step was to integrate the TFLM runtime library and the converted model into our existing embedded C frontend, which handles audio preprocessing and feature extraction.

Even though our frontend was not written with TFLM in mind, it was modular enough to allow easy integration by implementation of a single simple wrapper function, as follows:

1. Linked the TFLM runtime library into our embedded C application (WhisPro frontend)
2. Implemented a wrapper-over-setup function for mapping the model into a usable data structure, allocating the interpreter and tensors
3. Implemented a wrapper-over-execute function for mapping data passed from the WhisPro frontend into tflite tensors used by the actual execute function
4. Replaced the call to the original model execute function with a call to the TFLM implementation

Process Visualization

The process we described is performed by two components:

• The microcontroller supplier, in this case, CEVA – is responsible for optimizing TFLM for its hardware architecture.
• The microcontroller user, in this case, CEVA WhisPro developer – is responsible for deploying a neural network based model, using an optimized TFLM runtime library, on the target microcontroller.