Accelerating TensorFlow Lite Micro on Cadence Audio Digital Signal Processors

Posted by Raj Pawate (Cadence) and Advait Jain (Google)

However, productization paths for machine learning on DSPs can often be ad-hoc. In contrast, speech, audio, and video codecs have worldwide standards bodies such as ITU and 3GPP creating algorithms for compression and decompression addressing several aspects of quality measurement, fixed point arithmetic considerations and interoperability.

TensorFlow Lite Micro (TFLM) is a generic open-sourced inference framework that runs machine learning models on embedded targets, including DSPs. Similarly, .

Performance Improvements

Most recently, we have collaborated on adding optimizations for models that are , an example noise suppression neural net.

The following table shows the improvement with the optimized kernels relative to the reference implementations as measured with the Xtensa instruction set simulation tool.

How to use these optimizations

All of the code can be used from the of an LSTM-based key-word recognizer. We expect to expand on this and continue to bring optimized and production-ready implementations of the latest developments in AI/ML to Tensilica Xtensa DSPs.

Acknowledgements

We would like to acknowledge a number of our colleagues who have contributed to making this collaboration successful.

Cadence: Int16 optimizations: Manjunath CP, Bhanu Prakash Venkata, Anirban Mandal LSTM implementation: Niranjan Yadla, Lukman Rahumathulla, Manjunath CP, Pramodkumar Surana, Arjun Medinakere NNLIB optimizations: Vijay Pawar, Prasad Nikam, Harshavardhan, Mayur Jagtap, Raj Pawate

Google: Advait Jain, Deqiang Chen, Lawrence Chan, Marco Tagliasacchi, Nat Jeffries, Nick Kreeger, Pete Warden, Rocky Rhodes, Ting Yan, Yunpeng Li, Victor Ungureanu