Optimizing TensorFlow Lite Runtime Memory

The good news is that these intermediate tensors don't have to co-exist in memory thanks to data dependency analysis. This allows us to reuse the memory buffers of the intermediate tensors and reduce the total memory footprint of the inference engine. If the network has the shape of a simple chain, two large memory buffers are sufficient as they can be swapped back and forth interchangeably throughout the network. However, for arbitrary networks forming complicated graphs, this , we employ GL textures for these intermediate tensors. These come with a couple of interesting restrictions: (a) A texture's size can't be modified after its creation, and (b) only one shader program gets exclusive access to the texture object at a given time. In this Shared Memory Buffer Objects mode, the objective is to minimize the sum of the sizes of all created shared memory buffer objects in the object pool. This optimization is similar to the well-known , each using a different heuristic, but still reaching or getting very close to the