.

From the beginning, we've provided APIs in Python and C++ that enable developers to take advantage of the Edge TPU's local inference speed. Offline processing for machine learning models allows for considerable savings on bandwidth and cloud compute costs, it keeps data local, and it preserves user privacy. More recently, we've been hard at work to refactor our APIs and make them more modular, reusable and performant, while at the same time eliminating unnecessary API abstractions and surfacing more of the native TensorFlow Lite APIs that developers are familiar with.

So in our latest release, we're now offering two separate reusable libraries, each built upon the powerful TensorFlow Lite APIs and each isolated in their own repositories: for Python.

libcoral (C++)

Unlike some of our previous APIs, libcoral doesn't hide tflite::Interpreter. Instead, we're making this native TensorFlow Lite class a first-class component and offering some additional helper APIs that simplify some of your code when working with common models such as classification and detection.

With our new libcoral library, developers should typically follow the pattern below to perform an inference in C++:

1. Create tflite::Interpreter instance with the Edge TPU context and allocate memory.

   To simplify this step, libcoral provides the MakeEdgeTpuInterpreter() function:

   C++ Kopieren

    
   // Load the model
   auto model = coral::LoadModelOrDie(absl::GetFlag(FLAGS_model_path));
   
   // Get the Edge TPU context
   auto tpu_context = coral::ContainsEdgeTpuCustomOp(*model) ?
    	coral::GetEdgeTpuContextOrDie() :
    	nullptr;
   
   // Get the interpreter
   auto interpreter = coral::MakeEdgeTpuInterpreterOrDie(
    	*model,
    	tpu_context.get()); 

2. Configure the interpreter's input.
3. Invoke the interpreter:
C++ Kopieren

interpreter->Invoke();

As an alternative to Invoke(), you can achieve higher performance with the InvokeWithMemBuffer() and InvokeWithDmaBuffer() functions, which enable processing the input data without copying from another region of memory or from a DMA file descriptor, respectively.

4. Process the interpreter's output.

To simplify this step, libcoral provides some adapters, requiring less code from you:

auto result = coral::GetClassificationResults(
 	*interpreter,
 	/* threshold= */0.0f,
 	/*top_k=*/3);

The above is an example of the classification adapter, where developers can specify the minimum confidence threshold, as well as the maximum number of results to return. The API also features a detection adapter with its own result filtering parameters.

For a full view of the example application source code, see .

This new release also brings updates to on-device retraining with the decoupling of imprinting functions from inference on the updated ImprintingEngine. The new design makes the imprinting engine work with the tflite::Interpreter directly.

To easily address the Edge TPUs available on the host, libcoral supports labels such as "usb:0" or "pci:1". This should make it easier to manage resources on multi-Edge TPU systems.

Finally, we've made a number of performance improvements such as more efficient memory usage and memory-based instead of file-based abstractions. Also, the design of the API is more consistent by leveraging the .

Updates to the Coral model garden

With this release, we're further expanding the Coral model garden with MobileDet. MobileDets refer to a family of lightweight, single-shot detectors using the TensorFlow Object Detection API that achieve state-of-the-art accuracy-latency tradeoff on Edge TPUs. It is a lower-latency detection model that offers better accuracy, compared to the MobileNet family of models.

Check out the full and