Posted by , Google Research and (pronounced “movie nets”), a family of new mobile-optimized model architectures for video classification. The models are trained on the different human actions (such as playing trumpet, robot dancing, bowling, and more) and can classify video streams captured on a modern smartphone in real time. You can download the pre-trained TensorFlow Lite models from and and the
MoViNets are a family of convolutional neural networks which efficiently process video streams, outputting accurate predictions with a fraction of the latency of convolutional video classifiers like .
Frame-based classifiers output predictions on each 2D frame independently, resulting in sub-optimal performance due to their lack of temporal reasoning. On the other hand, 3D video classifiers offer high accuracy predictions by processing all frames in a video clip simultaneously, at a cost of significant memory and latency penalties as the number of input frames increases. MoViNets offer key advantages from both 2D frame-based classifiers and 3D video classifiers while mitigating their disadvantages.
The following figure shows a typical approach to using 3D networks with multi-clip evaluation, where the predictions of multiple overlapping subclips are averaged together. Shorter subclips result in lower latency, but reduce the overall accuracy.
 |
| Diagram illustrating Multi-Clip Evaluation for 3D Video Networks |
MoViNets take a hybrid approach, which proposes the use of by searching for efficient configurations of models on video datasets (specifically Kinetics 600) across network width, depth, and resolution.
The result is a set of action classifiers that can output temporally-stable predictions that smoothly transition based on frame content. Below is an example plot of MoViNet-A2 making predictions on each frame on a video clip of skateboarding. Notice how the initial scene with a small amount of motion has relatively constant predictions, while the next scene with much larger motion causes a dramatic shift in predicted classes.
 |
| A video plotting the top-5 predictions of MoViNet-A2 over time on an example 8-second (25 fps) skateboarding video clip. Create your own plots with , , which represent the smaller models that can feasibly run in real time (20 fps or higher). To effectively quantize MoViNet, we adapt a few modifications to the model architecture - the , and and use ) to reduce the model sizes and make them run faster on mobile CPUs. The integer-based post-training quantization process further introduces 2-3 p.p. accuracy loss. Compared to the original MoViNets, quantized MoViNets lag behind in accuracy on full 10-second Kinetics 600 clips (5-7 p.p. accuracy reduction in total), but in practice they are able to provide very accurate predictions on daily human actions, e.g., push ups, dancing, and playing piano. In the future, we plan to train with in the TensorFlow Model Garden. The provided Colab notebook provides specific steps on how to fine-tune a pretrained video classifier on another dataset. Future Steps We are excited to see on-device online video action recognition powered by MoViNets, which demonstrate highly efficient performance. In the future, we plan to support quantize-aware training for MoViNets to mitigate the quantization accuracy loss. We also are interested in extending MoViNets as the backbone for more on-device video tasks, e.g. video object detection, video object segmentation, visual tracking, pose estimation, and more. Acknowledgement We would like to extend a big thanks to Yeqing Li for supporting MoViNets in TensorFlow Model Garden, Boqing Gong, Huisheng Wang, and Ting Liu for project guidance, Lu Wang for code reviews, and the TensorFlow Hub team for hosting our models. Vollständiger Original-Bericht Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf blog.tensorflow.org. Wie bewertest du diesen Beitrag? 1 Klick Feedback Teilen mit Netzwerk & Team: Hat Ihnen dieser Tipp / Anleitung geholfen? Community-Analysen & Experten-Meinungen 0Verfasse deine eigene Analyse, teile Workarounds oder diskutiere diesen Vorfall im Blog. Noch keine Community-Analyse verfasst. Markiere einen Textabschnitt oder klicke oben auf „ Eigene Analyse verfassen“! Community Pulse: Relevanz-Einschätzung 1 Klick Experten-Votum 🔴 Akute Relevanz 0% 🟡 In Evaluierung 0% 🟢 Keine Auswirkung 0% Spannende Innovation 0% Verwandte Story-Cluster & Quellen (Vektor-KI) Tipp: Mit Pfeiltasten [ ← ] und [ → ] blättern 
Ähnliche Beiträge
🔍 Verwandte News
Auch interessante Nachrichten Video Classification on Edge Devices with TensorFlow Lite and MoViNetThematisch verwandte Begriffe: Video, Classification, Edge, Devices · 6 Treffer 🔧 AI Nachrichten MacDailyNews Apple accuses OpenAI of destroying evidence as trade-secrets fight intensifies 🔧 AI Nachrichten The Mac Observer GPT-6 Astra Release Today? OpenAI’s Next Major AI Model Is Almost Here
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
🔖 Gespeicherte Artikel
📂
Keine gespeicherten Artikel vorhanden.
📂 News
⏱️ 3 Min
vor 10 Min
Artikeldaten werden geladen...
tsecurity.de AppOffline-Lesen, Eilmeldungen & 0ms Ladezeit
Installiere tsecurity.de direkt auf deinen Home-Bildschirm für das ultimative Vollbild-Magazinerlebnis ohne Browser-Leisten.
Nächster Beitrag
🤖
Community Radar & Live Chat
Sentinel Bot online • Live-Stream
Dein Cluster:
Security Explorer
👥 Match:
lädt…
Verbindung zum Community-Stream wird aufgebaut...
📡 Aktivitäten deiner Analystenlädt…
💡 Neues Thema oder Eilmeldung einreichenReiche interessante Links, Zero-Days oder Debatten ein. Die Community entscheidet per Upvote über die Veröffentlichung. 🔥 Heiß diskutierte Einreichungen |
SOCIAL SHARE CARD GENERATOR