📺
blog.tensorflow.org
TensorFlow Lite has been widely adopted in many applications to provide machine learning features on edge devices such as mobile phones, microcontroller units, and Edge TPUs. Among all popular applications that make people’s life easier and more productive, Natural Language Understanding is one of the key areas that attracts much attention from both the research community and the industry. After the that encapsulate pretrained machine learning , ), transforming raw text data, and connecting the model’s inputs and outputs to generate prediction results,
- : Given an article and a user question, the model can answer the question within the article.
- . The chart below shows a comparison of the latency, size and F1 score between the models.
is a compact BERT model open sourced on and to optimize its model size and performance, so that it can utilize accelerators like GPU/DSP if available. The quantized MobileBERT is 16x smaller & 8x faster than the BERT base, with little accuracy loss. The using TensorFlow.js.
Compared with the original BERT base model (416MB), the below table shows the performance of quantized MobileBERT under the same setting. two models using projection methods, namely SGNN and PRADO.
We used . PRADO first computes trainable projected features from the sequence of word tokens, then applies convolution and attention to map features to a fixed-length encoding. By combining a projection layer, a convolutional and attention encoder mechanism, PRADO achieves similar accuracy as LSTM, but with 100x smaller model size.
The idea behind these models is to use projection to compute features from texts, so that the model does not need to maintain a big embedding table to convert text features to embeddings. In this way, we’ve proven the model will be much smaller than embedding based models, while maintaining similar performance and inference latency.Creating your own NLP Models
In addition to using pre-trained models, TensorFlow Lite also provides you with tools such as Model Maker to customize existing models for your own data.TensorFlow Lite Model Maker: Transfer Learning Toolkit for machine learning beginners
and of TensorFlow operators, you may have run into issues while converting your NLP model to TensorFlow Lite, either due to missing ops or unsupported data types (like RaggedTensor support, hash table support, and asset file handling, etc.). Here are a few tips on how to resolve the conversion issues in such cases.Run TensorFlow ops and TF.text ops in TensorFlow Lite
We have enhanced ops and RaggedTensor when training TensorFlow models, and now those models can be easily converted to TensorFlow Lite and run with necessary ops.
Furthermore, we provide the solution of using op selectively for NLP, such as Ngram, SentencePieceTokenizer, WordPieceTokenizer and WhitespaceTokenizer.
Previously, there were several restrictions blocking models with . We used the same mechanism to fuse TF.Text APIs into custom TensorFlow Lite ops, improving inference efficiency significantly. For example, the WhitespaceTokenizer API was made up of multiple ops, and took 0.9ms to run in the original graph in TensorFlow Lite. After fusing these ops into a single op, it finishes in 0.04ms, a 23x speed-up. This approach has been proven to bring a huge gain in inference latency in the SGNN model mentioned above.Hash table support
Hash table is important for many NLP models, since we usually need to utilize numeric computation in the language model by transforming words into token IDs and vice versa. Hash table will be enabled in TensorFlow Lite soon. It is supported by handling asset files natively in the TensorFlow Lite format and delivering op kernels as TensorFlow Lite built-in operators.Deployment: How to run NLP models on-device
Running inference with TensorFlow Lite is now much easier than before. You can use pre-built inference APIs to integrate your model within 5 lines of code, or use utilities to build your own Android/iOS inference APIs.Simple model deployment using TensorFlow Lite Task Library
The : classifies the input text to a set of known categories.- : answers questions based on the content of a given passage with BERT-family models.
The Task Library works cross-platform on both Android and iOS. The following example shows inference with a BertQA model in Java/Swift:JAVA// Initialization
BertQuestionAnswerer answerer = BertQuestionAnswerer.createFromFile(androidContext, modelFile);
// Answer a question
Listanswers = answerer.answer(context, question); Java code for Android SWIFT// Initialization
let mobileBertAnswerer = TFLBertQuestionAnswerer.mobilebertQuestionAnswerer(modelPath: modelPath)
// Answer a question
let answers = mobileBertAnswerer.answer(context: context, question: question)Swift code for iOS Customized Inference APIs
If your use case is not supported by the existing task libraries, you can also leverage the .Conclusion
In this article, we introduced the new support for NLP tasks in TensorFlow Lite. With the latest update of TensorFlow Lite, developers can easily create, convert and deploy NLP models on-device. We will continue providing more useful tools, and accelerate the development of on-device NLP models from research to production. We would love to hear your feedback, and suggestions for newer NLP tools and utilities. Please email .Acknowledgments
We like to thank Khanh LeViet, Arun Venkatesan, Max Gubin, Robby Neale, Terry Huang, Peter Young, Gaurav Nemade, Prabhu Kaliamoorthi, Ping Yu, Renjie Liu, Lu Wang, Xunkai Zhang, Yuqi Li, Sijia Ma, Thai Nguyen, Xingying Song, Chung-Ching Chang, Shuangfeng Li to contribute to the blogpost.↗ Original-Artikel auf blog.tensorflow.org lesenVollständiger Original-BerichtAusführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf blog.tensorflow.org.Wie bewertest du diesen Beitrag?1 Klick FeedbackTeilen mit Netzwerk & Team:Hat Ihnen dieser Tipp / Anleitung geholfen?Community-Analysen & Experten-Meinungen 0
Verfasse deine eigene Analyse, teile Workarounds oder diskutiere diesen Vorfall im Blog.Noch keine Community-Analyse verfasst. Markiere einen Textabschnitt oder klicke oben auf „ Eigene Analyse verfassen“!Community Pulse: Relevanz-Einschätzung1 Klick Experten-Votum🔴 Akute Relevanz 0%🟡 In Evaluierung 0%🟢 Keine Auswirkung 0%Spannende Innovation 0%Port 8095 EngineVerwandte Story-Cluster & Quellen (Vektor-KI)
Tipp: Mit Pfeiltasten [ ← ] und [ → ] blättern
Ähnliche Beiträge
🔍 Verwandte NewsAuch interessante Nachrichten What’s new in TensorFlow Lite for NLP
Thematisch verwandte Begriffe: Whats, TensorFlow, Lite · 6 Treffer
🔧 AI Nachrichten MacDailyNewsApple accuses OpenAI of destroying evidence as trade-secrets fight intensifies
🔧 AI Nachrichten The Mac ObserverGPT-6 Astra Release Today? OpenAI’s Next Major AI Model Is Almost Here
Laden...Videos werden geladen ...
Laden...Beiträge werden geladen ...
Laden...Videos werden geladen ...
Laden...Beiträge werden geladen ...
Laden...Videos werden geladen ...
Laden...Beiträge werden geladen ...
Laden...Videos werden geladen ...
🔖 Gespeicherte Artikel
📂 Keine gespeicherten Artikel vorhanden.📂 News ⏱️ 3 Min vor 10 MinArtikeldaten werden geladen...Zum Aktualisieren ziehen
tsecurity.de App
Offline-Lesen, Eilmeldungen & 0ms LadezeitInstalliere tsecurity.de direkt auf deinen Home-Bildschirm für das ultimative Vollbild-Magazinerlebnis ohne Browser-Leisten.
Nächster Beitrag🤖Community Radar & Live Chat
Sentinel Bot online • Live-StreamDein Cluster: Security Explorer👥 Match:lädt…Verbindung zum Community-Stream wird aufgebaut...📡 Aktivitäten deiner Analysten
lädt…💡 Neues Thema oder Eilmeldung einreichen
Reiche interessante Links, Zero-Days oder Debatten ein. Die Community entscheidet per Upvote über die Veröffentlichung.
🔥 Heiß diskutierte Einreichungen
- : answers questions based on the content of a given passage with BERT-family models.
SOCIAL SHARE CARD GENERATOR