A guest post by that brings this phenomenon to life through your browser and webcam.
In this post, I’ll walk through the specifics of the project’s technical design and implementation.
Animating the Mona Lisa with Deep Learning
Image animation is a technique that allows one to puppeteer a still image through a driving video. Using a deep-learning-based approach, I was able to generate a highly convincing animation of Mona Lisa’s gaze.
Specifically, I used the from the driving video. Diffs of these values between consecutive frames are then used as input to a network that predicts a dense
| An example of a head frame overlaid on top of the underlying image. To best illustrate the problem, the version shown here is from an earlier iteration of the project where there was further resolution loss in the head frame. |
This, however, produced its own set of challenges. If you look at the example above, you’ll notice that the lower-resolution output of the model––coupled with some subtle collateral background changes due to FOMM’s warping procedure––causes the head frame to visually jut out. In other words, it was a bit obvious that this was just a picture on top of another picture. To address this, I did some image processing in Python to “blend” the head image into the underlying one.
First, I resized the head frame to its original resolution. From there, I created a new frame using a weighted average of these blurred out pixels and the corresponding pixels in the underlying image, where the weight––or alpha––of a pixel in the head frame decreases as it moves away from the midpoint.
The function to determine alpha was adapted from a 2D sigmoid, and is expressed as:
After I applied the above procedure to all 33 frames in the animation set, the resulting superimpositions each appeared to be a single image to the unsuspecting eye:
BlazeFace is a deep-learning based object recognition model that detects human faces and facial landmarks. It is specifically trained for using mobile camera input. This worked well for my use case, as I expected most viewers to be using their webcam in a similar manner––with their heads in frame, front-facing, and fairly close to the camera––whether through their mobile devices or on their laptops.
My foremost consideration in selecting this model, however, was its extraordinary speed of detection. To make this project convincing, I needed to be able to run the entire animation in real time, including the facial recognition step. BlazeFace adapts the single-shot detection (SSD) model, a deep-learning-based object detection algorithm that simultaneously proposes bounding boxes and detects objects in just one forward pass of the network. BlazeFace’s lightweight detector is capable of recognizing facial landmarks at speeds as fast as 200 frames per second.
AcknowledgementsThanks to Andrew Fu for reading this post and providing me feedback, to Nick Platt for lending his ear and thoughts on a frontend bug, and to Jason Mayes along with the rest of the team at Google for their work in reaching out and amplifying this project. Vollständiger Original-Bericht Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf blog.tensorflow.org. Wie bewertest du diesen Beitrag? 1 Klick Feedback Teilen mit Netzwerk & Team: Hat Ihnen dieser Tipp / Anleitung geholfen? Community-Analysen & Experten-Meinungen 0Verfasse deine eigene Analyse, teile Workarounds oder diskutiere diesen Vorfall im Blog. Noch keine Community-Analyse verfasst. Markiere einen Textabschnitt oder klicke oben auf „ Eigene Analyse verfassen“! Community Pulse: Relevanz-Einschätzung 1 Klick Experten-Votum 🔴 Akute Relevanz 0% 🟡 In Evaluierung 0% 🟢 Keine Auswirkung 0% Spannende Innovation 0% Verwandte Story-Cluster & Quellen (Vektor-KI) Tipp: Mit Pfeiltasten [ ← ] und [ → ] blättern 
Ähnliche Beiträge
🔍 Verwandte News
Auch interessante Nachrichten Bringing the Mona Lisa Effect to Life with TensorFlow.jsThematisch verwandte Begriffe: Bringing, Mona, Lisa, Effect · 6 Treffer 🔧 AI Nachrichten MacDailyNews Apple accuses OpenAI of destroying evidence as trade-secrets fight intensifies 🔧 AI Nachrichten The Mac Observer GPT-6 Astra Release Today? OpenAI’s Next Major AI Model Is Almost Here
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
Laden...
Beiträge werden geladen ...
Laden...
Videos werden geladen ...
🔖 Gespeicherte Artikel
📂
Keine gespeicherten Artikel vorhanden.
📂 News
⏱️ 3 Min
vor 10 Min
Artikeldaten werden geladen...
tsecurity.de AppOffline-Lesen, Eilmeldungen & 0ms Ladezeit
Installiere tsecurity.de direkt auf deinen Home-Bildschirm für das ultimative Vollbild-Magazinerlebnis ohne Browser-Leisten.
Nächster Beitrag
🤖
Community Radar & Live Chat
Sentinel Bot online • Live-Stream
Dein Cluster:
Security Explorer
👥 Match:
lädt…
Verbindung zum Community-Stream wird aufgebaut...
📡 Aktivitäten deiner Analystenlädt…
💡 Neues Thema oder Eilmeldung einreichenReiche interessante Links, Zero-Days oder Debatten ein. Die Community entscheidet per Upvote über die Veröffentlichung. 🔥 Heiß diskutierte Einreichungen |
SOCIAL SHARE CARD GENERATOR