Posted by Jeremy Wortz, ML specialist, Google Cloud & Jordan Totten, Machine Learning Specialist
In a previous , (2) matrix factorization from . In this blog, we dive deep into option (3) and demonstrate how to build a playlist recommendation system by implementing an end-to-end candidate retrieval workflow from scratch with Vertex AI. Specifically, we will cover:
The evolution of retrieval modeling and why two-tower encoders are popular for deep retrieval tasks
Framing a playlist-continuation use-case using the (TFRS) library
Serving candidate embeddings in an approximate nearest neighbors (ANN) index with .
Background
To meet low latency serving requirements, large-scale recommenders are often deployed to production as such that semantically similar entities cluster closer together. This means, if we compute the between the two embedding vectors determines how close (similar) the candidate is to the query. Source: , Google Researchers address this speed-accuracy tradeoff with a novel compression algorithm that, compared to ), it can still be challenging to implement, tune, and scale. To help teams take advantage of this technology without the operational overhead, Google Cloud offers these capabilities (and more) as a managed service with by capturing the similarity between query, candidate pairs and mapping them to a shared embedding space. One of the major benefits to this .
In a two-tower architecture, each tower is a neural network that processes either query or candidate input features to produce an embedding representation of those features. Because the embedding representations are simply vectors of the same length, we can compute the
Teams can analyze the impact of modeling decisions by listening to retrieved candidate tracks (e.g., generate recommendations for your own Spotify playlists)
Training examples
Creating training examples for recommendation systems is a non-trivial task. Like any ML use case, training data should accurately represent the underlying problem we are trying to solve. Failure to do this can lead to poor model performance and unintended consequences for the user experience. One such lesson from the . Each tower is built separately as a callable to process input feature values, pass them through feature layers, and concatenate the results. This means the tower is simply producing one concatenated vector (i.e., the representation of the query or candidate; whatever the tower represents).
First, we define the basic structure of a tower and implement it as a subclassed Keras model:
classPlaylist_Tower(tf.keras.Model): '''
produced embedding represents the features
of a Playlist known at query time
''' def__init__(self, layer_sizes, vocab_dict): super().__init__()
# TODO: build sequential model for each feature here
defcall(self, data): '''
defines what happens when the model is called
'''
all_embs = tf.concat(
[ # TODO: concatenate output of all features defined above
], axis=1)
# pass output to dense/cross layers if self._cross_layer isnotNone:
cross_embs = self._cross_layer(all_embs) return self.dense_layers(cross_embs) else: return self.dense_layers(all_embs)
We further define the subclassed towers by creating Keras sequential models for each feature being processed by that tower:
Because the features represented in the playlist’s STRUCT are sequence features (lists), we need to reshape the embedding layer output and use 2D pooling (as opposed to the 1D pooling applied for non-sequence features):
Once both towers are built, we use the TFRS base model class ( guide for more details.
Feature engineering
As the factorization-based models offer a pure collaborative filtering approach, the advanced feature processing with NDR architectures allow us to extend this to also incorporate aspects of for more details.
TextVectorization() layers
The key to text features is to understand if creating additional NLP features with the , but fundamentally their approach seeks to compress the candidate vectors such that the original distances between vectors are preserved. Compared to previous solutions, this results in a more accurate relative ranking of a vector and its nearest neighbors, i.e., it minimizes distorting the vector similarities our model learned from the training data.
Fully managed vector database and ANN service
Matching Engine is a managed solution utilizing these techniques for efficient vector similarity search. It offers customers a highly scalable vector database and ANN service while alleviating the operational overhead of developing and maintaining similar solutions, such as the open sourced for more details)
Dynamic rebuilds: when an index grows beyond its original configuration, Matching Engine periodically re-organizes the index and serving structure to ensure optimal performance
Autoscaling: underlying infrastructure is autoscaled to ensure consistent performance at scale
Filtering and diversity: ability to include multiple restrict and crowding tags per vector. At query inference time, use boolean predicates to filter and diversify retrieved candidates (see strategy to build a distributed implementation of our candidate index. It combines two algorithms:
Distributed search tree for hierarchically organizing the embedding space. Each level of this tree is a clustering of the nodes at the next level down, where the final leaf-level is a clustering of our candidate embedding vectors
Asymmetric hashing (AH) for fast dot product approximation algorithm used to score similarity between a query vector and the search tree nodes
Figure 10: conceptual representation of the partitioned candidate vector dataset. During query inference, all partition centroids are scored. In the centroids most similar to the query vector, all candidate vectors are scored. The scored candidate vectors are aggregated and re-scored, returning the top N candidate vectors.
This strategy shards our embedding vectors into partitions, where each partition is represented by the centroid of the vectors it contains. The aggregate of these partition centroids form a smaller dataset summarizing the larger, distributed vector dataset. At inference time, Matching Engine scores all the partitioned centroids, then scores the vectors within the partitions whose centroids are most similar to the query vector.
Conclusion
In this blog we took a deep dive into understanding critical components of a candidate retrieval workflow using . We took a closer look at the foundational concepts of two-tower architectures, explored the semantics of query and candidate entities, and discussed how things like the structure of training examples can impact the success of candidate retrieval.
In a subsequent post we will demonstrate how to use Vertex AI and other Google Cloud services to implement these techniques at scale. We’ll show how to leverage BigQuery and Dataflow to structure training examples and convert them to TFRecords for model training. We’ll outline how to structure a Python application for training two-tower models with the Vertex AI Training service. And we’ll detail the steps for operationalizing the trained towers.
Vollständiger Original-Bericht
Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf blog.tensorflow.org.
SOCIAL SHARE CARD GENERATOR