How LinkedIn Personalized Performance for Millions of Members using TensorFlow.js

To do this, we trained a deep neural network to identify if a request to LinkedIn would result in a fast page load in real time. Based on the performance quality result predicted by this model we change the resolution of all images on a given user’s news feed before the resulting webpage was sent to the client. This led to an increase in the magnitude of billions for extra Feed Viral Actions (+0.23%) taken, millions more Engaged Feed Users (+0.16%) and Sponsored Revenue increased significantly for us too (+0.76%).

Image Quality Comparison: Image on the left uses 4x more memory than the one on the right which is less than ideal to send to users on slow network connections or when the device may be low on resources. Prior to using an ML model, we only showed the low resolution image which was not great for users that had capacity for higher quality images on newer devices.

We described in great detail why many of our performance optimization experiments failed back in 2017 and how we used those learnings to build a Performance Quality Model (PQM) in our data samples from around the world free to use for your own website performance optimizations! infrastructure which in turn routes the request to our performance prediction resource. To handle the request, the PaaS resource performs some feature generation based on the inputs and then makes an RPC call out to the Node.js process for the prediction.

The N Node.js processes are long-lived. They are started upon JVM startup and have already loaded the desired model using tf.node.loadSavedModel(). When a process receives a request for a prediction, it simply takes the input features, calls tf_model.predict(), and returns the result. Here is a simplified version of the Node.js code:

const tf = require(‘@tensorflow/tfjs-node’);

async function main() {
  // load the model when the process starts so it’s always ready
  const model = await tf.node.loadSavedModel(‘model_dir’);

  function predict(rawInput) {
    return tf.tidy(() => {
      // prepare the inputs as tensor arrays
      const x = {}
      for (const feature of Object.keys(predictionInput)) {
        x[feature] = tf.tensor([input[feature]], [1, 1]);
      }

      const output = model.predict(x, {});
      const probs = Array.from(output.probabilities.dataSync());
      const classes = Array.from(output.all_class_ids.dataSync());
      const result = Object.fromEntries(classes.map((classId, i) => [classId, probs[i]]));
      return result; // {0: 0.8, 1: 0.15, 2: 0.05} probability of each performance quality
    });
  }

  // Register our ‘predict’ RPC handler (pseudo-code)
  // process is an abstraction of the Node.js side of the communication channel
  // with the JVM
  process.registerHandler(‘predict’, input => {    
    const result = predict(input);
    return Promise.resolve(result);
  });
}

main();

to find out how we tested the model in Python and Node.js.

Looking to the future

Our current unique architecture does have some areas for improvement. Probably the biggest opportunity is to address the uniqueness of this multi stack architecture itself. The mix of both Java and Node.js technologies adds additional cognitive overhead and complexity during design, development, debugging, operations, maintenance - however as previously stated you could move the whole stack to Node to simplify matters, so this is a solvable problem.

Another potential area for improvement comes from currently using a single threaded architecture on the Node.js side. Because of this, only a single prediction currently occurs at a time so latency sometimes includes some amount of queueing time. This can potentially be worked around by using Node and and (Google) and Jason Mayes (Google) for their continued support and feedback.