📚 Measuring Java 11 Lambda cold starts with SnapStart - Part 6 priming the request invocation

Introduction

In the 5th part of the series we discussed a huge impact of priming for our scenario. During priming we invoke the DynamoDB Client getItem method which forced Jackson Marshallers to initialize which is quite expensive one time operation for the life cycle of the Lambda function. Only because of this optimization we observed a huge decrease (up to 900 miliseconds) of the cold start times for all scenarios. But this made me think: can we optimize it any further? In case of using the frameworks like Micronaut, Quarkus and especially Spring Boot we still observed bigger cold start times (especially at p90s) comparing to using the pure Lambda solution. This is because of the translation layer (aka proxy) between the programming model of the used framework and Lambda itself. In case of Spring Boot reflection adds on cold start additionally. So I wanted to figure out if I can use priming to make the faked request invocation and preload and prewarm things. So let's explore it.

Priming the request invocation

1) Mirconaut

Mirconaut uses io.micronaut.function.aws.proxy.MicronautLambdaHandler to proxy the incoming request and uses com.amazonaws.serverless.proxy.model.AwsProxyRequest (from the artefact aws-serverless-java-container-core ) as input.
So let's construct mocked AwsProxyRequest so that the request to "/products/0" will be processed by the method

  @Get("/products/{id}")
  public Optional<Product> getProductById(@PathVariable String id) 

of the GetProductByIdController . I took me a while to figure out the minimal information to be passed, as it's a (faked) internal invocation without authorization, header and other metadata required. I came up with the following solution :

final AwsProxyRequest awsProxyRequest = new AwsProxyRequest ();
awsProxyRequest.setHttpMethod("GET");
awsProxyRequest.setPath("/products/0");
awsProxyRequest.setResource("/products/{id}");
awsProxyRequest.setPathParameters(Map.of("id","0"));
final AwsProxyRequestContext awsProxyRequestContext = new AwsProxyRequestContext();
final ApiGatewayRequestIdentity apiGatewayRequestIdentity= new ApiGatewayRequestIdentity();
apiGatewayRequestIdentity.setApiKey("blabla");
awsProxyRequestContext.setIdentity(apiGatewayRequestIdentity);
awsProxyRequest.setRequestContext(awsProxyRequestContext);

We'll also use com.amazonaws.serverless.proxy.internal.testutils.MockLambdaContext from the same artefact aws-serverless-java-container-core to mock the com.amazonaws.services.lambda.runtime.Context.

So let's use this in priming

 @Override
    public void beforeCheckpoint(Context<? extends Resource> context) throws Exception {
    try (MicronautLambdaHandler micronautLambdaHandler = new MicronautLambdaHandler()) {        

micronautLambdaHandler.handleRequest(getAwsProxyRequest(), new MockLambdaContext());
  }

The getProductById method of the GetProductByIdController class will subsequently invoke the getItem on the DynamoDB Client with product id 0 itself, so we'll get the accumulated effect of priming. Before presenting the results, let's explore how to do the same with Quarkus.

2) Quarkus

Quarkus uses io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler to proxy the incoming request and uses java.io.InputStream. I decided to go the same way as with Micronaut and used com.amazonaws.serverless.proxy.model.AwsProxyRequest (from the artefact aws-serverless-java-container-core ) to construct exact the same input. I then converted the AwsProxyRequest object to the byte array using Jackson

    ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
    return ow.writeValueAsBytes(getAwsProxyRequest());

and then proxied the request during priming like this

@Override
    public void beforeCheckpoint(org.crac.Context<? extends Resource> context) throws Exception {
    new QuarkusStreamHandler().handleRequest
    (new ByteArrayInputStream(convertAwsProxRequestToJsonBytes()), new ByteArrayOutputStream(), new MockLambdaContext());
    }

This will proxy the request "/products/0" to the handleRequest method of the GetProductByIdHandler.
It will be nice if Quarkus will directly support AwsProxyRequest instead of InputStream in QuarkusStreamHandler in the future, which will make our code a bit cleaner. Before presenting the results, let's explore how to do the same with Spring Boot.

3) Spring Boot

Conceptually this works the same as with Micronaut. We construct exactly the same AwsProxyRequest input and use already created SpringBootLambdaContainerHandler handler to proxy the stream. SpringBootLambdaContainerHandler already supports proxying the AwsProxyRequest directly as one of the offered options. So priming looks like this

@Override
    public void beforeCheckpoint(org.crac.Context<? extends Resource> context) throws Exception {
         handler.proxy(getAwsProxyRequest(), new MockLambdaContext());
    }

This will proxy the request "/products/0" to ProductController's method

    @RequestMapping(path = "/products/{id}", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
    public Optional<Product> getProductById(@PathVariable("id") String id)

Now let's compare the cold start times due to effect of priming of DynamoDB getItem invocation (prefix d in the column name) described in the 5th part and the whole (faked) request invocation through the proxy including subsequent DynamoDB getItem invocation (prefix a) described in this article. Please note that this optimization doesn't have any effect on the pure Lambda example without using any framework.

We see a very big effect of this optimization especially for Micronaut and Spring Boot frameworks. The cold start times of all 3 frameworks are now really much closer to the pure Lambda ones.

Measuring end to end AWS API Gateway latency

Now it's time to re-measure APIGateway end to end request latencies in case of cold starts from the previous article. We'll use the same prefixes as in the previous table. Please note that this optimization doesn't have any effect on the pure Lambda example without using any framework.

We also observe a very big improvement here especially for Spring Boot, but also for Micronaut and Quarkus frameworks at various percentiles.

Conclusions

With priming of the invocation of the entire request we could achieve further significant reduction of the cold start times using all 3 frameworks Micronaut, Quarkus and Spring Boot for our scenario. The measure cold start times became much closer to the ones of the pure Lambda function. Of course end to end APIGateway request latency also reduced. We required to write additional code for that, but the already existing AwsProxyRequest class made our life a bit easier, as we had to set only small amount of properties to make if work. Maybe adding some additional utilities provided out of the box for this purpose can reduce our amount of work further. Anyway we have to understand the internals of frameworks used and whether there is an optimization potential through the whole invocation chain. This optimization works the same way for all downstream services from AWS or not, that you invoke in your Lambda implementation through abstractions provided by Micronaut, Quarkus and Spring Boot frameworks.

Is this the end state of what we can optimize for the Serverless architectures like API Gateway -> SnapStart enabled Lambda written in Java (optionally using frameworks) (-> DynamoDB)?

Maybe not. I'll have to think about other optimization ideas and try them out. Stay tuned!